天天看點
傳回

Java GenericObjectPool 對象池化技術-SpringBoot sftp 連接配接池工具類

作者：湯圓說Java

通常一個對象建立、銷毀非常耗時的時候，我們不會頻繁的建立和銷毀它，而是考慮複用。複用對象的一種做法就是對象池，将建立好的對象放入池中維護起來，下次再用的時候直接拿池中已經建立好的對象繼續用，這就是池化的思想。

Apache Commons Pool是一個對象池的架構，他提供了一整套用于實作對象池化的API。它提供了三種對象池：GenericKeyedObjectPool，SoftReferenceObjectPool和GenericObjectPool，其中GenericObjectPool是我們最常用的對象池，内部實作也最複雜。

GenericObjectPool

GenericObjectPool 是一個通用對象池架構，我們可以借助它實作一個健壯的對象池，UML圖如下所示：

Java GenericObjectPool 對象池化技術-SpringBoot sftp 連接配接池工具類

GenericObjectPool 實作了ObjectPool接口，而ObjectPool就是對象池的核心接口，它定義了一個對象池應該實作的行為。

public interface ObjectPool<T> extends Closeable {
    /**
     * 從池中借走到一個對象
     */
    T borrowObject() throws Exception, NoSuchElementException, IllegalStateException;
    /**
     * 把對象歸還給對象池
     */
    void returnObject(T var1) throws Exception;
    /**
     * 驗證對象的有效性
     */
    void invalidateObject(T var1) throws Exception;

    /**
     * 往池中添加一個對象
     */
    void addObject() throws Exception, IllegalStateException, UnsupportedOperationException;
    /**
     * 傳回對象池中有多少對象是空閑的，也就是能夠被借走的對象的數量。
     */
    int getNumIdle();
    /**
     * 傳回對象池中有對象對象是活躍的，也就是已經被借走的，在使用中的對象的數量。
     */
    int getNumActive();
    /**
     * 清理對象池。注意是清理不是清空，該方法要求的是，清理所有空閑對象，釋放相關資源。
     */
    void clear() throws Exception, UnsupportedOperationException;
    /**
     * 關閉對象池。這個方法可以達到清空的效果，清理所有對象以及相關資源。
     */
    void close();
}

BasePooledObjectFactory

Java BasePooledObjectFactory 對象池化技術

使用GenericObjectPool隻需要建立一個對象工廠類，繼承BasePooledObjectFactory并重寫它的create()和destroyObject()。

如下文中的：SftpPool.java

public interface PooledObjectFactory<T> {
    /**
     * 建立一個可由池提供服務的執行個體，并将其封裝在由池管理的PooledObject中。
     */
    PooledObject<T> makeObject() throws Exception;

    /**
     *  銷毀池不再需要的執行個體
     */
    void destroyObject(PooledObject<T> var1) throws Exception;

    /**
     * 確定執行個體可以安全地由池傳回
     */
    boolean validateObject(PooledObject<T> var1);

    /**
     * 重新初始化池傳回的執行個體
     */
    void activateObject(PooledObject<T> var1) throws Exception;

    /**
     * 取消初始化要傳回到空閑對象池的執行個體
     */
    void passivateObject(PooledObject<T> var1) throws Exception;
}

配置類GenericObjectPoolConfig

GenericObjectPoolConfig是封裝GenericObject池配置的簡單“結構”，此類不是線程安全的；它僅用于提供建立池時使用的屬性。大多數情況，可以使用GenericObjectPoolConfig提供的預設參數就可以滿足日常的需求。

工作原理流程

  1. 構造方法

    當我們執行構造方法時，主要工作就是建立了一個存儲對象的LinkedList類型容器，也就是概念意義上的“池”

  2. 從對象池中擷取對象

    擷取池中的對象是通過borrowObject()指令，源碼比較複雜，簡單而言就是去LinkedList中擷取一個對象，如果不存在的話，要調用構造方法中第一個參數Factory工廠類的makeObject()方法去建立一個對象再擷取，擷取到對象後要調用validateObject方法判斷該對象是否是可用的，如果是可用的才拿去使用。LinkedList容器減一

  3. 歸還對象到線程池

    簡單而言就是先調用validateObject方法判斷該對象是否是可用的，如果可用則歸還到池中，LinkedList容器加一，如果是不可以的則調用destroyObject方法進行銷毀

上面三步就是最簡單的流程，由于取和還的流程步驟都在borrowObject和returnObject方法中固定的，是以我們隻要重寫Factory工廠類的makeObject()和validateObject以及destroyObject方法即可實作最簡單的池的管理控制，通過構造方法傳入該Factory工廠類對象則可以建立最簡單的對象池管理類。這算是比較好的解耦設計模式，借和還的流程如下圖所示：

Java GenericObjectPool 對象池化技術-SpringBoot sftp 連接配接池工具類

使用Demo 

<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-pool2</artifactId>
    <version>2.7.0</version>
</dependency>

<!-- https://mvnrepository.com/artifact/com.jcraft/jsch -->
<dependency>
    <groupId>com.jcraft</groupId>
    <artifactId>jsch</artifactId>
    <version>0.1.55</version>
</dependency>

點選檢視代碼

application.yaml

server:
  port: 8088
  application:
    name: sftp Demo


sftp:
  host: 172.16.3.88 # 伺服器ip
  port: 22 # ssh端口
  username: root # 使用者名
  password: root # 密碼
  # 連接配接池參數
  pool:
    max-total: 10
    max-idle: 10
    min-idle: 5

SftpPoolException.java

package com.vipsoft.sftp.exception;


/**
 * sftp連接配接池異常
 */
public class SftpPoolException extends RuntimeException {

    private static final long serialVersionUID = 1L;

    /**
     * Constructs a new runtime exception with {@code null} as its
     * detail message.  The cause is not initialized, and may subsequently be
     * initialized by a call to {@link #initCause}.
     */
    public SftpPoolException() {
    }

    /**
     * Constructs a new runtime exception with the specified detail message.
     * The cause is not initialized, and may subsequently be initialized by a
     * call to {@link #initCause}.
     *
     * @param message the detail message. The detail message is saved for
     *                later retrieval by the {@link #getMessage()} method.
     */
    public SftpPoolException(String message) {
        super(message);
    }

    /**
     * Constructs a new runtime exception with the specified detail message and
     * cause.  <p>Note that the detail message associated with
     * {@code cause} is <i>not</i> automatically incorporated in
     * this runtime exception's detail message.
     *
     * @param message the detail message (which is saved for later retrieval
     *                by the {@link #getMessage()} method).
     * @param cause   the cause (which is saved for later retrieval by the
     *                {@link #getCause()} method).  (A <tt>null</tt> value is
     *                permitted, and indicates that the cause is nonexistent or
     *                unknown.)
     * @since 1.4
     */
    public SftpPoolException(String message, Throwable cause) {
        super(message, cause);
    }

    /**
     * Constructs a new runtime exception with the specified cause and a
     * detail message of <tt>(cause==null ? null : cause.toString())</tt>
     * (which typically contains the class and detail message of
     * <tt>cause</tt>).  This constructor is useful for runtime exceptions
     * that are little more than wrappers for other throwables.
     *
     * @param cause the cause (which is saved for later retrieval by the
     *              {@link #getCause()} method).  (A <tt>null</tt> value is
     *              permitted, and indicates that the cause is nonexistent or
     *              unknown.)
     * @since 1.4
     */
    public SftpPoolException(Throwable cause) {
        super(cause);
    }

    /**
     * Constructs a new runtime exception with the specified detail
     * message, cause, suppression enabled or disabled, and writable
     * stack trace enabled or disabled.
     *
     * @param message            the detail message.
     * @param cause              the cause.  (A {@code null} value is permitted,
     *                           and indicates that the cause is nonexistent or unknown.)
     * @param enableSuppression  whether or not suppression is enabled
     *                           or disabled
     * @param writableStackTrace whether or not the stack trace should
     *                           be writable
     * @since 1.7
     */
    public SftpPoolException(String message, Throwable cause, boolean enableSuppression, boolean writableStackTrace) {
        super(message, cause, enableSuppression, writableStackTrace);
    }

}

config

SftpConfig.java

package com.vipsoft.sftp.config;

import com.vipsoft.sftp.pool.SftpFactory;
import com.vipsoft.sftp.pool.SftpPool;
import com.vipsoft.sftp.utils.SftpUtil;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
@EnableConfigurationProperties(SftpProperties.class)
public class SftpConfig {
    // 工廠
    @Bean
    public SftpFactory sftpFactory(SftpProperties properties) {
        return new SftpFactory(properties);
    }

    // 連接配接池
    @Bean
    public SftpPool sftpPool(SftpFactory sftpFactory) {
        return new SftpPool(sftpFactory);
    }

    // 輔助類
    @Bean
    public SftpUtil sftpUtil(SftpPool sftpPool) {
        return new SftpUtil(sftpPool);
    }
}

SftpProperties.java

package com.vipsoft.sftp.config;

import com.jcraft.jsch.ChannelSftp;
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;
import org.springframework.boot.context.properties.ConfigurationProperties;

@ConfigurationProperties(prefix = "sftp")
public class SftpProperties {

    private String host;
    private int port = 22;
    private String username = "root";
    private String password = "root";
    private Pool pool = new Pool();

    public String getHost() {
        return host;
    }

    public void setHost(String host) {
        this.host = host;
    }

    public int getPort() {
        return port;
    }

    public void setPort(int port) {
        this.port = port;
    }

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public String getPassword() {
        return password;
    }

    public void setPassword(String password) {
        this.password = password;
    }

    public Pool getPool() {
        return pool;
    }

    public void setPool(Pool pool) {
        this.pool = pool;
    }

    public static class Pool extends GenericObjectPoolConfig<ChannelSftp> {

        private int maxTotal = DEFAULT_MAX_TOTAL;
        private int maxIdle = DEFAULT_MAX_IDLE;
        private int minIdle = DEFAULT_MIN_IDLE;

        public Pool() {
            super();
        }
        @Override
        public int getMaxTotal() {
            return maxTotal;
        }
        @Override
        public void setMaxTotal(int maxTotal) {
            this.maxTotal = maxTotal;
        }
        @Override
        public int getMaxIdle() {
            return maxIdle;
        }
        @Override
        public void setMaxIdle(int maxIdle) {
            this.maxIdle = maxIdle;
        }
        @Override
        public int getMinIdle() {
            return minIdle;
        }
        @Override
        public void setMinIdle(int minIdle) {
            this.minIdle = minIdle;
        }
    }
}

Pool

SftpFactory.java

package com.vipsoft.sftp.pool;

import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
import com.vipsoft.sftp.config.SftpProperties;
import com.vipsoft.sftp.exception.SftpPoolException;
import org.apache.commons.pool2.BasePooledObjectFactory;
import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Properties;

public class SftpFactory extends BasePooledObjectFactory<ChannelSftp> {

    private  final Logger logger = LoggerFactory.getLogger(this.getClass());

    private SftpProperties properties;

    public SftpProperties getProperties() {
        return properties;
    }

    public void setProperties(SftpProperties properties) {
        this.properties = properties;
    }

    public SftpFactory(SftpProperties properties) {
        this.properties = properties;
    }

    @Override
    public ChannelSftp create() {
        try {
            JSch jsch = new JSch();
            Session sshSession = jsch.getSession(properties.getUsername(), properties.getHost(), properties.getPort());
            sshSession.setPassword(properties.getPassword());
            Properties sshConfig = new Properties();
            sshConfig.put("StrictHostKeyChecking", "no");
            sshSession.setConfig(sshConfig);
            sshSession.connect();
            ChannelSftp channel = (ChannelSftp) sshSession.openChannel("sftp");
            channel.connect();
            return channel;
        } catch (JSchException e) {
            throw new SftpPoolException("連接配接sfpt失敗", e);
        }
    }

    @Override
    public PooledObject<ChannelSftp> wrap(ChannelSftp channelSftp) {
        return new DefaultPooledObject<>(channelSftp);
    }

    // 銷毀對象
    @Override
    public void destroyObject(PooledObject<ChannelSftp> p) {
        ChannelSftp channelSftp = p.getObject();
        channelSftp.disconnect();
    }

}

SftpPool.java

package com.vipsoft.sftp.pool;

import com.jcraft.jsch.ChannelSftp;
import org.apache.commons.pool2.impl.GenericObjectPool;

public class SftpPool<T> extends GenericObjectPool<ChannelSftp> {

    public SftpPool(SftpFactory factory) {
        super(factory,factory.getProperties().getPool());
    }

    /**
     * 擷取一個sftp連接配接對象
     * @return sftp連接配接對象
     */
    @Override
    public ChannelSftp borrowObject() throws Exception {
        return super.borrowObject();
    }

    /**
     * 歸還一個sftp連接配接對象
     * @param channelSftp sftp連接配接對象
     */
    @Override
    public void returnObject(ChannelSftp channelSftp) {
        if (channelSftp!=null) {
            super.returnObject(channelSftp);
        }
    }

}

Utils

ByteUtil.java

package com.vipsoft.sftp.utils;

import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.SftpException;
import com.vipsoft.sftp.exception.SftpPoolException;
import com.vipsoft.sftp.pool.SftpPool;

import java.io.InputStream;

public class SftpUtil {

    private SftpPool pool;

    public SftpUtil(SftpPool pool) {
        this.pool = pool;
    }

    /**
     * 下載下傳檔案
     *
     * @param dir  遠端目錄
     * @param name 遠端檔案名
     * @return 檔案位元組數組
     */
    public byte[] download(String dir, String name) {
        ChannelSftp sftp = null;
        try {
            sftp = pool.borrowObject();
            sftp.cd(dir);
            InputStream in = sftp.get(name);
            return ByteUtil.inputStreamToByteArray(in);
        } catch (Exception e) {
            throw new SftpPoolException("sftp下載下傳檔案出錯", e);
        } finally {
            pool.returnObject(sftp);
        }
    }

    /**
     * 上傳檔案
     *
     * @param dir  遠端目錄
     * @param name 遠端檔案名
     * @param in   輸入流
     */
    public void upload(String dir, String name, InputStream in) {
        ChannelSftp sftp = null;
        try {
            sftp = pool.borrowObject();
            mkdirs(sftp, dir);
            sftp.cd(dir);
            sftp.put(in, name);
        } catch (Exception e) {
            throw new SftpPoolException("sftp上傳檔案出錯", e);
        } finally {
            pool.returnObject(sftp);
        }
    }

    /**
     * 删除檔案
     *
     * @param dir  遠端目錄
     * @param name 遠端檔案名
     */
    public void delete(String dir, String name) {
        ChannelSftp sftp = null;
        try {
            sftp = pool.borrowObject();
            sftp.cd(dir);
            sftp.rm(name);
        } catch (Exception e) {
            throw new SftpPoolException("sftp删除檔案出錯", e);
        } finally {
            pool.returnObject(sftp);
        }
    }

    /**
     * 遞歸建立多級目錄
     *
     * @param dir 多級目錄
     */
    private void mkdirs(ChannelSftp sftp, String dir) {
        String[] folders = dir.split("/");
        try {
            sftp.cd("/");
            for (String folder : folders) {
                if (folder.length() > 0) {
                    try {
                        sftp.cd(folder);
                    } catch (Exception e) {
                        sftp.mkdir(folder);
                        sftp.cd(folder);
                    }
                }
            }
        } catch (SftpException e) {
            throw new SftpPoolException("sftp建立目錄出錯", e);
        }
    }

}

Test

SftpTest.java

package com.vipsoft.sftp;

import com.vipsoft.sftp.utils.SftpUtil;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

@SpringBootTest
public class SftpTest {

    @Autowired
    private SftpUtil sftpUtil;

    @Test
    void downloadTest() {
        byte[] dockerfiles = sftpUtil.download("/opt/demo/", "Dockerfile");
        System.out.println("FileSize =>" + dockerfiles.length);
    }

}
Java GenericObjectPool 對象池化技術-SpringBoot sftp 連接配接池工具類
池化技術 ObjectPool generic spring pring ring sftp Pool
上一篇: 檔案傳輸協定：FTP、TFTP、SFTP有什麼差別？
下一篇: Hi，程式員！點選列印你的「年度關鍵詞」

繼續閱讀