前言

公司部門老項目要遷移更新java版本，需要進行緩存相關操作，原架構未支援這部分，經過調研java相關緩存方案大緻分為ehcache和redis兩種，redis的value最大值為500mb且超過1mb會對存取有性能影響，業務系統需要支援清單查詢緩存就不可避免的涉及到大量的資料存取過濾，ehcache支援記憶體+磁盤緩存不用擔心緩存容量問題，是以架構初步版本決定內建ehcache3，設計流程結構如下圖所示

緩存配置

maven引用

<dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-cache</artifactId>
        </dependency>
        <dependency>
            <groupId>org.ehcache</groupId>
            <artifactId>ehcache</artifactId>
        </dependency>

個性化配置

#緩存配置
  cache:
    ehcache:
      heap: 1000
      offheap: 100
      disk: 500
      diskDir: tempfiles/cache/

@Component
@ConfigurationProperties("frmae.cache.ehcache")
public class EhcacheConfiguration {
    /**
     * ehcache heap大小
     * jvm記憶體中緩存的key數量
     */
    private int heap;
    /**
     * ehcache offheap大小
     * 堆外記憶體大小, 機關: MB
     */
    private int offheap;
    /**
     * 磁盤持久化目錄
     */
    private String diskDir;
    /**
     * ehcache disk
     * 持久化到磁盤的大小, 機關: MB
     * diskDir有效時才生效
     */
    private int disk;

    public EhcacheConfiguration(){
        heap = 1000;
        offheap = 100;
        disk = 500;
        diskDir = "tempfiles/cache/";
    }
}

代碼注入配置

因為springboot預設緩存優先注入redis配置，是以需要手動聲明bean進行注入，同時ehcache的value值必須支援序列化接口，不能使用Object代替，這裡聲明一個緩存基類，所有緩存value對象必須繼承該類

public class BaseSystemObject implements Serializable {
    
}

@Configuration
@EnableCaching
public class EhcacheConfig {
    @Autowired
    private EhcacheConfiguration ehcacheConfiguration;
    @Autowired
    private ApplicationContext context;

    @Bean(name = "ehCacheManager")
    public CacheManager getCacheManager() {
        //資源池生成器配置持久化
        ResourcePoolsBuilder resourcePoolsBuilder = 				  ResourcePoolsBuilder.newResourcePoolsBuilder()
                // 堆内緩存大小
                .heap(ehcacheConfiguration.getHeap(), EntryUnit.ENTRIES)
                // 堆外緩存大小
                .offheap(ehcacheConfiguration.getOffheap(), MemoryUnit.MB)
                // 檔案緩存大小
                .disk(ehcacheConfiguration.getDisk(), MemoryUnit.MB);
        //生成配置
        ExpiryPolicy expiryPolicy = ExpiryPolicyBuilder.noExpiration();
        CacheConfiguration config = CacheConfigurationBuilder.newCacheConfigurationBuilder(String.class, BaseSystemObject.class, resourcePoolsBuilder)
                //設定永不過期
                .withExpiry(expiryPolicy)
                .build();

        CacheManagerBuilder cacheManagerBuilder = CacheManagerBuilder.newCacheManagerBuilder()
                		 .with(CacheManagerBuilder.persistence(ehcacheConfiguration.getDiskDir()));
        return cacheManagerBuilder.build(true);
    }
}

緩存操作

緩存預熱

針對緩存架構選擇的雙寫政策，即資料庫和緩存同時寫入，是以在系統啟動時需要預先将資料庫資料加載到緩存中

針對單表聲明自定義注解，個性化緩存定義自定義接口

@Target({ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
public @interface HPCache {

}

public interface IHPCacheInitService {

    String getCacheName();

    void initCache();
}

系統初始化時同步進行緩存初始化，掃描注解實體類與接口實作Bean

@Async
    public void initCache(Class runtimeClass, List<String> extraPackageNameList) {
        List<Class<?>> cacheEntityList = new ArrayList<>();
        if (!runtimeClass.getPackage().getName().equals(Application.class.getPackage().getName())) {
            cacheEntityList.addAll(ScanUtil.getAllClassByPackageName_Annotation(runtimeClass.getPackage(), HPCache.class));
        }
        for (String packageName : extraPackageNameList) {
            cacheEntityList.addAll(ScanUtil.getAllClassByPackageName_Annotation(packageName, HPCache.class));
        }

        for (Class clazz : cacheEntityList) {
            TableName tableName = (TableName) clazz.getAnnotation(TableName.class);
            List<LinkedHashMap<String, Object>> resultList = commonDTO.selectList(tableName.value(), "*", "1=1", "", new HashMap<>(), false);
            for (LinkedHashMap<String, Object> map : resultList) {
                Cache cache = cacheManager.getCache(clazz.getName(), String.class, BaseSystemObject.class);
                String unitguid = ConvertOp.convert2String(map.get("UnitGuid"));
                try {
                    Object obj = clazz.newInstance();
                    obj = ConvertOp.convertLinkHashMapToBean(map, obj);
                    cache.put(unitguid, obj);
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        }

        //自定義緩存
        Map<String, IHPCacheInitService> res = context.getBeansOfType(IHPCacheInitService.class);
        for (Map.Entry en : res.entrySet()) {
            IHPCacheInitService service = (IHPCacheInitService) en.getValue();
            service.initCache();
        }

        System.out.println("緩存初始化完畢");
    }

需要注意，在EhcacheConfig配置類中需要進行緩存名稱的提前注冊，否則會導緻操作緩存時空指針異常

Map<String, Object> annotatedBeans = context.getBeansWithAnnotation(SpringBootApplication.class);
        Class runtimeClass = annotatedBeans.values().toArray()[0].getClass();
        //do，dao掃描
        List<String> extraPackageNameList = new ArrayList<String>();
        extraPackageNameList.add(Application.class.getPackage().getName());
        List<Class<?>> cacheEntityList = new ArrayList<>();
        if (!runtimeClass.getPackage().getName().equals(Application.class.getPackage().getName())) {
            cacheEntityList.addAll(ScanUtil.getAllClassByPackageName_Annotation(runtimeClass.getPackage(), HPCache.class));
        }
        for (String packageName : extraPackageNameList) {
            cacheEntityList.addAll(ScanUtil.getAllClassByPackageName_Annotation(packageName, HPCache.class));
        }

        for (Class clazz : cacheEntityList) {
            cacheManagerBuilder = cacheManagerBuilder.withCache(clazz.getName(), config);
        }

        //自定義緩存
        Map<String, IHPCacheInitService> res = context.getBeansOfType(IHPCacheInitService.class);
        for (Map.Entry en :res.entrySet()) {
            IHPCacheInitService service = (IHPCacheInitService)en.getValue();
            cacheManagerBuilder = cacheManagerBuilder.withCache(service.getCacheName(), config);
        }

更新操作

手動擷取ehcache的bean對象，調用put，repalce，delete方法進行操作

private  CacheManager cacheManager = (CacheManager) SpringBootBeanUtil.getBean("ehCacheManager");
    public void executeUpdateOperation(String cacheName, String key, BaseSystemObject value) {
        Cache cache = cacheManager.getCache(cacheName, String.class, BaseSystemObject.class);
        if (cache.containsKey(key)) {
            cache.replace(key, value);
        } else {
            cache.put(key, value);
        }
    }

    public void executeDeleteOperation(String cacheName, String key) {
        Cache cache = cacheManager.getCache(cacheName, String.class, BaseSystemObject.class);
        cache.remove(key);
    }

查詢操作

緩存存儲單表以主鍵—object形式存儲，個性化緩存為key-object形式存儲，單條記錄可以通過getCache方法查詢，清單查詢需要取出整個緩存按條件進行過濾

public Object getCache(String cacheName, String key){
        Cache cache = cacheManager.getCache(cacheName, String.class, BaseSystemObject.class);
        return cache.get(key);
    }

    public List<Object> getAllCache(String cacheName){
        List result = new ArrayList<>();
        Cache cache = cacheManager.getCache(cacheName, String.class, BaseSystemObject.class);
        Iterator iter = cache.iterator();
        while (iter.hasNext()) {
            Cache.Entry entry = (Cache.Entry) iter.next();
            result.add(entry.getValue());
        }
        return result;
    }

緩存與資料庫資料一緻性

資料庫資料操作與緩存操作順序為先操作資料後操作緩存，在開啟資料庫事務的情況下針對單條資料單次操作是沒有問題的，如果是組合操作一旦資料庫操作發生異常復原，緩存并沒有復原就會導緻資料的不一緻，比如執行順序為dbop1=》cacheop1=》dbop2=》cacheop2，dbop2異常，cacheop1的操作已經更改了緩存

這裡選擇的方案是在資料庫全部執行完畢後統一操作緩存，這個方案有一個缺點是如果緩存操作發生異常還是會出現上述問題，實際過程中緩存隻是對記憶體的操作異常機率較小，對緩存操作持樂觀狀态，同時我們提供手動重置緩存的功能，算是一個折中方案，下面概述該方案的一個實作

聲明自定義緩存事務注解

@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
public @interface CacheTransactional {

}

聲明切面監聽，在标記了CacheTransactional注解的方法執行前進行Redis辨別，統一執行完方法體後執行緩存操作

@Aspect
@Component
@Order(value = 101)
public class CacheExecuteAspect {
    @Autowired
    private CacheExecuteUtil cacheExecuteUtil;


    /**
     * 切面點 指定注解
     */
    @Pointcut("@annotation(com.haopan.frame.common.annotation.CacheTransactional) " +
            "|| @within(com.haopan.frame.common.annotation.CacheTransactional)")
    public void cacheExecuteAspect() {

    }

    /**
     * 攔截方法指定為 repeatSubmitAspect
     */
    @Around("cacheExecuteAspect()")
    public Object around(ProceedingJoinPoint point) throws Throwable {
        MethodSignature signature = (MethodSignature) point.getSignature();
        Method method = signature.getMethod();
        CacheTransactional cacheTransactional = method.getAnnotation(CacheTransactional.class);
        if (cacheTransactional != null) {
            cacheExecuteUtil.putCacheIntoTransition();
               try{
                   Object obj = point.proceed();
                   cacheExecuteUtil.executeOperation();
                   return obj;
               }catch (Exception e){
                   e.printStackTrace();
                   throw  e;
               }
        } else {
            return point.proceed();
        }
    }
}

将緩存操作以線程id區分放入待執行隊列中序列化到redis，提供方法統一操作

public class CacheExecuteModel implements Serializable {
    private String obejctClazzName;
    private String cacheName;
    private String key;
    private BaseSystemObject value;
    private String executeType;
}

private  CacheManager cacheManager = (CacheManager) SpringBootBeanUtil.getBean("ehCacheManager");

    @Autowired
    private RedisUtil redisUtil;

    public void putCacheIntoTransition(){
        String threadID = Thread.currentThread().getName();
        System.out.println("init threadid:"+threadID);
        CacheExecuteModel cacheExecuteModel = new CacheExecuteModel();
        cacheExecuteModel.setExecuteType("option");
        redisUtil.redisTemplateSetForCollection(threadID,cacheExecuteModel, GlobalEnum.RedisDBNum.Cache.get_value());
        redisUtil.setExpire(threadID,5, TimeUnit.MINUTES, GlobalEnum.RedisDBNum.Cache.get_value());
    }

    public void putCache(String cacheName, String key, BaseSystemObject value) {
        if(checkCacheOptinionInTransition()){
            String threadID = Thread.currentThread().getName();
            CacheExecuteModel cacheExecuteModel = new CacheExecuteModel("update", cacheName, key, value.getClass().getName(),value);
            redisUtil.redisTemplateSetForCollection(threadID,cacheExecuteModel, GlobalEnum.RedisDBNum.Cache.get_value());
            redisUtil.setExpire(threadID,5, TimeUnit.MINUTES, GlobalEnum.RedisDBNum.Cache.get_value());
        }else{
            executeUpdateOperation(cacheName,key,value);
        }

    }

    public void deleteCache(String cacheName, String key) {
        if(checkCacheOptinionInTransition()){
            String threadID = Thread.currentThread().getName();
            CacheExecuteModel cacheExecuteModel = new CacheExecuteModel("delete", cacheName, key);
            redisUtil.redisTemplateSetForCollection(threadID,cacheExecuteModel, GlobalEnum.RedisDBNum.Cache.get_value());
            redisUtil.setExpire(threadID,5, TimeUnit.MINUTES, GlobalEnum.RedisDBNum.Cache.get_value());
        }else{
            executeDeleteOperation(cacheName,key);
        }
    }

    public void executeOperation(){
        String threadID = Thread.currentThread().getName();
        if(checkCacheOptinionInTransition()){
            List<LinkedHashMap> executeList =  redisUtil.redisTemplateGetForCollectionAll(threadID, GlobalEnum.RedisDBNum.Cache.get_value());
            for (LinkedHashMap obj:executeList) {
                String executeType = ConvertOp.convert2String(obj.get("executeType"));
                if(executeType.contains("option")){
                    continue;
                }
                String obejctClazzName = ConvertOp.convert2String(obj.get("obejctClazzName"));
                String cacheName = ConvertOp.convert2String(obj.get("cacheName"));
                String key = ConvertOp.convert2String(obj.get("key"));
                LinkedHashMap valueMap = (LinkedHashMap)obj.get("value");
                String valueMapJson =  JSON.toJSONString(valueMap);
                try{
                    Object valueInstance = JSON.parseObject(valueMapJson,Class.forName(obejctClazzName));
                    if(executeType.equals("update")){
                        executeUpdateOperation(cacheName,key,(BaseSystemObject)valueInstance);
                    }else if(executeType.equals("delete")){
                        executeDeleteOperation(cacheName,key);
                    }
                }catch (Exception e){
                    e.printStackTrace();
                }
            }
            redisUtil.redisTemplateRemove(threadID,GlobalEnum.RedisDBNum.Cache.get_value());
        }

    }

    public boolean checkCacheOptinionInTransition(){
        String threadID = Thread.currentThread().getName();
        System.out.println("check threadid:"+threadID);
        return redisUtil.isValid(threadID, GlobalEnum.RedisDBNum.Cache.get_value());
    }

SpringBoot整合Ehcache3

前言

緩存配置

maven引用

個性化配置

代碼注入配置

緩存操作

緩存預熱

更新操作

查詢操作

緩存與資料庫資料一緻性

繼續閱讀

SpringBoot 應用打包為 Docker 鏡像，并設定時區同步

@Value的幾種常用用法

Android的前後端互動架構搭建（okhhtp+springboot+mysql完整版）

start to use skywalkingInstall JDK8Install ESInstall skywalkingRun Springboot JAR

關于Springboot中JSCH使用說明1. JSCH簡介2. JSCH依賴3. 使用方法4. 使用連接配接池

路徑錯誤Maven：Non-resolvable parent POM: Failure to find錯誤

jsch連接配接伺服器執行指令傳回的資料中文亂碼問題問題分析解決

springboot項目時間不一緻解決方案一、springboot項目時間不一緻解決方案

spring mvc 頁面傳遞數組背景接收方式spring mvc 頁面傳遞數組背景接收方式

Swagger-UI的使用一、添加依賴二、建立配置類三、注解

将swagger-ui中的接口導入postman

聊一聊Swagger-UI的那些事情軟體簡介Swagger 原生Bootstrap-ui Bootstrap的UIKnife4j Bootstrap的更新版的UI

springboot使用 swagger-ui 2.10.5 有關版本更新帶來的問題

微服務-性能壓測\緩存redis和分布式鎖redisson和SpringCache

java 8 Lamda 求 List 中 Bigdecimal類型的各種值求最大值求最小值求和求平均值

(SpringBoot)日志種類：log、monitor、access、out、gc、backup