這段時間利用周末學習了一下redis的主從同步,同時配合sentinel哨兵機制,sentinel是redis實作HA的一種方式。為了能學以緻用,這裡使用springboot 把redis 主從同步及sentinel整合起來實作一個商品搶購的demo,同時在開發過程中遇到的問題也都整理下來了。一方面加深對所學知識的印象,另一方面希望可以為那些剛接觸過這些知識的同學提供點實際幫助,springboot 整合redis主從及sentinel哨兵,自己從零開始,也花了一些時間,當然對那些大牛來說,此文可以繞過。
一、redis主從及sentinel環境配置
1、官方網站下載下傳redis https://redis.io/download
2、解壓壓縮包
tar -xvf redis-4.0.1.tar.gz
3、解壓完成後,進入目錄 redis-4.0.1
cd redis-4.0.1
4、執行make指令
make
執行make報錯,提示cc:未找到指令,原因是虛拟機系統中缺少gcc,安裝gcc即可
虛拟機安裝gcc指令
安裝指令:yum -y install gcc automake autoconf libtool make
安裝gcc後,執行make 繼續報錯:zmalloc.h:50:31: 緻命錯誤:jemalloc/jemalloc.h:沒有那個檔案或目錄
解決方案:
執行指令由make 改成 make MALLOC=libc
5、安裝redis服務到指定的目錄 /usr/local/redis
make PREFIX=/usr/local/redis install
6、建立配置檔案
mkdir /etc/redis
複制配置檔案到/etc/redis/ 下面去
cp redis.cnf /etc/redis/
7、啟動redis用戶端 進入redis 的bin 目錄
./redis-server
8、檢視redis是否正常啟動
ps -ef | grep redis
或者檢視redis端口是否被監聽
netstat -tunple | grep 6379
9、修改redis 配置檔案 背景啟動
vi /etc/redis/redis.conf
修改daemonize no 将no改成yes 即可
./redis-server /etc/redis/redis-conf 使用配置檔案背景啟動
10、關閉linux背景運作的redis服務
進入 bin 目錄
使用 pkill 指令
pkill -9 redis-server
11、redis用戶端連接配接
./redis-cli -h 192.168.137.30 -p 6379
set name hello
get name
12、redis 主從同步,slave啟動時,會給master發送一個同步指令,然後master以檔案的形式同步給slave;
第一次是全量同步,以後會以增量的形式同步,
master同步時資料是非阻塞的,slave同步時時阻塞的(當slave正在同步時,如果應用發送請求過來,必須等slave同步完之後,才能接受請求)
哨兵機制 切換回來之前的主從 一是修改sentinel配置檔案,二是關掉sentinel程序,重新開機redis主從
master 讀寫并行
slave 隻讀
./redis-cli -h 192.168.137.30 -p 6379 用戶端連接配接
進入後
192.168.137.30:6379> info 檢視配置資訊
13、哨兵背景啟動
修改配置檔案sentinel.conf 增加 daemonize yes
啟動 ./redis-sentinel /etc/redis/sentinel.conf
通過以上步驟,redis環境配置基本上就完成了。
開始建立springboot項目,這裡使用的是idea,建立project-->Spring Initializr 然後next,定義相關包名 路徑即可.
1、application.yml配置如下:
spring:
redis:
hostName: 192.168.137.30
port: 6379
password:
pool:
maxActive: 200
maxWait: -1
maxIdle: 8
minIdle: 0
timeout: 0
database: 0
sentinel:
master: mymaster
nodes: 192.168.137.32
port: 26379
server:
port: 8080
2、pom.xml增加redis依賴
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
<version>1.5.6.RELEASE</version>
</dependency>
<dependency>
<groupId>redis.clients</groupId>
<artifactId>jedis</artifactId>
</dependency>
3、RedisConfig.java redis相關配置類,容器啟動時會加載。
@Configuration
@EnableAutoConfiguration
public class RedisConfig {
private static Logger logger = LoggerFactory.getLogger(RedisConfig.class);
@Value("${spring.redis.sentinel.master}")
private String master;
@Value("${spring.redis.sentinel.nodes}")
private String sentinelHost;
@Value("${spring.redis.sentinel.port}")
private Integer sentinelPort;
@Bean
@ConfigurationProperties(prefix="spring.redis")
public JedisPoolConfig getRedisConfig(){
JedisPoolConfig config = new JedisPoolConfig();
return config;
}
@Bean
@ConfigurationProperties(prefix="spring.redis")
public JedisConnectionFactory getConnectionFactory(){
JedisPoolConfig config = getRedisConfig();
JedisConnectionFactory factory = new JedisConnectionFactory(getRedisSentinelConfig(), config);
factory.setPoolConfig(config);
return factory;
}
@Bean
@ConfigurationProperties(prefix = "spring.sentinel")
public RedisSentinelConfiguration getRedisSentinelConfig(){
RedisSentinelConfiguration sentinelConfiguration = new RedisSentinelConfiguration();
sentinelConfiguration.setMaster(master);
sentinelConfiguration.sentinel(sentinelHost,sentinelPort);
return sentinelConfiguration;
}
@Bean
public RedisTemplate<?, ?> getRedisTemplate(){
RedisTemplate<?,?> redisTemplate = new StringRedisTemplate(getConnectionFactory());
return redisTemplate;
}
}
4、redis服務接口,這裡隻寫了一個下單是否成功方法
public interface IRedisService {
public boolean isOrderSuccess(int buyCount,long flashSellEndDate);
}
5、redis服務接口的實作類
@Service
public class RedisServiceImpl implements IRedisService {
private Logger logger = LoggerFactory.getLogger(getClass());
@Autowired
RedisTemplate redisTemplate;
private static final int GOODS_TOTAL_COUNT = 200;//商品總數量
private static final String LOCK_KEY = "checkLock";//線程鎖key
private static final String SELL_COUNT_KEY = "sellCountKeyNew2";//redis中存放的已賣數量的key
private static final int LOCK_EXPIRE = 6 * 1000; //鎖占有時長
/**
* 檢查下單是否成功
* @param buyCount 購買數量
* @param flashSellEndDate 截止時間
* @return
*/
public boolean isOrderSuccess(int buyCount,long flashSellEndDate) {
if(flashSellEndDate <= 0){
logger.info("搶購活動已經結束:" + flashSellEndDate);
return false;
}
boolean resultFlag = false;
try {
if (redisLock(LOCK_KEY, LOCK_EXPIRE)) {
Integer haveSoldCount = (Integer) this.getValueByKey(SELL_COUNT_KEY);
Integer totalSoldCount = (haveSoldCount == null ? 0 : haveSoldCount) + buyCount;
if (totalSoldCount <= GOODS_TOTAL_COUNT) {
this.setKeyValueWithExpire(SELL_COUNT_KEY, totalSoldCount, flashSellEndDate);
resultFlag = true;
logger.info("已買數量: = " + totalSoldCount);
logger.info("剩餘數量:= " + (GOODS_TOTAL_COUNT - totalSoldCount));
}else{
if(haveSoldCount < GOODS_TOTAL_COUNT){
logger.info("對不起,您購買的數量已經超出商品庫存數量,請重新下單.");
}else{
logger.info("對不起,商品已售完.");
}
}
this.removeByKey(LOCK_KEY);
} else {
Integer soldCount = (Integer) this.getValueByKey(LOCK_KEY);
if(soldCount != null && soldCount >= GOODS_TOTAL_COUNT){
//商品已經售完
logger.info("all goods have sold out");
return false;
}
Thread.sleep(1000);//沒有擷取到鎖 1s後重試
return isOrderSuccess(buyCount,flashSellEndDate);
}
}catch (Exception e){
e.printStackTrace();
}
return resultFlag;
}
/**
* redis 鎖
* @param lock 鎖的key
* @param expire 鎖的時長
* @return
*/
public Boolean redisLock(final String lock, final int expire) {
return (Boolean) redisTemplate.execute(new RedisCallback<Boolean>() {
@Override
public Boolean doInRedis(RedisConnection connection) throws DataAccessException {
boolean locked = false;
byte[] lockKeyName = redisTemplate.getStringSerializer().serialize(lock);
byte[] lockValue = redisTemplate.getValueSerializer().serialize(getDateAferExpire(expire));
locked = connection.setNX(lockKeyName, lockValue);
if (locked){
connection.expire(lockKeyName, TimeoutUtils.toSeconds(expire, TimeUnit.MILLISECONDS));
}
return locked;
}
});
}
/**
*
* 判斷key是否存在
* @param key
*/
public boolean existsKey(final String key) {
return redisTemplate.hasKey(key);
}
/**
* 擷取指定時長後的Date
* @param expireTime
* @return
*/
public Date getDateAferExpire(int expireTime){
Calendar calendar = Calendar.getInstance();
calendar.add(Calendar.MILLISECOND, expireTime);
return calendar.getTime();
}
/**
* 根據key 擷取對應的value
*
* @param key
*/
public Object getValueByKey(final String key) {
Object result = null;
ValueOperations<Serializable, Object> operations = redisTemplate.opsForValue();
result = operations.get(key);
return result;
}
/**
* 删除指定的key
*
* @param key
*/
public void removeByKey(final String key) {
if (existsKey(key)) {
redisTemplate.delete(key);
}
}
/**
* 設定帶有指定時長的key value
* @param key
* @param value
* @param expireTime
* @return
*/
public boolean setKeyValueWithExpire(final String key, Object value, Long expireTime) {
boolean result = false;
try {
ValueOperations<Serializable, Object> operations = redisTemplate.opsForValue();
operations.set(key, value);
if (expireTime != null) {
redisTemplate.expire(key, expireTime, TimeUnit.SECONDS);
}
result = true;
} catch (Exception e) {
e.printStackTrace();
}
return result;
}
}
6、測試類TestOrderController
@RestController
public class TestOrderController {
@Autowired
IRedisService redisService;
private int buyCount = 20;
private static final int TEST_NUM = 15;
private static final String SELL_END_DATE = "2017-09-24 23:50:00";
private CountDownLatch cdl = new CountDownLatch(TEST_NUM);
@RequestMapping("orderTest/{buyCountParam}")
public String orderTest(@PathVariable int buyCountParam){
buyCount = buyCountParam;
for (int i=0; i<TEST_NUM; i++){
new Thread(new MyThread()).start();
cdl.countDown();
}
return "success";
}
private class MyThread implements Runnable{
@Override
public void run() {
try {
cdl.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
Calendar calendar = Calendar.getInstance();
Calendar calendar1 = Calendar.getInstance();
try {
calendar1.setTime(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").parse(SELL_END_DATE));
} catch (ParseException e) {
e.printStackTrace();
}
redisService.isOrderSuccess(buyCount,(calendar1.getTime().getTime() - calendar.getTime().getTime()) / 1000);
}
}
}
7、測試位址:http://localhost:8080/orderTest/20
控制台輸出:
2017-09-24 22:58:57.069 INFO 3856 --- [ Thread-35] c.h.h.miaosha.service.RedisServiceImpl : 已買數量: = 20
2017-09-24 22:58:57.069 INFO 3856 --- [ Thread-35] c.h.h.miaosha.service.RedisServiceImpl : 剩餘數量:= 180
2017-09-24 22:59:03.081 INFO 3856 --- [ Thread-37] c.h.h.miaosha.service.RedisServiceImpl : 已買數量: = 40
2017-09-24 22:59:03.081 INFO 3856 --- [ Thread-37] c.h.h.miaosha.service.RedisServiceImpl : 剩餘數量:= 160
2017-09-24 22:59:09.096 INFO 3856 --- [ Thread-25] c.h.h.miaosha.service.RedisServiceImpl : 已買數量: = 60
2017-09-24 22:59:09.097 INFO 3856 --- [ Thread-25] c.h.h.miaosha.service.RedisServiceImpl : 剩餘數量:= 140
2017-09-24 22:59:15.120 INFO 3856 --- [ Thread-32] c.h.h.miaosha.service.RedisServiceImpl : 已買數量: = 80
2017-09-24 22:59:15.120 INFO 3856 --- [ Thread-32] c.h.h.miaosha.service.RedisServiceImpl : 剩餘數量:= 120
2017-09-24 22:59:21.141 INFO 3856 --- [ Thread-29] c.h.h.miaosha.service.RedisServiceImpl : 已買數量: = 100
2017-09-24 22:59:21.141 INFO 3856 --- [ Thread-29] c.h.h.miaosha.service.RedisServiceImpl : 剩餘數量:= 100
2017-09-24 22:59:27.154 INFO 3856 --- [ Thread-38] c.h.h.miaosha.service.RedisServiceImpl : 已買數量: = 120
2017-09-24 22:59:27.154 INFO 3856 --- [ Thread-38] c.h.h.miaosha.service.RedisServiceImpl : 剩餘數量:= 80
2017-09-24 22:59:33.172 INFO 3856 --- [ Thread-26] c.h.h.miaosha.service.RedisServiceImpl : 已買數量: = 140
2017-09-24 22:59:33.172 INFO 3856 --- [ Thread-26] c.h.h.miaosha.service.RedisServiceImpl : 剩餘數量:= 60
2017-09-24 22:59:39.180 INFO 3856 --- [ Thread-31] c.h.h.miaosha.service.RedisServiceImpl : 已買數量: = 160
2017-09-24 22:59:39.180 INFO 3856 --- [ Thread-31] c.h.h.miaosha.service.RedisServiceImpl : 剩餘數量:= 40
2017-09-24 22:59:45.190 INFO 3856 --- [ Thread-36] c.h.h.miaosha.service.RedisServiceImpl : 已買數量: = 180
2017-09-24 22:59:45.190 INFO 3856 --- [ Thread-36] c.h.h.miaosha.service.RedisServiceImpl : 剩餘數量:= 20
2017-09-24 22:59:51.210 INFO 3856 --- [ Thread-34] c.h.h.miaosha.service.RedisServiceImpl : 已買數量: = 200
2017-09-24 22:59:51.211 INFO 3856 --- [ Thread-34] c.h.h.miaosha.service.RedisServiceImpl : 剩餘數量:= 0
2017-09-24 22:59:57.234 INFO 3856 --- [ Thread-27] c.h.h.miaosha.service.RedisServiceImpl : 對不起,商品已售完.
2017-09-24 23:00:03.243 INFO 3856 --- [ Thread-28] c.h.h.miaosha.service.RedisServiceImpl : 對不起,商品已售完.
2017-09-24 23:00:09.254 INFO 3856 --- [ Thread-24] c.h.h.miaosha.service.RedisServiceImpl : 對不起,商品已售完.
2017-09-24 23:00:15.266 INFO 3856 --- [ Thread-30] c.h.h.miaosha.service.RedisServiceImpl : 對不起,商品已售完.
2017-09-24 23:00:21.275 INFO 3856 --- [ Thread-33] c.h.h.miaosha.service.RedisServiceImpl : 對不起,商品已售完.
8、進入sentinel用戶端 可以檢視:
info
目前master位址: 192.168.137.30 : 6379
# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=192.168.137.30:6379,slaves=1,sentinels=1
此時kill掉目前的master
檢視sentinel日志:
2934:X 24 Sep 23:03:21.659 # +switch-master mymaster 192.168.137.30 6379 192.168.137.31 6379
2934:X 24 Sep 23:03:21.659 * +slave slave 192.168.137.30:6379 192.168.137.30 6379 @ mymaster 192.168.137.31 6379
2934:X 24 Sep 23:03:51.723 # +sdown slave 192.168.137.30:6379 192.168.137.30 6379 @ mymaster 192.168.137.31 6379
可以看到sentinel已經進行了 switch-master 将之前的slave切換成master,而之前的master則轉換成了slave
項目控制台輸出:
2017-09-24 23:06:27.336 INFO 3856 --- [8.137.32:26379]] redis.clients.jedis.JedisSentinelPool : Created JedisPool to master at 192.168.137.31:6379
這個時候也可以看到master由之前的192.168.137.30:6379 切換成 192.168.137.31:6379
到此,基本完成了,時間不早了,休息了,明天還要上班。