Hystrix 熔斷機制原理

相關配置

circuitBreaker.enabled  是否開啟熔斷
circuitBreaker.requestVolumeThreshold  熔斷最低觸發請求數門檻值
circuitBreaker.sleepWindowInMilliseconds  産生熔斷後恢複視窗
circuitBreaker.errorThresholdPercentage  錯誤率門檻值
circuitBreaker.forceOpen  強制打開熔斷
circuitBreaker.forceClosed  強制關閉熔斷
           

狀态圖

執行流程

指令執行前調用

circuitBreaker.attemptExecution()

，正常情況下會執行傳回true，但是如果發生熔斷，則需要通過sleepWindows來進行恢複

public boolean attemptExecution() {
    if (properties.circuitBreakerForceOpen().get()) {
        return false;
    }
    if (properties.circuitBreakerForceClosed().get()) {
        return true;
    }
    if (circuitOpened.get() == -1) {
        return true;
    } else {
        if (isAfterSleepWindow()) {
            if (status.compareAndSet(Status.OPEN, Status.HALF_OPEN)) {
                //only the first request after sleep window should execute
                return true;
            } else {
                return false;
            }
        } else {
            return false;
        }
    }
}
           

發生熔斷流程

在新版本1.5.12中，會有一個背景線程訂閱metrics流實時計算：

1. 如果沒有達到RequestVolume，則直接傳回，不計算是否需要熔斷
2. 如果目前錯誤率大于設定的門檻值，則觸發熔斷，狀态由CLOSED切換到OPEN，并設定目前熔斷的時間（用于後續SleepWindows判斷使用）

if (hc.getTotalRequests() < properties.circuitBreakerRequestVolumeThreshold().get()) {
    // we are not past the minimum volume threshold for the stat window,
    // so no change to circuit status.
    // if it was CLOSED, it stays CLOSED
    // if it was half-open, we need to wait for a successful command execution
    // if it was open, we need to wait for sleep window to elapse
} else {
    if (hc.getErrorPercentage() < properties.circuitBreakerErrorThresholdPercentage().get()) {
        //we are not past the minimum error threshold for the stat window,
        // so no change to circuit status.
        // if it was CLOSED, it stays CLOSED
        // if it was half-open, we need to wait for a successful command execution
        // if it was open, we need to wait for sleep window to elapse
    } else {
        // our failure rate is too high, we need to set the state to OPEN
        if (status.compareAndSet(Status.CLOSED, Status.OPEN)) {
            circuitOpened.set(System.currentTimeMillis());
        }
    }
}
           

熔斷恢複流程

 當發生熔斷，達到SleepWindows指定時間後，則狀态會由OPEN轉換為HALF_OPEN，此時隻會讓一個請求通過，如果該請求執行成功，狀态則會轉換為CLOSED，否則會回到OPEN狀态，并且熔斷時間設定為目前時間，重新等待SleepWindows的時間

// 執行成功後調用
public void markSuccess() {
    if (status.compareAndSet(Status.HALF_OPEN, Status.CLOSED)) {
        //This thread wins the race to close the circuit - it resets the stream to start it over from 0
        metrics.resetStream();
        Subscription previousSubscription = activeSubscription.get();
        if (previousSubscription != null) {
            previousSubscription.unsubscribe();
        }
        Subscription newSubscription = subscribeToStream();
        activeSubscription.set(newSubscription);
        circuitOpened.set(-1L);
    }
}
           

// 執行失敗後調用
public void markNonSuccess() {
    if (status.compareAndSet(Status.HALF_OPEN, Status.OPEN)) {
        //This thread wins the race to re-open the circuit - it resets the start time for the sleep window
        circuitOpened.set(System.currentTimeMillis());
    }
}
           

參考

1. https://github.com/Netflix/Hystrix/wiki/Configuration
2. https://github.com/Netflix/Hystrix/wiki/How-it-Works#CircuitBreaker
3. HystrixCircuitBreaker源碼