天天看點

[Kafaka Broker Configs]--log.retention.hours

我們先看看官方最新說明:

  • log.retention.hours  :The number of hours to keep a log file before deleting it (in hours), tertiary to log.retention.ms property
  • log.retention.minutes:The number of minutes to keep a log file before deleting it (in minutes), secondary to log.retention.ms property. If not set, the value in log.retention.hours is used
  • log.retention.ms       :The number of milliseconds to keep a log file before deleting it (in milliseconds), If not set, the value in log.retention.minutes is used
  • log.retention.bytes   :The maximum size of the log before deleting it
  • log.cleanup.policy     :The default cleanup policy for segments beyond the retention window. A comma separated list of valid policies. Valid policies are: "delete" and "compact"
  • log.cleaner.enable     :Enable the log cleaner process to run on the server. Should be enabled if using any topics with a cleanup.policy=compact including the internal offsets topic. If disabled those topics will not be compacted and continually grow in size.
  • log.cleaner.delete.retention.ms:How long are delete records retained?

解釋:

日志儲存時間 (hours|minutes),預設為7天(168小時)。

超過這個時間會根據policy處理資料。

bytes和minutes無論哪個先達到都會觸發。

log.retention.minutes:在删除日志檔案之前保留日志檔案的分鐘數(以分鐘為機關),如果未設定,則使用log.retention.hours中的值

log.retention.ms:在删除日志檔案之前保留日志檔案的毫秒數(以毫秒為機關),如果未設定,則使用log.retention.minutes中的值

log.retention.bytes:日志資料存儲的最大位元組數。超過這個時間會根據policy處理資料

log.cleanup.policy: 日志清理政策[compact, delete](預設是delete)

log.cleaner.enable:是否開啟壓縮(預設:true)

log.cleaner.delete.retention.ms :對于壓縮的日志保留的最長時間(預設:86400000ms)

注意:

Consumer Configs:

fetch.message.max.bytes

每個提取請求中為每個主題分區提取的消息位元組數。要大于等于message.max.bytes

Producer Configs:

max.request.size

請求的最大大小為位元組。要小于 message.max.bytes

Broker Configs:

message.max.bytes=5242880 

允許的最大記錄批大小

replica.fetch.max.bytes=5342880 

每個分區試圖擷取的消息位元組數。要大于等于message.max.bytes

  1. ​​https://kafka.apache.org/documentation/​​
  2. ​​http://debugo.com/kafka-params/​​