我們先看看官方最新說明:
- log.retention.hours :The number of hours to keep a log file before deleting it (in hours), tertiary to log.retention.ms property
- log.retention.minutes:The number of minutes to keep a log file before deleting it (in minutes), secondary to log.retention.ms property. If not set, the value in log.retention.hours is used
- log.retention.ms :The number of milliseconds to keep a log file before deleting it (in milliseconds), If not set, the value in log.retention.minutes is used
- log.retention.bytes :The maximum size of the log before deleting it
- log.cleanup.policy :The default cleanup policy for segments beyond the retention window. A comma separated list of valid policies. Valid policies are: "delete" and "compact"
- log.cleaner.enable :Enable the log cleaner process to run on the server. Should be enabled if using any topics with a cleanup.policy=compact including the internal offsets topic. If disabled those topics will not be compacted and continually grow in size.
- log.cleaner.delete.retention.ms:How long are delete records retained?
解釋:
日志儲存時間 (hours|minutes),預設為7天(168小時)。 超過這個時間會根據policy處理資料。 bytes和minutes無論哪個先達到都會觸發。 log.retention.minutes:在删除日志檔案之前保留日志檔案的分鐘數(以分鐘為機關),如果未設定,則使用log.retention.hours中的值 log.retention.ms:在删除日志檔案之前保留日志檔案的毫秒數(以毫秒為機關),如果未設定,則使用log.retention.minutes中的值 log.retention.bytes:日志資料存儲的最大位元組數。超過這個時間會根據policy處理資料 log.cleanup.policy: 日志清理政策[compact, delete](預設是delete) log.cleaner.enable:是否開啟壓縮(預設:true) log.cleaner.delete.retention.ms :對于壓縮的日志保留的最長時間(預設:86400000ms) 注意: Consumer Configs: fetch.message.max.bytes 每個提取請求中為每個主題分區提取的消息位元組數。要大于等于message.max.bytes Producer Configs: max.request.size 請求的最大大小為位元組。要小于 message.max.bytes Broker Configs: message.max.bytes=5242880 允許的最大記錄批大小 replica.fetch.max.bytes=5342880 每個分區試圖擷取的消息位元組數。要大于等于message.max.bytes |
- https://kafka.apache.org/documentation/
- http://debugo.com/kafka-params/