天天看点

[Kafaka Broker Configs]--log.retention.hours

我们先看看官方最新说明:

  • log.retention.hours  :The number of hours to keep a log file before deleting it (in hours), tertiary to log.retention.ms property
  • log.retention.minutes:The number of minutes to keep a log file before deleting it (in minutes), secondary to log.retention.ms property. If not set, the value in log.retention.hours is used
  • log.retention.ms       :The number of milliseconds to keep a log file before deleting it (in milliseconds), If not set, the value in log.retention.minutes is used
  • log.retention.bytes   :The maximum size of the log before deleting it
  • log.cleanup.policy     :The default cleanup policy for segments beyond the retention window. A comma separated list of valid policies. Valid policies are: "delete" and "compact"
  • log.cleaner.enable     :Enable the log cleaner process to run on the server. Should be enabled if using any topics with a cleanup.policy=compact including the internal offsets topic. If disabled those topics will not be compacted and continually grow in size.
  • log.cleaner.delete.retention.ms:How long are delete records retained?

解释:

日志保存时间 (hours|minutes),默认为7天(168小时)。

超过这个时间会根据policy处理数据。

bytes和minutes无论哪个先达到都会触发。

log.retention.minutes:在删除日志文件之前保留日志文件的分钟数(以分钟为单位),如果未设置,则使用log.retention.hours中的值

log.retention.ms:在删除日志文件之前保留日志文件的毫秒数(以毫秒为单位),如果未设置,则使用log.retention.minutes中的值

log.retention.bytes:日志数据存储的最大字节数。超过这个时间会根据policy处理数据

log.cleanup.policy: 日志清理策略[compact, delete](默认是delete)

log.cleaner.enable:是否开启压缩(默认:true)

log.cleaner.delete.retention.ms :对于压缩的日志保留的最长时间(默认:86400000ms)

注意:

Consumer Configs:

fetch.message.max.bytes

每个提取请求中为每个主题分区提取的消息字节数。要大于等于message.max.bytes

Producer Configs:

max.request.size

请求的最大大小为字节。要小于 message.max.bytes

Broker Configs:

message.max.bytes=5242880 

允许的最大记录批大小

replica.fetch.max.bytes=5342880 

每个分区试图获取的消息字节数。要大于等于message.max.bytes

  1. ​​https://kafka.apache.org/documentation/​​
  2. ​​http://debugo.com/kafka-params/​​