我们先看看官方最新说明:
- log.retention.hours :The number of hours to keep a log file before deleting it (in hours), tertiary to log.retention.ms property
- log.retention.minutes:The number of minutes to keep a log file before deleting it (in minutes), secondary to log.retention.ms property. If not set, the value in log.retention.hours is used
- log.retention.ms :The number of milliseconds to keep a log file before deleting it (in milliseconds), If not set, the value in log.retention.minutes is used
- log.retention.bytes :The maximum size of the log before deleting it
- log.cleanup.policy :The default cleanup policy for segments beyond the retention window. A comma separated list of valid policies. Valid policies are: "delete" and "compact"
- log.cleaner.enable :Enable the log cleaner process to run on the server. Should be enabled if using any topics with a cleanup.policy=compact including the internal offsets topic. If disabled those topics will not be compacted and continually grow in size.
- log.cleaner.delete.retention.ms:How long are delete records retained?
解释:
日志保存时间 (hours|minutes),默认为7天(168小时)。 超过这个时间会根据policy处理数据。 bytes和minutes无论哪个先达到都会触发。 log.retention.minutes:在删除日志文件之前保留日志文件的分钟数(以分钟为单位),如果未设置,则使用log.retention.hours中的值 log.retention.ms:在删除日志文件之前保留日志文件的毫秒数(以毫秒为单位),如果未设置,则使用log.retention.minutes中的值 log.retention.bytes:日志数据存储的最大字节数。超过这个时间会根据policy处理数据 log.cleanup.policy: 日志清理策略[compact, delete](默认是delete) log.cleaner.enable:是否开启压缩(默认:true) log.cleaner.delete.retention.ms :对于压缩的日志保留的最长时间(默认:86400000ms) 注意: Consumer Configs: fetch.message.max.bytes 每个提取请求中为每个主题分区提取的消息字节数。要大于等于message.max.bytes Producer Configs: max.request.size 请求的最大大小为字节。要小于 message.max.bytes Broker Configs: message.max.bytes=5242880 允许的最大记录批大小 replica.fetch.max.bytes=5342880 每个分区试图获取的消息字节数。要大于等于message.max.bytes |
- https://kafka.apache.org/documentation/
- http://debugo.com/kafka-params/