kip

最近研究kafka-connect, 相关的kip 整理:

KIP-521: Enable redirection of Connect’s log4j messages to a file by default

KIP-507: Securing Internal Connect REST Endpoints

KIP-495: Dynamically Adjust Log Levels in Connect

可以通过 网络请求 动态调整 loglevel. 重启后修改是丢失的

KIP-481: SerDe Improvements for Connect Decimal type in JSON

KIP-475: New Metrics to Measure Number of Tasks on a Connector

添加了 connector-total-task-count 、connector-running-task-count、connector-paused-task-count、connector-failed-task-count、connector-unassigned-task-count、connector-destroyed-task-count 的监控指标

KIP-465: Add Consolidated Connector Endpoint to Connect REST API

KIP-458: Connector Client Config Override Policy

KIP-454: Expansion of the ConnectClusterState interface

KIP-449: Add connector contexts to Connect worker logs

log4j.properties 添加 %X{connector.context} 信息.

KIP-440: Extend Connect Converter to support headers

在 Converter 的两个 序列化/反序列化方法中 添加了 header 参数

KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

KIP-411: Make default Kafka Connect worker task client IDs distinct

KIP-305: Add Connect primitive number converters

KIP-298: Error Handling in Connect

错误重试次数、最大重试次数的支持
deadletterqueue 的支持
消息header中添加了部分元数据
metrics 支持

KIP-297: Externalizing Secrets for Connect Configurations

KIP-285: Connect Rest Extension Plugin

support custom authentication filter 
support custom authorization filter

KIP-238: Expose Kafka cluster ID in Connect REST API

KIP-215: Add topic regex support for Connect sinks

KIP-212: Enforce set of legal characters for connector names

KIP-208: Add SSL support to Kafka Connect REST interface

KIP-197 Connect REST API should include the connector type when describing a connector

KIP-196: Add metrics to Kafka Connect framework

KIP-154 Add Kafka Connect configuration properties for creating internal topics

KIP-151 Expose Connector type in REST API

KIP-146 - Classloading Isolation in Connect

KIP-145 - Expose Record Headers in Kafka Connect

KIP-128: Add ByteArrayConverter for Kafka Connect

KIP-89: Allow sink connectors to decouple flush and offset commit

KIP-75 - Add per-connector Converters

KIP-66: Single Message Transforms for Kafka Connect

KIP-65: Expose timestamps to Connect

KIP-52: Connector Control APIs

提供 pause、resume、restart(connector/tasks)

KIP-51 - List Connectors REST API

提供了 connector list 接口

KIP-26 - Add Kafka Connect framework for data import/export

最初的设计. 提出了 kafka-connect 框架.