Apache Kafka Connect

A Apache Kafka Connect is a streaming data system within a Kafka platform.

• Example(s):
  • the one introduced in Kafka v0.9.0.
  • …
• Counter-Example(s):
  • Apache Flink.
  • Apache Spark Streaming.
• See: Import Export Data Processing, Apache Kafka.

---

---

References

2021

• (Wikipedia, 2021) ⇒ https://en.wikipedia.org/wiki/Apache_Kafka#Connect_API Retrieved:2021-4-12.
  • Kafka Connect (or Connect API) is a framework to import/export data from/to other systems. It was added in the Kafka 0.9.0.0 release and uses the Producer and Consumer API internally. The Connect framework itself executes so-called "connectors" that implement the actual logic to read/write data from other systems. The Connect API defines the programming interface that must be implemented to build a custom connector. Many open source and commercial connectors for popular data systems are available already. However, Apache Kafka itself does not include production ready connectors. …

2020

• https://docs.confluent.io/platform/current/connect/index.html
  • QUOTE: ... Kafka Connect is a tool for scalably and reliably streaming data between Apache Kafka® and other data systems. It makes it simple to quickly define connectors that move large data sets into and out of Kafka. Kafka Connect can ingest entire databases or collect metrics from all your application servers into Kafka topics, making the data available for stream processing with low latency. An export connector can deliver data from Kafka topics into secondary indexes like Elasticsearch or into batch systems such as Hadoop for offline analysis. …

---