Page tree
Skip to end of metadata
Go to start of metadata

The Kafka connector fetches messages from a Kafka message stream.

On this page


Features

1) Performs incremental and full crawls.

  • An incremental crawl obtains all messages that have been added after the last full crawl, and then keeps running until it is paused or stopped.
  • A full crawl obtains all available messages from the topic stream at the time of crawl, and then stops.

2) Contains configurable message offsets (for full crawls).


Content Retrieved


The Kafka connector retrieves messages for the subscribed topic. The content body is the message value, which is retrieved by the text extraction component.

If you need the content element populated, then text extraction must be enabled.

Include

The following metadata are included:

  • Message key
  • Message value (same as content if text extraction is enabled)
  • Topic name
  • Partition number
  • Message offset


Limitations


Cannot track expired messages; it cannot send updates or deletes to the publisher.


  • No labels