The Aspider Web Crawler Apache Kafka Connector can be configured using the Aspire Admin UI. It requires the following entities to be created:

• Connection
• Connector
• Seed

Create Connection 

1. On the Aspire Admin UI go to the connections page
2. All existing connections will be listed. Click on the new button
3. Enter the new connection description. 
4. Select Apache Kafka from the Type list.
5. General:
   1. Servers: The Kafka server(s) hostnames fololowed by the port. Ex. localhost:9092
   2. Stop on scan error: Check if you want the connector to stop crawing when an error is encountered.
6. Starting offset for ful crawl: 
   1. Extract text: the auto offset reset configuration. 
   2. Debug: Debug messages on/off.

Create Connector Instance

For the creation of the Connector object using the Admin UI check this page.

Create Seed 

1. On the Aspire Admin UI go to the seeds page
2. All existing seed will be listed. Click on the new button
3. Enter the new seed description.
4. Select Apache Kafka from the Type list.
5. Seed:
   1. Topic: Select the topic you want to crawl.
6. Connector:
   1. Connector: Select a Kafka connector.
7. Connection:
   1. Connection: Select a Kafka connection.
8. Worklow:
   1. Workflows: Select the workflow(s) for the seed.