The Aspider Web Crawler can be configured using the Aspire Admin UI. It requires the following entities to be created:

• Connection
• Connector
• Seed

Create Connection 

1. On the Aspire Admin UI go to the connections page page Image Added
2. All existing connections will be listed. Click on the new button button Image Added
3. Enter the new connection description. 
4. Select Apache Kafka from the Type list.
5. General:
   1. Servers: The Kafka server(s) hostnames fololowed by the port. Ex. localhost:9092
   2. Stop on scan error: Check if you want the connector to stop crawing when an error is encountered.
6. Starting offset for ful crawl: 
   1. Extract text: the auto offset reset configuration. 
   2. Debug: Debug messages on/off.

Create Connector Instance

For the creation of the Connector object using the Admin UI check this page.

Create Seed 

1. On the Aspire Admin UI go to the seeds page pageImage Added
2. All existing seed will be listed. Click on the new button buttonImage Added
3. Enter the new seed description.
4. Select Apache Kafka from the Type list.