Page History
The Apache Kafka Connector will crawl content from a repository.
| Easy Heading Free | ||||||
|---|---|---|---|---|---|---|
|
Introduction
The Apache Kafka Connector will crawl content from a file system with PLAINTEXT protocol.
Environment and Access Requirements
Repository Support
The Apache Kafka connector supports crawling the following the repositories:
| Repository | Version | Connector Version |
|---|---|---|
| Windows | Aspireall | 5.0.2 |
| Linux | all | 5.0.2 |
Account Privileges
For the Apache Kafka connector the Apache Kafka connector to be able to crawl content, the Aspire Worker nodes must be run with an account with with.
Environment Requirements
| Requirement | version |
|---|---|
| Apache Kafka | 3.0.0 |
Framework and Connector Features
Framework Features
| Name | Supported |
|---|---|
| Content Crawling | yes Yes |
| Identity Crawling | noNo |
| Snapshot-based Incrementals | yesYes |
| Non-snapshot-based Incrementals | yesNo |
| Document Hierarchy | noNo |
Connector Features
The Apache Kafka connector has the following features:
- Extract documents from multiple Kafka topics.
- Select between one and multiple servers to crawl.
Content Crawled
The Apache Kafka connector is able to crawl the following objects:
| Name | TypeRelevant Metadata | Content Fetch and Extraction | Description | |
|---|---|---|---|---|
| FileTopicdocument | System.Stringpartition | Yes | offsetThe topic name | N/A |
| Partitions | System.Collections.Generic.List<PartitionMetadata> | Yes | Metadata for the partitions of the topic | The files contained by the directories in the crawled file system. |
Limitations
The Apache Kafka Connector has the following limitations:
- REST API imlpementation implementation is part of our future development plan.
- No authentication mechanisms have been implemented to date.
Overview
Content Tools