Create Connection
Field | Required | Default | Multiple | Notes | Example |
|---|---|---|---|---|---|
| type | Yes | - | No | The value must be kafka | kafka |
| description | Yes | - | No | Name of the connection object. | "MyApache KafkaConnection" |
| throttlePolicy | No | - | No | Id of the throttle policy that applies to this connection object. | "f5587cee-9116-4011-b3a9-6b235b333a1b" |
| routingPolicies | No | [ ] | Yes | The ids of the routing policies that this connection will use. | ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"] |
| properties | Yes | - | No | Configuration object | |
| server | yes | - | Yes | The Kafka server(s) hostnames fololowed by the port. | "localhost:9092" |
| stopOnScanError | yes | true | no | Check if you want the connector to stop crawing when an error is encountered. | "true" |
| offsets | yes | earliest | no | Starting offset for ful crawl. | "offsets": { "altPull": "Alpha", "debug": "Test Text Outer field set" } |
Example
POST aspire/_api/connections
{
"type": "kafka",
"description": "kafkaConnection_1",
"properties": {
"server": "localhost:9093",
"stopOnScanError": true,
"offsets": {
"altPull": "Alpha",
"debug": "Test Text Outer field set"
}
}
}
Update Connection
Field | Required | Default | Multiple | Notes | Example |
|---|---|---|---|---|---|
| id | Yes | - | No | Id of the connection to update | "89d6632a-a296-426c-adb0-d442adcab4b0", |
| description | No | - | No | Name of the connection object. | "My |
| throttlePolicy | No | - | No | Id of the throttle policy that applies to this connection object. | "f5587cee-9116-4011-b3a9-6b235b333a1b" |
| routingPolicies | No | [ ] | Yes | The ids of the routing policies that this connection will use. | ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"] |
| properties | Yes | - | No | Configuration object | |
Example
{
"type": "kafka",
"description": "Kafka Demo",
"properties": {
"multipleStartPoints": "false",
"url": "//opt//aspire",
"partialScan": "true",
"subDirUrl": null,
"indexContainers": "false",
"scanRecursively": "true",
"scanExcludedItems": "false",
"ignoreScanErrors": "false",
"ignoreSymLinks": "false",
"useACLs": "false",
"acls": null,
"includes": null,
"excludes": null
}
}
Update Connection
Field | Required | Default | Multiple | Notes | Example |
|---|---|---|---|---|---|
| id | Yes | - | No | Id of the connection to update | "89d6632a-a296-426c-adb0-d442adcab4b0", |
| description | No | - | No | Name of the connection object. | "My |
| throttlePolicy | No | - | No | Id of the throttle policy that applies to this connection object. | "f5587cee-9116-4011-b3a9-6b235b333a1b" |
| routingPolicies | No | [ ] | Yes | The ids of the routing policies that this connection will use. | ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"] |
| properties | Yes | - | No | Configuration object | |
| stopOnScanError | No | true | No | If enabled, the crawl will stop if there is an error on the scan phase. | true / false |
| url | Yes | - | No | Path of the base directory to crawl. All the seeds will be prefixed with this value to form the full path. All seeds will be prefixed with this value to form the full path | "C:\\Directory" |
Example
Create Connector Instance
For the creation of the Connector object using the Rest API check this page
Create Seed
Field | Required | Default | Multiple | Notes | Example |
|---|---|---|---|---|---|
| seed | Yes | - | No | <seed description> | |
| type | Yes | - | No | The value must be kafka. | kafka |
| description | Yes | - | No | Name of the seed object. | "My Apache Kafka Seed" |
| connector | Yes | - | No | The id of the connector to be used with this seed. The connector type must match the seed type. | "82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31" |
| connection | Yes | - | No | The id of the connection to be used with this seed. The connection type must match the seed type. | "602d3700-28dd-4a6a-8b51-e4a663fe9ee6" |
| workflows | No | [ ] | Yes | The ids of the workflows that will be executed for the documents crawled. | ["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"] |
| throttlePolicy | No | - | No | Id of the throttle policy that applies to this connection object. | "f5587cee-9116-4011-b3a9-6b235b333a1b" |
| routingPolicies | No | [ ] | Yes | The ids of the routing policies that this seed will use. | ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"] |
| tags | No | [ ] | Yes | The tags of the seed. These can be used to filter the seed | ["tag1", "tag2"] |
Example
POST aspire/_api/seeds
{
"type": "kafka",
"seed": "directory",
"connector": "82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31",
"description": "kafka_Test_Seed",
"connection": "602d3700-28dd-4a6a-8b51-e4a663fe9ee6",
"workflows": ["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]
}
Update Seed
Field | Required | Default | Multiple | Notes | Example |
|---|---|---|---|---|---|
| id | Yes | - | No | Id of the seed to update. | "2f287669-d163-4e35-ad17-6bbfe9df3778" |
| seed | No | - | No | <seed description> | |
| description | No | - | No | Name of the seed object. | "MyApache KafkaSeed" |
| connector | No | - | No | The id of the connector to be used with this seed. The connector type must match the seed type. | "82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31" |
| connection | No | - | No | The id of the connection to be used with this seed. The connection type must match the seed type. | "602d3700-28dd-4a6a-8b51-e4a663fe9ee6" |
| workflows | No | [ ] | Yes | The ids of the workflows that will be executed for the documents crawled. | ["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"] |
| workflows.add | No | [ ] | Yes | The ids of the workflows to add. | ["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"] |
| workflows.remove | No | [ ] | Yes | The ids of the workflows to remove. | ["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"] |
| throttlePolicy | No | - | No | Id of the throttle policy that applies to this connection object. | "f5587cee-9116-4011-b3a9-6b235b333a1b" |
| routingPolicies | No | [ ] | Yes | The ids of the routing policies that this seed will use. | ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"] |
| routingPolicies.add | No | [ ] | Yes | The ids of the routingPolicies to add. | ["b4d2579f-1a0a-4a8b-9fd4-d42780003b36"] |
| routingPolicies.remove | No | [ ] | Yes | The ids of the routingPolicies to remove. | ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7"] |
| tags | No | [ ] | Yes | The tags of the seed. These can be used to filter the seed | ["tag1", "tag3"] |
| tags.add | No | [ ] | Yes | The tags to add | ["tag4"] |
| tags.remove | No | [ ] | Yes | The tags to remove | ["tag2"] |
Example
PUT aspire/_api/seeds/2f287669-d163-4e35-ad17-6bbfe9df3778
{
"id": "2f287669-d163-4e35-ad17-6bbfe9df3778",
"seed": "C:\\test\\",
"connector": "82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31",
"description": "kafka_Test_Seed",
"throttlePolicy": "6b8b5f23-fc77-47a1-9b58-106577162e7b",
"routingPolicies": ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"],
"connection": "602d3700-28dd-4a6a-8b51-e4a663fe9ee6",
"workflows": ["b255e950-1dac-46dc-8f86-1238b2fbdf27"],
}
Overview
Content Tools
Copyright © 2024 Accenture
