The Aspider Web Crawler can be configured using the Aspire Admin UI. It requires the following entities to be created:
Credential
Connection
Connector
Seed
Create Credential
On the Aspire Admin UI, go to the credentials page
All existing credentials will be listed. Click on the new button
Enter the new credential description.
Select Dropbox from the Type list.
General: In Authentication, you need to insert:
Application Identifier: Name of your app.
Access Token: Token represents a user and app pair, to operate on that team, to the extent allowed by that app's permissions.
Note: On September 30th, 2021, Dropbox will retire the creation of long-lived access tokens — resulting in all new tokens being short-lived and expires in 4 hours.
Create Connection
On the Aspire Admin UI, go to the connections page
All existing connections will be listed. Click on the new button
Enter the new connection description.
Select Dropbox from the Type list.
Policies
Throttle Policy: Select the throttle policy that applies to this connection object.
Routing Policies: Select the routing policies that this connection will use.
Create Connector Instance
For the creation of the Connector object using the Admin UI, refer to thispage.
Create Seed
On the Aspire Admin UI, go to the seeds page
All existing seed will be listed. Click on the new button
Enter the new seed description.
Select Dropbox from the Type list.
Dropbox Seed: The subdirectory to crawl. This value will be appended to the base path of the connection.
Connector: The ID of the connector to be used with this seed. The connector type must match the seed type.
Connection: The ID of the connection to be used with this seed. The connection type must match the seed type.
Workflow: The IDs of the workflows that will be executed for the documents crawled.
Tag: The tags of the seed, these can be used to filter the seed.
Policies
Throttle Policy: Select the throttle policy that applies to this connection object.
Routing Policies: Select the routing policies that this connection will use.