The Azure Data Lake Connector can be configured using the Aspire Admin UI. It requires the following entities to be created:
Credential
Connection
Connector
Seed
Create Credential
On the Aspire Admin UI go to the credentials page
All existing credentials will be listed. Click on the new button
Enter the new credential description.
Select Azure Data Lake from the Type list.
Account Name: Storage Account name
Application ID: Client ID for your application
Application Secret:Key supplied by Azure
Tenant ID:Tenant ID for your Application
Create Connection
On the Aspire Admin UI go to the connections page
All existing connections will be listed. Click on the new button
Enter the new connection description.
Select Azure Data Lake from the Type list.
Select if all file systems are to be scanned
File System Name: Specify the name of the file system
Index Containers: Select if folders are to be indexed
Scan Recursively:Select if sub-folder are to be scanned
Scan Excluded Items:If selected, the scanner will scan sub items of container items that have been excluded by a pattern (because it matches an exclude pattern or because it doesn't match an include pattern)
Include patterns: Specify regex display URL patterns to include
Exclude patterns: Specify regex display URL patterns to exclude
Create Connector Instance
For the creation of the Connector object using the Admin UI check thispage.
Create Seed
On the Aspire Admin UI go to the seeds page
All existing seed will be listed. Click on the new button
Enter the new seed description.
Select Azure Data Lake from the Type list.
Specific Path: Specific path to crawl. If “Scan all Filesystems” in the Connection was checked, this path will be ignored.