Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Version of Azure Data Lake required

The Aspire Azure Data Lake connector was created and tested using version 3.2.

Before installing the Azure Data Lake connector, make sure that:

  • You have the necessary the Service Service to Service Application account created with pertinent access to your Data Lake.
  • The Azure Data Lake is up and running.
  • You have Admin rights to allow Read and Execute permissions on folders to Crawl.

Version of Azure Data Lake Required


The Aspire Azure Data Lake connector was created and tested using Version 3.2.


Panel
titleOn this page

Table of Contents

User Account Requirements 


In order to access the Azure Data Lake, an Application Account with sufficient privileges must be supplied. The following fields must be configured in order to setup set up a new Data Lake connection:


Get Application Account

This link will guide you through 1.  See Microsoft's Use portal to create an Azure Active Directory application and service principal that can access resources for the steps on how to properly create an Application ID and its key.

Note

Make sure to write down your Application Key at the time of creation. It will not be shown again after you exit the portal. It's also important to grant the necessary Reader access to your application.

2.  This connector uses a OAuth 2.0 authorization via Token End Point. Azure will supply this authorization, please follow this link and see Step 4. . See Microsoft's Step 4: Get the OAuth 2.0 token endpoint (only for Java-based applications)

     After Once these steps are completed, and you will have a created a valid Application created, make .

3.  Make sure to grant Read and Execute access - (at least - ) to files and folders and files to crawl. See Microsoft's Step 3: Assign the Azure AD application to the Azure Data Lake Store account file or folder.

4.  Follow the recommended Steps are included on this link (Step 3). Kindly take recommendation to use Advance Features of Data Lake file explorer File Explorer to recursively apply the same Parent folder permissions to sub-folders folder via "Apply folder permissions to sub-folders" option. In any case

     The Application does not have access to an any specific folder. Aspire will log this warning during the crawling process


Windows or Linux


The Azure Data Lake connector run runs on either a Windows or Linux instance of Aspire.