Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Panel
titleOn this page

Table of Contents

Version of Azure Data Lake required


The Aspire Azure Data Lake connector was created and tested using version 3.3.

Before installing the Azure Data Lake connector, make sure that:

  • You have necessary
Service
  • the Service to Service Application account created with pertinent access to your Data Lake
  • The Azure Data Lake is up and running
  • You have
admin
  • Admin rights to allow Read and Execute permissions on folders to Crawl




Panel
titleOn this page

Table of Contents

User Account Requirements 


In order to access the Azure Data Lake an Lake, an Application Account with sufficient privileges must be supplied. The following fields are mandatory to must be configured in order to setup a new Data Lake connection:

  • Authorization Token End Point: p.e. https://login.microsoftonline.com/[yourkey]/oauth2/token
  • Application ID
  • Application Secret (Application Key)
  • Fully Qualified Domain Name (FQDN): p.e [yourdomain].azuredatalakestore.net. No HTTP prefix is required

Get Application Account

This link will guide you through the steps on how to properly create an Application ID and it's key. Make sure to write down your Application Key at time of creation as it will not be shown again once you exit portal. It's also important to grant necessary reader access to your application, these steps are included.its key.

Note




This connector uses a OAuth 2.0 authorization via Token End Point. Azure will supply this authorization, please follow this link and see Step 4. 

Once these steps are completed, and you have a valid Application created, make sure to grant Read and Execute access - at least - to folders and files to crawl. Steps are included on this link (Step 3). Kindly take recommendation to use Advance Features of Data Lake file explorer to recursively apply same Parent folder permissions to sub-folders folder via "Apply folder permissions to sub-folders" option. In any case Application does not have access to an specific folder. Aspire will log this warning during crawling process

Windows or Linux


The Azure Data Lake connector run on either Windows or Linux instance of Aspire.