You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 7 Next »

The Azure File Storage Connector can be configured using the Rest API. It requires the following entities to be created:

  • Credential
  • Connection
  • Connector
  • Seed

Below are the examples of how to create the Connection and the Seed. For the Connector please check this page.

Create Credential


Field

Required

Default

Multiple

NotesExample
typeYes-No

The value must be "azure-file-storage".

"azure-file-storage"

descriptionYes-NoName of the credential object.

"Azure File StorageCredential"

propertiesYes-NoConfiguration object
storageConnectionStringYes-NoAzure Storage Connection String"DefaultEndpointsProtocol=http;AccountName=myAccount;AccountKey=myKey;"

Example

POST aspire/_api/credentials
{
    "type": "azure-file-storage",
    "description": "<Connector Name> Credential",
    "properties": {   
		"storageConnectionString": "DefaultEndpointsProtocol=http;AccountName=myAccount;AccountKey=myKey;"     
	}
}

Update Credential


Field

Required

Default

Multiple

NotesExample
idYes-NoId of the credential to update."2f287669-d163-4e35-ad17-6bbfe9df3778"
descriptionYes-NoName of the credential object.

"Azure File StorageCredential"

propertiesYes-NoConfiguration object
storageConnectionStringYes-NoAzure Storage Connection String"DefaultEndpointsProtocol=http;AccountName=myAccount;AccountKey=myKey;"

Example 

PUT aspire/_api/credentials/2a5ca234-e328-4d40-bb2a-2df3e550b065
{
    "id": "2a5ca234-e328-4d40-bb2a-2df3e550b065",
    "description": "<Connector Name> Credential",
    "properties": {    
		"storageConnectionString": "DefaultEndpointsProtocol=http;AccountName=myAccount;AccountKey=myKey;"           
	}
}

Create Connection


Field

Required

Default

Multiple

Notes

Example

typeYes-No

The value must be azure-file-storage

azure-file-storage

descriptionYes-NoName of the connection object.

"MyAzure File StorageConnection"

throttlePolicyNo-NoId of the throttle policy that applies to this connection object."f5587cee-9116-4011-b3a9-6b235b333a1b"
credentialYes-NoID of the credential that applies to this connection object."d42e1872-02c8-4a90-a714-44f15577389a"
routingPoliciesNo[ ]YesThe ids of the routing policies that this connection will use.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]
propertiesYes-NoConfiguration object
scanRecursivelyNoTRUENoSelect if subfolders are to be scanned.TRUE
scanExcludedItemsNoFALSENoSelect so that the scanner will scan sub items of container items excluded by a patternFALSE
includesNo-YesList of regex URL patterns to include[{"include":".*tmp[^/]$"}]
includeNo-Noregex URL patterns to include".*tmp[^/]$"
excludesNo-YesList of regex URL patterns to exclude[{"include":".*tmp[^/]$"}]
excludeNo-Noregex URL patterns to exclude".*tmp[^/]$"

Example

POST aspire/_api/connections
{
    "type": "azure-file-storage",
	"credential": "d42e1872-02c8-4a90-a714-44f15577389a",
    "throttlePolicy": "",
    "routingPolicies": ["5c7274ef-429b-46ef-8f73-f010e479a467", "9dee4fba-14f2-4afc-a74d-297bcbbd359a"],
    "description": "<Connector Name> Test Connector",
    "properties": {           
        "scanRecursively": true,
        "scanExcludedItems": false,
        "includes": [
            {
                "include": ".*tmp[^/]$"
            }
        ],
        "excludes": [
            {
                "exclude": ".*tmp[^/]$"
            }
        ]      
	}
}

Update Connection


Field

Required

Default

Multiple

Notes

Example

idYes-NoId of the connection to update"89d6632a-a296-426c-adb0-d442adcab4b0",
descriptionNo-NoName of the connection object.

"MyConnection"

throttlePolicyNo-NoId of the throttle policy that applies to this connection object."f5587cee-9116-4011-b3a9-6b235b333a1b"
credentialNo-NoID of the credential that applies to this connection object."d42e1872-02c8-4a90-a714-44f15577389a"
routingPoliciesNo[ ]YesThe ids of the routing policies that this connection will use.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]
propertiesYes-NoConfiguration object
scanRecursivelyNoTRUENoSelect if subfolders are to be scanned.TRUE
scanExcludedItemsNoFALSENoSelect so that the scanner will scan sub items of container items excluded by a patternFALSE
includesNo-YesList of regex URL patterns to include[{"include":".*tmp[^/]$"}]
includeNo-Noregex URL patterns to include".*tmp[^/]$"
excludesNo-YesList of regex URL patterns to exclude[{"include":".*tmp[^/]$"}]
excludeNo-Noregex URL patterns to exclude".*tmp[^/]$"

Example

{
    "id": "89d6632a-a296-426c-adb0-d442adcab4b0",
	"credential": "d42e1872-02c8-4a90-a714-44f15577389a",
    "throttlePolicy": "",
    "routingPolicies": ["5c7274ef-429b-46ef-8f73-f010e479a467", "9dee4fba-14f2-4afc-a74d-297bcbbd359a"],
    "description": "<Connector Name> Test Connector",
    "properties": {         
        "scanRecursively": true,
        "scanExcludedItems": false,
        "includes": [
            {
                "include": ".*tmp[^/]$"
            }
        ],
        "excludes": [
            {
                "exclude": ".*tmp[^/]$"
            }
        ]     
	}
}

Create Connector Instance


For the creation of the Connector object using the Rest API check this page

Update Connector Instance


For the update of the Connector object using the Rest API check this page

Create Seed


Field

Required

Default

Multiple

Notes

Example

seedYes-No<seed description>
typeYes-No

The value must be azure-file-storage.

azure-file-storage

descriptionYes-NoName of the seed object.

"My Azure File Storage Seed"

connectorYes-NoThe id of the connector to be used with this seed. The connector type must match the seed type."82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31"
connectionYes-NoThe id of the connection to be used with this seed. The connection type must match the seed type."602d3700-28dd-4a6a-8b51-e4a663fe9ee6"
workflowsNo[ ]YesThe ids of the workflows that will be executed for the documents crawled.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]
throttlePolicyNo-NoId of the throttle policy that applies to this connection object."f5587cee-9116-4011-b3a9-6b235b333a1b"
routingPoliciesNo[ ]YesThe ids of the routing policies that this seed will use.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]
tagsNo[ ]YesThe tags of the seed. These can be used to filter the seed["tag1", "tag2"]
propertiesYes-NoConfiguration object
seedYes-NoThis value must be azure_file_storage_seedazure_file_storage_seed
urlNo-NoOnly required when sourceType "useRootURL" is selected. Azure File Share URL which will serve as the start of the crawl. http://hostname/tennant/fileShare
useExtraBlobContainersNofalseNoAdd More Azure Blob Containersfalse
siteCollectionsToCrawlNo-YesOnly required when "useExtraBlobContainers" is enabled. List of Azure Blob Container to crawl[{"siteCollectionUrl": "http://hostname/tennant/fileShare1"},{"siteCollectionUrl": "http://hostname/tennant/fileShare2"}]
siteCollectionUrlNo-NoAzure Blob Container to crawl"http://hostname/tennant/blobContainer1"

Example 

POST aspire/_api/seeds
{
    "type": "<Connector Type>",
    "seed": "directory",
    "connector": "82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31",
    "description": "<connector>_Test_Seed",
    "throttlePolicy": "6b8b5f23-fc77-47a1-9b58-106577162e7b",
    "routingPolicies": ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"],
    "connection": "602d3700-28dd-4a6a-8b51-e4a663fe9ee6",
    "workflows": ["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"],
    "tags": ["tag1", "tag2"],
    "properties": {
		"seed": "azure_file_storage_seed",
        "url": "http://hostname/tennant/fileShare",
        "useExtraBlobContainers": true,
        "siteCollectionsToCrawl": [
            {
                "siteCollectionUrl": "http://hostname/tennant/blobContainer1"
            },
            {
                "siteCollectionUrl": "http://hostname/tennant/blobContainer2"
            }
        ] 
     }
}

Update Seed


Field

Required

Default

Multiple

Notes

Example

idYes-NoId of the seed to update."2f287669-d163-4e35-ad17-6bbfe9df3778"
seedNo-No<seed description>
descriptionNo-NoName of the seed object.

"MyAzure File StorageSeed"

connectorNo-NoThe id of the connector to be used with this seed. The connector type must match the seed type."82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31"
connectionNo-NoThe id of the connection to be used with this seed. The connection type must match the seed type."602d3700-28dd-4a6a-8b51-e4a663fe9ee6"
workflowsNo[ ]YesThe ids of the workflows that will be executed for the documents crawled.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]
workflows.addNo[ ]YesThe ids of the workflows to add.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]
workflows.removeNo[ ]YesThe ids of the workflows to remove.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]
throttlePolicyNo-NoId of the throttle policy that applies to this connection object."f5587cee-9116-4011-b3a9-6b235b333a1b"
routingPoliciesNo[ ]YesThe ids of the routing policies that this seed will use.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]
routingPolicies.addNo[ ]YesThe ids of the routingPolicies to add.["b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]
routingPolicies.removeNo[ ]YesThe ids of the routingPolicies to remove.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7"]
tagsNo[ ]YesThe tags of the seed. These can be used to filter the seed["tag1", "tag3"]
tags.addNo[ ]YesThe tags to add["tag4"]
tags.removeNo[ ]YesThe tags to remove["tag2"]
propertiesYes-NoConfiguration object
seedYes-NoThis value must be azure_file_storage_seedazure_file_storage_seed
urlNo-NoOnly required when sourceType "useRootURL" is selected. Azure File Share URL which will serve as the start of the crawl. http://hostname/tenant/fileShare
useExtraBlobContainersNofalseNoAdd More Azure File Sharesfalse
siteCollectionsToCrawlNo-YesOnly required when "useExtraBlobContainers" is enabled. List of Azure Blob Container to crawl[{"siteCollectionUrl": "http://hostname/tenant/fileShare1"},{"siteCollectionUrl": "http://hostname/tenant/fileShare2"}]
siteCollectionUrlNo-NoAzure File Share to crawl"http://hostname/tenant/fileshare1"

Example 

PUT aspire/_api/seeds/2f287669-d163-4e35-ad17-6bbfe9df3778
{
    "id": "2f287669-d163-4e35-ad17-6bbfe9df3778",
    "seed": "<seed example>",
    "connector": "82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31",
    "description": "<connector>_Test_Seed",
    "throttlePolicy": "6b8b5f23-fc77-47a1-9b58-106577162e7b",
    "routingPolicies": ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"],
    "connection": "602d3700-28dd-4a6a-8b51-e4a663fe9ee6",
    "workflows": ["b255e950-1dac-46dc-8f86-1238b2fbdf27", "f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"],
    "tags": ["tag", "tag2"],
    "properties": {         
		seed": "azure_file_storage_seed",
        "url": "http://hostname/tennant/fileShare",
        "useExtraBlobContainers": true,
        "siteCollectionsToCrawl": [
            {
                "siteCollectionUrl": "http://hostname/tenant/fileshare1"
            },
            {
                "siteCollectionUrl": "http://hostname/tenant/fileshare2"
            }
        ]  
	}
}
  • No labels