Versions Compared

Old Version 1

changes.mady.by.user David Grobelny

Saved on

compared with

New Version Current

changes.mady.by.user Pablo Bonilla

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The File System SMB Connector can be configured using the Rest API. It requires the following entities to be created:

  • Connection
  • Connector
  • Seed
  • Credential

Below Bellow are the examples of how to create the Connection and , the Seed, the Credential. For the General Connector Configuration, please check this page.

Easy Heading Free
navigationTitleOn this Page
navigationExpandOptionexpand-all-by-default


Create Connection

Field
Optional
RequiredDefaultMultipleNotesExample
type
No
Yes-NoThe value must be "
filesystem
smb"."
filesystem
smb"
description
No
Yes-NoName of the connection object."
MyFileSystemConnection
smbConnection"
throttlePolicy
credentialYes-No
Id
The ID of the
throttle policy that applies to this connection object
credential to be used with this seed. The credential type must match the seed type."
f5587cee
602d3700-
9116
28dd-
4011
4a6a-
b3a9
8b51-
6b235b333a1b
e4a663fe9ee6"
routingPolicies
hostnameYes
[ ]
-
YesThe ids of the routing policies that this connection will use.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]
NoHostname where the shared directory is located."shared.example.com"
portYes445NoPort where the SMB protocol is used."445"
propertiesYes-NoConfiguration object
disableFetch
NofalseNoCheck to disable the connector fetcher, only metadata will be collected.true / false
verboseSMBJ
NofalseNoCheck to enable SMBJ logging. (WARNING) Enabling this would decrease performance
propertiesNo-NoConfiguration objecturlNo-NoPath of the base directory to crawl. All the seeds will be prefixed with this value to form the full path. All the seeds will be prefixed with this value to form the full path"C:\\Directory"ignoreSymLinksYesfalseNoIf enabled symbolic links will not be processed and links in the root items will cause an error
.true / false
stopOnScanError
Yes
NotrueNoIf enabled, the crawl will stop if there is an error on the scan phase.true / false
indexContainers
Yes
NofalseNoEnable to index the directories.true / false
scanRecursively
Yes
NotrueNoEnable to scan discovered directories recursively.true / false
include
Yes
No

[ ]

YesPatterns to match against document URL, if any of them match, the document will be included in the crawl.[ ".*pdf$", ".*docx$" ]
exclude
Yes
No[ ]YesPatterns to match against document URL, if any of them match, the document will be excluded from the crawl.[ ".*png$", ".*jpeg$" ]
scanExcludedItems
Yes
NofalseNoEnable to force the scan of excluded directories, so child items within the scope can be found.true / false
staticAcl
fetchACLs
Yes
No
[ ]
true
YesStatic ACL configuration objectname
NoCheck to retrieve owner, group and ACL information.true / false
resolveSIDs
No
-
trueNo
Name of the static 
Check to resolve retrieved SIDs from owner, group and ACL.
"group1"domainYes""NoDomain of the static ACL."testDomain"entityYes"user"NoEntity (user / group) represented by the static ACL."user" / "group"accessYes"allow"NoAccess (allow / deny) granted by the ACL."allow" / "deny"
true / false
addACLSID
NofalseNoCheck to include SID value on ACL output.true / false
addACLEncodedSID
NofalseNoCheck to include Encoded SID (Base 32) value on ACL output.true / false
addACLFlags
NofalseNoCheck to include ACL flags on ACL output.true / false
addACLType
NofalseNoCheck to include ACL type on ACL output.true / false
addACLAccessMask
NofalseNoCheck to include ACL access mask on ACL output.true / false
enableDFS
NotrueNoDistributed File System (DFS) resolutiontrue /false
connectionTimeout
Yes6000NoTimeout in milliseconds for each SMB request."6000"
maxRetries
Yes5NoMaximum retries permitted per document."5"
baseBackoff
Yes500NoBase time for the back off sleeps (in ms)."500"
backoffMultiplier
Yes2.0NoMultiplier factor to be used for the back off time."2.0"
lastAccessedUpdates
NofalseNoCheck to restore the last accessed date on the documents processed by the connector. WARNING: Requires a user with permissions for writing. This is not supported by windows.true /false
staticAclNo

[ ]

YesStatic ACL configuration object
nameYes-NoName of the static ACL."group1"
domainNo""NoDomain of the static ACL."testDomain"
entityNo"user"NoEntity (user / group) represented by the static ACL."user" / "group"
accessNo"allow"NoAccess (allow / deny) granted by the ACL."allow" / "deny"

Example

Code Block
themeRDark
titlePOST aspire/_api/connections
{
    "type": "smb",
    "description": "SMB Test Connector",
	"credential": "2a5ca234-e328-4d40-bb2a-2df3e550b065",
    "properties": {
        "host": "192.168.0.80",
        "port":"445",
        "disableFetch": false,
        "verboseSMBJ": false,
        "stopOnScanError": true,
        "indexContainers": true,
        "scanExcludedItems": true,
        "includes": ".*\\.txt",
        "excludes": ".*\\.png",
		"fetchACLs":true,
        "resolveSIDs": true,
		"addACLSID": false,
		"addACLEncodedSID": false,
		"addACLFlags": false,
		"addACLType" : false,
		"addACLAccessMask": false,
		"enableDFS": true,
		"connectionTimeout": 60000,
		"maxRetries": 5,
		"baseBackoff": 500,
		"backoffMultiplier": 2.0,
		"lastAccessedUpdates": false,
        "staticAcl": [{
                "name": "test-user",
                "domain": "test-domain",
                "entity": "user",
                "access": "allow"
            }, {
                "name": "test-group",
                "domain": "",
                "entity": "group",
                "access": "deny"
            }
        ]
    }
}

Update Connection

FieldRequiredDefaultMultipleNotesExample
typeYes-NoThe value must be "smb"."smb"
descriptionYes-NoName of the connection object."smbConnection"
credentialYes-NoThe ID of the credential to be used with this seed. The credential type must match the seed type."602d3700-28dd-4a6a-8b51-e4a663fe9ee6"
hostnameYes-NoHostname where the shared directory is located."shared.example.com"
portYes445NoPort where the SMB protocol is used."445"
propertiesYes-NoConfiguration object
disableFetch
NofalseNoCheck to disable the connector fetcher, only metadata will be collected.true / false
verboseSMBJ
NofalseNoCheck to enable SMBJ logging. (WARNING) Enabling this would decrease performance.true / false
stopOnScanErrorNotrueNoIf enabled, the crawl will stop if there is an error on the scan phase.true / false
indexContainersNofalseNoEnable to index the directories.true / false
scanRecursivelyNotrueNoEnable to scan discovered directories recursively.true / false
includeNo

[ ]

YesPatterns to match against document URL, if any of them match, the document will be included in the crawl.[ ".*pdf$", ".*docx$" ]
excludeNo[ ]YesPatterns to match against document URL, if any of them match, the document will be excluded from the crawl.[ ".*png$", ".*jpeg$" ]
scanExcludedItemsNofalseNoEnable to force the scan of excluded directories, so child items within the scope can be found.true / false
fetchACLsNotrueNoCheck to retrieve owner, group and ACL information.true / false
resolveSIDs
NotrueNo
Check to resolve retrieved SIDs from owner, group and ACL.
true / false
addACLSID
NofalseNoCheck to include SID value on ACL output.true / false
addACLEncodedSID
NofalseNoCheck to include Encoded SID (Base 32) value on ACL output.true / false
addACLFlags
NofalseNoCheck to include ACL flags on ACL output.true / false
addACLType
NofalseNoCheck to include ACL type on ACL output.true / false
addACLAccessMask
NofalseNoCheck to include ACL access mask on ACL output.true / false
enableDFS
NotrueNoDistributed File System (DFS) resolutiontrue /false
connectionTimeout
Yes6000NoTimeout in milliseconds for each SMB request."6000"
maxRetries
Yes5NoMaximum retries permitted per document."5"
baseBackoff
Yes500NoBase time for the back off sleeps (in ms)."500"
backoffMultiplier
Yes2.0NoMultiplier factor to be used for the back off time."2.0"
lastAccessedUpdates
NofalseNoCheck to restore the last accessed date on the documents processed by the connector. WARNING: Requires a user with permissions for writing. This is not supported by windows.true /false
staticAclNo

[ ]

YesStatic ACL configuration object
nameYes-NoName of the static ACL."group1"
domainNo""NoDomain of the static ACL."testDomain"
entityNo"user"NoEntity (user / group) represented by the static ACL."user" / "group"
accessNo"allow"NoAccess (allow / deny) granted by the ACL."allow" / "deny"

Example

Code Block
themeRDark
titlePUT aspire/_api/connections/89d6632a-a296-426c-adb0-d442adcab4b0
{
    "type": "smb",
    "description": "SMB Test Connector",
	"credential": "2a5ca234-e328-4d40-bb2a-2df3e550b065",
    "properties": {

Example

Code Block
themeRDark
titlePOST aspire/_api/connections
{
    "type": "filesystem",
    "description": "FileSystem Test Connector",
    "properties": {
        "url": "C:\\Directory",
        "ignoreSymLinks": true,
        "stopOnScanError": true,
        "indexContainers": true,
        "scanExcludedItems": true,
        "ignoreSymLinks": true,
        "includes": ".*\\.txt",
        "excludes": ".*\\.png",
        "staticAcl": [{
                "name": "test-user",
                "domain": "test-domain",
                "entity": "user",
                "access": "allow"
            }, {
                "name": "test-group",
                "domain": "",
                "entity": "group",
                "accesshost": "deny"192.168.0.80",
            }
    "port":"445",
    ]
    }
}

Update Connection

FieldOptionalDefaultMultipleNotesExampleidNo-NoId of the connection to update"89d6632a-a296-426c-adb0-d442adcab4b0",typeNo-NoThe value must be "filesystem"."filesystem"descriptionYes-NoName of the connection object."MyFileSystemConnection"throttlePolicyYes-NoId of the throttle policy that applies to this connection object."f5587cee-9116-4011-b3a9-6b235b333a1b"routingPoliciesYes[ ]YesThe ids of the routing policies that this connection will use.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]propertiesNo-NoConfiguration objecturlNo-NoPath of the base directory to crawl. All the seeds will be prefixed with this value to form the full path. All seeds will be prefixed with this value to form the full path"C:\\Directory"ignoreSymLinksYesfalseNoIf enabled symbolic links will not be processed and links in the root items will cause an error.true / falsestopOnScanErrorYestrueNoIf enabled, the crawl will stop if there is an error on the scan phase.true / falseindexContainersYesfalseNoEnable to index the directories.true / falsescanRecursivelyYestrueNoEnable to scan discovered items recursively.true / falseincludeYes

[ ]

YesPatterns to match against document URL, if any of them match, the document will be included in the crawl.[ ".*pdf$", ".*docx$" ]excludeYes[ ]YesPatterns to match against document URL, if any of them match, the document will be excluded from the crawl.[ ".*png$", ".*jpeg$" ]scanExcludedItemsYesfalseNoEnable to force the scan of excluded directories, so child items within the scope can be found.true / falsestaticAclYes

[ ]

YesStatic ACL configuration objectnameNo-NoName of the static ACL."group1"domainYes""NoDomain of the static ACL."testDomain"entityYes"user"NoEntity (user / group) represented by the static ACL."user" / "group"accessYes"allow"NoAccess (allow / deny) granted by the ACL."allow" / "deny"

Example

"disableFetch": false,
        "verboseSMBJ": false,
        "stopOnScanError": true,
        "indexContainers": true,
        "scanExcludedItems": true,
        "includes": ".*\\.txt",
        "excludes": ".*\\.png",
		"fetchACLs":true,
        "resolveSIDs": true,
		"addACLSID": false,
		"addACLEncodedSID": false,
		"addACLFlags": false,
		"addACLType" : false,
		"addACLAccessMask": false,
		"enableDFS": true,
		"connectionTimeout": 60000,
		"maxRetries": 5,
		"baseBackoff": 500,
		"backoffMultiplier": 2.0,
		"lastAccessedUpdates": false,
        "staticAcl": [{
                "name": "test-user",
                "domain": "test-domain",
                "entity": "user",
                "access": "allow"
            }, {
                "name": "test-group",
                "domain": "",
                "entity": "group",
                "access": "deny"
            }
        ]
    }
}

Create Connector

For the creation of the Connector object using the Rest API, check this page

Update Connector

For the update of the Connector object using the Rest API, check this page

Create Seed

FieldRequiredDefaultMultipleNotesExample
seedYes-NoPath to the element to be crawled, can be a directory or a file."myDirectory/levelTwo"
typeYes-NoThe value must be "filesystem"."smb"
descriptionYes-NoName of the seed object."MySMB"
seedFileNofalseNoIf checked, the path will be processed as a file instead of a directory. WARNING: The crawler will only process the seed and then will stop.true /false
connectorYes-NoThe ID of the connector to be used with this seed. The connector type must match the seed type."82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31"
connectionYes-NoThe ID of the connection to be used with this seed. The connection type must match the seed type."602d3700-28dd-4a6a-8b51-e4a663fe9ee6"
workflowsNo[ ]YesThe IDs of the workflows that will be executed for the documents crawled.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]
throttlePolicyNo-NoID of the throttle policy that applies to this connection object."f5587cee-9116-4011-b3a9-6b235b333a1b"
routingPoliciesNo[ ]YesThe IDs of the routing policies that this seed will use.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]
tagsNo[ ]YesThe tags of the seed. These can be used to filter the seed["tag1", "tag2"]

Example

Code Block
themeRDark
titlePOST aspire/_api/seeds
{
    "type": "smb",
    "seed": "myDirectory/levelTwo",
    "connector": "82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31",
    "description": "FileSystem_Test_Seed",
    "throttlePolicy": "6b8b5f23-fc77-47a1-9b58-106577162e7b",
    "routingPolicies": ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"],
    "connection": "602d3700-28dd-4a6a-8b51-e4a663fe9ee6",
    "workflows": ["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"],
    "tags": ["tag1", "tag2"],
    "properties": {
        "seedFile": false
    }
}

Update Seed

FieldRequiredDefaultMultipleNotesExample
idYes-NoID of the seed to update."2f287669-d163-4e35-ad17-6bbfe9df3778"
Code Block
themeRDark
titlePUT aspire/_api/connections/89d6632a-a296-426c-adb0-d442adcab4b0
{
    "id": "89d6632a-a296-426c-adb0-d442adcab4b0",
    "type": "filesystem",
    "description": "FileSystem Test Connector",
    "properties": {
        "url": "C:\\Directory",
        "ignoreSymLinks": true,
        "stopOnScanError": true,
        "indexContainers": true,
        "scanRecursively": true,
        "scanExcludedItems": true,
        "includes": ".*\\.txt",
        "excludes": ".*\\.png",
        "staticAcl": [{
                "name": "test-user",
                "domain": "test-domain",
                "entity": "user",
                "access": "allow"
            }, {
                "name": "test-group",
                "domain": "",
                "entity": "group",
                "access": "deny"
            }
        ]
    }
}

Create Connector

For the creation of the Connector object using the Rest API check this page

Update Connector

For the update of the Connector object using the Rest API check this page

Create Seed

Code Block
themeRDark
titlePOST aspire/_api/seeds
{ "type": "filesystem", "seed": "directory", "connector": "82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31", "description": "FileSystem_Test_Seed", "throttlePolicy": "6b8b5f23-fc77-47a1-9b58-106577162e7b", "routingPolicies":
FieldOptionalDefaultMultipleNotesExample
seedNo-NoThe subdirectory to crawl. This value will be appended to the
url
URL of the connection."
directory"typeNo-NoThe value must be "filesystem"."filesystem"
myDirectory/levelTwo"
descriptionNo-NoName of the seed object."
MyFileSystemConnection
MySMB"
connector
seedFileNo
-
falseNo
The id of the connector to be used with this seed. The connector type must match the seed type."82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31"
If checked, the path will be processed as a file instead of a directory. WARNING: The crawler will only process the seed and then will stop.true /false
connector
connection
No-NoThe
id
ID of the
connection
connector to be used with this seed. The
connection
connector type must match the seed
type."602d3700-28dd-4a6a-8b51-e4a663fe9ee6"
type."82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31"
connectionNo
workflowsYes[ ]YesThe ids of the workflows that will be executed for the documents crawled.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]throttlePolicyYes
-No
Id
The ID of the
throttle policy that applies to this connection object
connection to be used with this seed. The connection type must match the seed type."
f5587cee
602d3700-
9116
28dd-
4011
4a6a-
b3a9
8b51-
6b235b333a1b
e4a663fe9ee6"
routingPolicies
workflows
Yes
No[ ]YesThe
ids
IDs of the
routing policies that this seed will use
workflows that will be executed for the documents crawled.["
313de87c
f8c414cb-
3cb9
1f5d-
4fe0
42ef-
a2cb
9cc9-
17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]tags
5696c3f0bda4"]
workflows.addNo
Yes
[ ]YesThe
tags
IDs of the
seed. These can be used to filter the seed["tag1", "tag2"]

Example

workflows to add.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]
workflows.removeNo[ ]YesThe IDs of the workflows to remove.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]
throttlePolicyNo-NoID of the throttle policy that applies to this connection object."f5587cee-9116-4011-b3a9-6b235b333a1b"
routingPoliciesNo[ ]YesThe IDs of the routing policies that this seed will use.
["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7",
"b4d2579f-1a0a-4a8b-9fd4-d42780003b36"
], "connection": "602d3700-28dd-4a6a-8b51-e4a663fe9ee6", "workflows": ["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"], "tags":
]
routingPolicies.addNo[ ]YesThe IDs of the routingPolicies to add.["b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]
routingPolicies.removeNo[ ]YesThe IDs of the routingPolicies to remove.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7"]
tagsNo[ ]YesThe tags of the seed. These can be used to filter the seed["tag1",
"
tag2"], "properties": { "seed": "directory" } }

Update Seed

FieldOptionalDefaultMultipleNotesExample
tag3"]
tags.addNo[ ]YesThe tags to add["tag4"]
tags.removeNo[ ]YesThe tags to remove["tag2"]

Example

Code Block
themeRDark
titlePUT aspire/_api/seeds/2f287669-d163-4e35-ad17-6bbfe9df3778
{
    "id": "
idNo-NoId of the seed to update."
2f287669-d163-4e35-ad17-6bbfe9df3778
"seedYes-NoThe subdirectory to crawl. This value will be appended to the url of the connection."directory"descriptionYes-NoName of the seed object."MyFileSystemConnection"connectorYes-NoThe id of the connector to be used with this seed. The connector type must match the seed type.
",
    "type": "smb",
    "seed": "myDirectory/levelTwo",
    "connector": "82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31"
connectionYes-NoThe id of the connection to be used with this seed. The connection type must match the seed type."602d3700-28dd-4a6a-8b51-e4a663fe9ee6"workflowsYes[ ]YesThe ids of the workflows that will be executed for the documents crawled.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]workflows.addYes[ ]YesThe ids of the workflows to add.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]workflows.removeYes[ ]YesThe ids of the workflows to remove.["
,
    "description": "FileSystem_Test_Seed",
    "throttlePolicy": "6b8b5f23-fc77-47a1-9b58-106577162e7b",
    "routingPolicies": ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"],
    "connection": "602d3700-28dd-4a6a-8b51-e4a663fe9ee6",
    "workflows": ["b255e950-1dac-46dc-8f86-1238b2fbdf27", "f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"],
    "tags": ["tag", "tag2"],
    "properties": {
        "seedFile": false
    }
}

Create Credential

FieldRequiredDefaultMultipleNotesExample
typeYes-NoThe value must be "smb"."smb"
descriptionYes-NoName of the credential object."smbCredential"
domainNo-NoDomain of the account that will crawl the shared directory. If the user is a local account leave blank"WORKGROUP"
usernameYes-NoAccount user"admin"
passwordYes-NoAccount password"234dfc22re!?"
throttlePolicyYes-NoId of the throttle policy that applies to this connection object."f5587cee-9116-4011-b3a9-6b235b333a1b"routingPoliciesYes[ ]YesThe ids of the routing policies that this seed will use.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]routingPolicies.addYes[ ]YesThe ids of the routingPolicies to add.["b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]routingPolicies.removeYes[ ]YesThe ids of the routingPolicies to remove.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7"]tagsYes[ ]YesThe tags of the seed. These can be used to filter the seed["tag1", "tag3"]tags.addYes[ ]YesThe tags to add["tag4"]tags.removeYes[ ]YesThe tags to remove["tag2"]

Example

Code Block
themeRDark
titlePUT POST aspire/_api/seeds/2f287669-d163-4e35-ad17-6bbfe9df3778api/credentials
{
    "idtype": "2f287669-d163-4e35-ad17-6bbfe9df3778smb",
    "typedescription": "filesystemSMB snapshot",
    "seed"properties": {
        "username": "directorytest",
        "connectorpassword": "82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31test1",
        "descriptiondomain": "FileSystem_Test_SeedWORKGROUP",
    "throttlePolicy": "6b8b5f23-fc77-47a1-9b58-}
}

Update Credential

FieldRequiredDefaultMultipleNotesExample
typeYes-NoThe value must be "smb"."smb"
descriptionYes-NoName of the credential object."smbCredential"
domainNo-NoDomain of the account that will crawl the shared directory. If the user is a local account leave blank"WORKGROUP"
usernameYes-NoAccount user"admin"
passwordNo-NoAccount password"234dfc22re!?"

Example

Code Block
themeRDark
titlePUT aspire/_api/credentials/2a5ca234-e328-4d40-bb2a-2df3e550b065
{106577162e7b",
    "routingPolicies": ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"],
    "connectiontype": "602d3700-28dd-4a6a-8b51-e4a663fe9ee6smb",
    "workflowsdescription": ["b255e950-1dac-46dc-8f86-1238b2fbdf27", "f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]"SMB snapshot",
    "tagsproperties": ["tag", "tag2"],
{
        "username": "test",
        "propertiespassword": {"test1",
        "seeddomain": "directoryWORKGROUP"
    }
}