You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 6 Next »

The SMB Connector can be configured using the Rest API. It requires the following entities to be created:

  • Connection
  • Connector
  • Seed
  • Credential

Below are the examples of how to create the Connection, the Seed, the Credential. For the General Connector Configuration, please check this page.


Create Connection



FieldRequiredDefaultMultipleNotesExample
typeYes-NoThe value must be "documentum-DQL"."documentum-DQL"
descriptionYes-NoName of the connection object."documentum-DQL connection"
credentialYes-NoThe ID of the credential to be used with this seed. The credential type must match the seed type."602d3700-28dd-4a6a-8b51-e4a663fe9ee6"
propertiesYes-NoConfiguration object
webtopUrlYes-NoHostname where the shared directory is located."shared.example.com"

serverUrl

Yes-NoThe default url for your documentum server"dctm://server-name:port"
dfcPropsFilePathYes-NoSpecify the filepath where the dfc.properties file is located. Typically this is placed in the "config" directory inside your Aspire Home."config/dfc.properties"
errorTolerant
NofalseNoIf selected, the connector will index only item's metadata in case there is not document content available. If not select, an error will be reported when there is no content (the item won't be index).true / false
useRenditionType
NofalseNoIf selected, include a list of rendition formats for documents to be extracted and indexedtrue/false
renditionType

No[]YesIf enabled, the crawl will stop if there is an error on the scan phase."html,jpg"
useAttributesNofalseNoIf selected, include a list of attributes names to index. If not selected, all document/folder's metadata fields will be indexed.",true / false
attributeListNo []YesList of metadata fields name to index separated by comma ','."object_name,r_object_type,r_creation_date,r_modify_date,i_chronicle_id,title"
fullSelectDQLNo

-

NoFull crawl SELECT statement. Aspire needs at least these key fields in the SELECT - r_object_id and i_chronicle_id. This query will return the object_id value of all items that will be crawled and indexedSELECT r_object_id, i_chronicle_id FROM dm_document
incrementalStrategyYes-NoIncremental StrategyAudit trail / Event queue
incrementalSelectDQLYes-NoIncremental crawl SELECT statement.  Aspire needs at least these key fields in the SELECT - r_object_id and i_chronicle_id. This query will return the object_id value of all items that will be crawled and indexed. The variable ${crawlTimeStamp} will be expanded upon incremental start.SELECT r_object_id, i_chronicle_id FROM dm_document WHERE r_modify_date > ${crawlTimeStamp}
selectAuditAclChangesDQLYes-NoAudit ACL updates SELECT statement for incremental.  Aspire needs at least these key fields in the SELECT - r_object_id and chronicle_id. The variable ${crawlTimeStamp} will be expanded upon incremental start.
SELECT r_object_id, chronicle_id FROM dm_audittrail 
WHERE event_name = 'dm_save' AND time_stamp_utc > ${crawlTimeStamp}
selectAuditDeletesDQLYes-NoAudit deletes SELECT statement for incremental. Aspire needs at least these fields in the SELECT - r_object_id and chronicle_id. The variable ${crawlTimeStamp} will be expanded upon incremental start.SELECT r_object_id, chronicle_id FROM dm_audittrail WHERE (event_name = 'dm_destroy' OR event_name = 'dm_prune')  AND time_stamp_utc > ${crawlTimeStamp}
selectAuditSafeguardDQL
Yes-NoCheck to include SID value on ACL output.Audit safeguard SELECT statement for incremental. The variable ${auditChronicleId} will be expanded for current audit chronicle id.SELECT r_object_id FROM dm_document WHERE i_chronicle_id = ${auditChronicleId} AND -whereFromFullCrawl-
deleteAuditItems
NofalseNoIf selected, audit items used to track document changes will be deleted after incrementaltrue / false
repositoryZoneId
No-NoValid repository time zone id. Not mandatory - UTC being the default valueEurope/Minsk, UTC+3, UTC-6
selectEventsSafeguardDQL
No-NoEvent safeguard SELECT statement for incremental. Not mandatory. The variable ${eventId} will be expanded for current event itemId.SELECT i_chronicle_id FROM dm_document WHERE r_object_id = ${eventId} AND -whereFromFullCrawl-
deleteEvents
No-No"List of delete events names separated by comma ','."dm_destroy, dm_prune"
maxFileSize
NoUnlimitedYesSpecify the limit for the size of the content to be crawled. Value should be submitted as MB. Type UNLIMITED if you do not wish to set a max content size."10"

Example

POST aspire/_api/connections
{
    "type": "smb",
    "description": "SMB Test Connector",
	"credential": "2a5ca234-e328-4d40-bb2a-2df3e550b065",
    "properties": {
        "host": "192.168.0.80",
        "port":"445",
        "disableFetch": false,
        "verboseSMBJ": false,
        "stopOnScanError": true,
        "indexContainers": true,
        "scanExcludedItems": true,
        "includes": ".*\\.txt",
        "excludes": ".*\\.png",
		"fetchACLs":true,
        "resolveSIDs": true,
		"addACLSID": false,
		"addACLEncodedSID": false,
		"addACLFlags": false,
		"addACLType" : false,
		"addACLAccessMask": false,
		"enableDFS": true,
		"connectionTimeout": 60000,
		"maxRetries": 5,
		"baseBackoff": 500,
		"backoffMultiplier": 2.0,
		"lastAccessedUpdates": false,
        "staticAcl": [{
                "name": "test-user",
                "domain": "test-domain",
                "entity": "user",
                "access": "allow"
            }, {
                "name": "test-group",
                "domain": "",
                "entity": "group",
                "access": "deny"
            }
        ]
    }
}

Update Connection


FieldRequiredDefaultMultipleNotesExample
typeYes-NoThe value must be "documentum-DQL"."documentum-DQL"
descriptionYes-NoName of the connection object."documentum-DQL connection"
credentialYes-NoThe ID of the credential to be used with this seed. The credential type must match the seed type."602d3700-28dd-4a6a-8b51-e4a663fe9ee6"
propertiesYes-NoConfiguration object
webtopUrlYes-NoHostname where the shared directory is located."shared.example.com"

serverUrl

Yes-NoThe default url for your documentum server"dctm://server-name:port"
dfcPropsFilePathYes-NoSpecify the filepath where the dfc.properties file is located. Typically this is placed in the "config" directory inside your Aspire Home."config/dfc.properties"
errorTolerant
NofalseNoIf selected, the connector will index only item's metadata in case there is not document content available. If not select, an error will be reported when there is no content (the item won't be index).true / false
useRenditionType
NofalseNoIf selected, include a list of rendition formats for documents to be extracted and indexedtrue/false
renditionType

No[]YesIf enabled, the crawl will stop if there is an error on the scan phase."html,jpg"
useAttributesNofalseNoIf selected, include a list of attributes names to index. If not selected, all document/folder's metadata fields will be indexed.",true / false
attributeListNo []YesList of metadata fields name to index separated by comma ','."object_name,r_object_type,r_creation_date,r_modify_date,i_chronicle_id,title"
fullSelectDQLNo

-

NoFull crawl SELECT statement. Aspire needs at least these key fields in the SELECT - r_object_id and i_chronicle_id. This query will return the object_id value of all items that will be crawled and indexedSELECT r_object_id, i_chronicle_id FROM dm_document
incrementalStrategyYes-NoIncremental StrategyAudit trail / Event queue
incrementalSelectDQLYes-NoIncremental crawl SELECT statement.  Aspire needs at least these key fields in the SELECT - r_object_id and i_chronicle_id. This query will return the object_id value of all items that will be crawled and indexed. The variable ${crawlTimeStamp} will be expanded upon incremental start.SELECT r_object_id, i_chronicle_id FROM dm_document WHERE r_modify_date > ${crawlTimeStamp}
selectAuditAclChangesDQLYes-NoAudit ACL updates SELECT statement for incremental.  Aspire needs at least these key fields in the SELECT - r_object_id and chronicle_id. The variable ${crawlTimeStamp} will be expanded upon incremental start.
SELECT r_object_id, chronicle_id FROM dm_audittrail 
WHERE event_name = 'dm_save' AND time_stamp_utc > ${crawlTimeStamp}
selectAuditDeletesDQLYes-NoAudit deletes SELECT statement for incremental. Aspire needs at least these fields in the SELECT - r_object_id and chronicle_id. The variable ${crawlTimeStamp} will be expanded upon incremental start.SELECT r_object_id, chronicle_id FROM dm_audittrail WHERE (event_name = 'dm_destroy' OR event_name = 'dm_prune')  AND time_stamp_utc > ${crawlTimeStamp}
selectAuditSafeguardDQL
Yes-NoCheck to include SID value on ACL output.Audit safeguard SELECT statement for incremental. The variable ${auditChronicleId} will be expanded for current audit chronicle id.SELECT r_object_id FROM dm_document WHERE i_chronicle_id = ${auditChronicleId} AND -whereFromFullCrawl-
deleteAuditItems
NofalseNoIf selected, audit items used to track document changes will be deleted after incrementaltrue / false
repositoryZoneId
No-NoValid repository time zone id. Not mandatory - UTC being the default valueEurope/Minsk, UTC+3, UTC-6
selectEventsSafeguardDQL
No-NoEvent safeguard SELECT statement for incremental. Not mandatory. The variable ${eventId} will be expanded for current event itemId.SELECT i_chronicle_id FROM dm_document WHERE r_object_id = ${eventId} AND -whereFromFullCrawl-
deleteEvents
No-No"List of delete events names separated by comma ','."dm_destroy, dm_prune"
maxFileSize
NoUnlimitedYesSpecify the limit for the size of the content to be crawled. Value should be submitted as MB. Type UNLIMITED if you do not wish to set a max content size."10"

Example

Connection with incremental strategy "Audit trail"
PUT aspire/_api/connections/89d6632a-a296-426c-adb0-d442adcab4b0
{
         "id": "1a1da314-4ccb-49fa-ac0c-aa3a726e0e3a",
         "type": "documentum-DQL",
         "description": "DQL_audit_trail_connection",
         "credential": "8153ae2e-7260-40d4-91e6-2501ce62fc91",
         "properties": {
            "serverUrl": "dctm://10.89.26.139:1489/",
            "dfcPropsFilePath": "config/dfc.properties",
            "errorTolerant": true,
            "useRenditionType": true,
            "renditionType": "html,jpg",
            "useAttributes": true,
            "attributeList": "object_name,",
            "webtopUrl": "http://10.89.26.139:8080/da/component/main",
            "fullSelectDQL": "SELECT r_object_id, i_chronicle_id FROM dm_document",
            "incrementalStrategy": "auditTrail",
            "incrementalSelectDQL": "SELECT r_object_id, i_chronicle_id FROM dm_document WHERE r_modify_date > ${crawlTimeStamp}",
            "selectAuditAclChangesDQL": "SELECT r_object_id, chronicle_id FROM dm_audittrail WHERE event_name = 'dm_save'  AND time_stamp_utc > ${crawlTimeStamp}",
            "selectAuditDeletesDQL": "SELECT r_object_id, chronicle_id FROM dm_audittrail WHERE (event_name = 'dm_destroy' OR event_name = 'dm_prune' )  AND time_stamp_utc >${crawlTimeStamp}",
            "selectAuditSafeguardDQL": "SELECT r_object_id FROM dm_document WHERE i_chronicle_id = ${auditChronicleId}",
            "deleteAuditItems": false,
            "repositoryZoneId": "",
            "maxFileSize": "Unlimited"
         }
}

Create Connector


For the creation of the Connector object using the Rest API, check this page

Update Connector


For the update of the Connector object using the Rest API, check this page

Create Seed


FieldRequiredDefaultMultipleNotesExample
seedYes-NoThe repository to crawl. This value will be appended to the URL of the connection."/repository"
typeYes-NoThe value must be "documentu-DQL"."documentu-DQL"
descriptionYes-NoName of the seed object."documentu-DQL seed"
connectorYes-NoThe ID of the connector to be used with this seed. The connector type must match the seed type."82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31"
connectionYes-NoThe ID of the connection to be used with this seed. The connection type must match the seed type."602d3700-28dd-4a6a-8b51-e4a663fe9ee6"
workflowsNo[ ]YesThe IDs of the workflows that will be executed for the documents crawled.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]
throttlePolicyNo-NoID of the throttle policy that applies to this connection object."f5587cee-9116-4011-b3a9-6b235b333a1b"
routingPoliciesNo[ ]YesThe IDs of the routing policies that this seed will use.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]
tagsNo[ ]YesThe tags of the seed. These can be used to filter the seed["tag1", "tag2"]

Example

POST aspire/_api/seeds
   {
      "type":"documentum-DQL",
      "description":"dql_seed",
      "connector":"d859a13d-db0b-457b-bc4f-e0311cf8162c",
      "seed":"/repository",
      "priority":"medium",
      "connection":"2a5ca234-e328-4d40-bb2a-2df3e550b065"
      "workflows": ["b255e950-1dac-46dc-8f86-1238b2fbdf27", "f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"],
      "tags": ["tag1", "tag2"],
      "throttlePolicy": "6b8b5f23-fc77-47a1-9b58-106577162e7b",
      "routingPolicies": ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]
    }

Update Seed


FieldRequiredDefaultMultipleNotesExample
idYes-NoID of the seed to update."2f287669-d163-4e35-ad17-6bbfe9df3778"
seedNo-NoThe repository to crawl. This value will be appended to the URL of the connection."/repository"
descriptionNo-NoName of the seed object."documentum-DQL seed"
connectorNo-NoThe ID of the connector to be used with this seed. The connector type must match the seed type."82f7f0a4-8d28-47ce-8c9d-e3ca414b0d31"
connectionNo-NoThe ID of the connection to be used with this seed. The connection type must match the seed type."602d3700-28dd-4a6a-8b51-e4a663fe9ee6"
workflowsNo[ ]YesThe IDs of the workflows that will be executed for the documents crawled.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]
workflows.addNo[ ]YesThe IDs of the workflows to add.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]
workflows.removeNo[ ]YesThe IDs of the workflows to remove.["f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"]
throttlePolicyNo-NoID of the throttle policy that applies to this connection object."f5587cee-9116-4011-b3a9-6b235b333a1b"
routingPoliciesNo[ ]YesThe IDs of the routing policies that this seed will use.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]
routingPolicies.addNo[ ]YesThe IDs of the routingPolicies to add.["b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]
routingPolicies.removeNo[ ]YesThe IDs of the routingPolicies to remove.["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7"]
tagsNo[ ]YesThe tags of the seed. These can be used to filter the seed["tag1", "tag3"]
tags.addNo[ ]YesThe tags to add["tag4"]
tags.removeNo[ ]YesThe tags to remove["tag2"]

Example

PUT aspire/_api/seeds/2f287669-d163-4e35-ad17-6bbfe9df3778
   {
      "id":"2f287669-d163-4e35-ad17-6bbfe9df3778",
      "type":"documentum-DQL",
      "description":"dql_seed",
      "connector":"d859a13d-db0b-457b-bc4f-e0311cf8162c",
      "seed":"/repository",
      "priority":"medium",
      "connection":"2a5ca234-e328-4d40-bb2a-2df3e550b065"
      "workflows": ["b255e950-1dac-46dc-8f86-1238b2fbdf27", "f8c414cb-1f5d-42ef-9cc9-5696c3f0bda4"],
      "tags": ["tag1", "tag2"],
      "throttlePolicy": "6b8b5f23-fc77-47a1-9b58-106577162e7b",
      "routingPolicies": ["313de87c-3cb9-4fe0-a2cb-17f75ce7d0c7", "b4d2579f-1a0a-4a8b-9fd4-d42780003b36"]
    }

Create Credential


FieldRequiredDefaultMultipleNotesExample
typeYes-NoThe value must be "documentum-DQL"."documentum-DQL"
descriptionYes-NoName of the credential object."documentum-DQL Credential"
authenticationYes-NoIt can be just "basic" for now"basic"
usernameYes-NoAccount user"admin"
passwordYes-NoAccount password"234dfc22re!?"

Example

POST aspire/_api/credentials
   {
      "type":"documentum-DQL",
      "description": "documentum DQL - description",
      "authentication":"basic",
      "username":"docuser",
      "password":"encrypted:A9AC6E19DF4ECFA129A3F2FA23C326E6B280D901E0E20958BD51A0B2FFE2C14D"
    }

Update Credential

FieldRequiredDefaultMultipleNotesExample
typeYes-NoThe value must be "documentum-DQL"."documentum-DQL"
descriptionYes-NoName of the credential object."documentum-DQL Credential"
authenticationYes-NoIt can be just "basic" for now"basic"
usernameYes-NoAccount user"admin"
passwordYes-NoAccount password"234dfc22re!?"

Example

PUT aspire/_api/credentials/2a5ca234-e328-4d40-bb2a-2df3e550b065
   {
      "id":"2a5ca234-e328-4d40-bb2a-2df3e550b065",
      "type":"documentum-DQL",
      "description": "documentum DQL - description",
      "authentication":"basic",
      "username":"docuser",
      "password":"encrypted:A9AC6E19DF4ECFA129A3F2FA23C326E6B280D901E0E20958BD51A0B2FFE2C14D"
    }
  • No labels