Versions Compared

Old Version 3

changes.mady.by.user Elisa Argueta

Saved on

compared with

New Version 4

changes.mady.by.user Elisa Argueta

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Step 1. Launch Aspire and open the Content Source Management Page

Launch Aspire (if it's not already running). See:

...

Code Block
{
    "acl": [
        {
            "type": "user",
            "value": "d411eb08-42e2-4316-aab5-2df8e9d9c21b",
            "accessType": "grant",
            "identitySource": "Azure Active Directory"
        }
    ],
    "createdDateTime": "2017-11-08T19:06:17Z",
    "modifiedDateTime": "2017-11-08T19:06:17Z",
    "createdBy": "empty",
    "lastModifiedBy": "empty",
    "title": "sample document",
    "url": "http://the.url.com",
    "name": "name.txt",
    "extension": "txt",
    "size": 10,
    "content": "the content/n"
}

ExternalItem schema

As mentioned before, the ExternalItem schema has limited customization capabilities.  It expects the following information:

  • acl:  The list of ACLs for the document
  • properties:  A name/value list of custom properties of predefined types
  • content:  The document's content

When configured to use custom schema, the publisher component expects a txt file with the following structure:

Code Block
{
    "properties": [
        {
            "name": "propertyName",
            "type": "String",
            "isSearchable": "true",
            "isRetrievable": "true",
            "isQueryable": "true"
        }
    ]
}

Where:

  • name:  The property name in JSON friendly characters
  • type:  The data type of the property. Can be one of the following:
    • String
    • Int64
    • Double
    • DateTime (as UTC offset: YYYY-MM-DDTHH:mm:ss.nnZ)
    • Boolean
    • Collection(String)
    • Collection(Int64)
    • Collection(Double)
    • Collection(DateTime)
  • isSearchable:  (optional) If true, the property will be indexed as containing searchable content
  • isRetrievable:  (optional) If true, the property can be retrieved as part of search results
  • isQueryable:  (optional) If true, the property can be queried for specific values

This is a the default schema file provided with the component (schemaProperties.json):




Code Block
{
    "properties": [
        {
            "name": "id",
            "type": "String"
        },
        {
            "name": "name",
            "type": "String",
            "isSearchable": "true",
            "isRetrievable": "true",
            "isQueryable": "true"
        },
        {
            "name": "createdDateTime",
            "type": "String",
            "isSearchable": "true",
            "isRetrievable": "true",
            "isQueryable": "true"
        },
        {
            "name": "modifiedDateTime",
            "type": "String",
            "isSearchable": "true",
            "isRetrievable": "true",
            "isQueryable": "true"
        },
        {
            "name": "title",
            "type": "String",
            "isSearchable": "true",
            "isRetrievable": "true"
        },
        {
            "name": "url",
            "type": "String",
            "isSearchable": "true",
            "isRetrievable": "true",
            "isQueryable": "true"
        },
        {
            "name": "description",
            "type": "String",
            "isSearchable": "true",
            "isRetrievable": "true",
            "isQueryable": "true"
        }
    ]
}

Groovy Transformation file

The Groovy transformation file makes it easy to output data that is customized to the client's needs and also that can be safely conveyed to Microsoft Search through the REST API.  The output of the transformation must match the expected schema structure.

This is the default Groovy transformation file that is provided with the component (aspireToMicrosoftSearchBulk.groovy):

Code Block
import java.math.BigInteger;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.nio.charset.Charset;
 
def connectorSpecificMap = [
    'isContainer':'is_container'
]
 
def getContent(String content) {
    try {
        if(content.getBytes().length > 16777216L) {
            return content.substring(0,10485760) + "...";
        } else {
            return content
        }
    } catch(Throwable t) {
        return "";
    }
}
 
def getMD5(String id) {
   
  MessageDigest digest = MessageDigest.getInstance("MD5") 
  String md5name = new BigInteger(1, digest.digest(id.getBytes())).toString(16)
  return md5name;
}
 
// Function that process the children of a connector specific field
def getChildren(name, parent) {
 
  builder."$name"() {
    parent.getChildren().each() { val ->
 
      def attr = val.getName();
 
      //if it has other children
      if(val.getChildren().size() > 0) {
        getChildren(attr, val);
      } else {
        builder."$attr"() {
 
          //All the attributes
          val.getAttributeNames().each() { attrName ->
            "@$attrName" val.getAttribute(attrName);
          }
 
          //Main content
          if (val?.getText() != null) {
            '_$' val?.getText();
          }
        }
      }
    }
  }
}
 
//***************************************************
//
// Main routine
//
 
// Action of the job
String action = doc.action.getText();
 
if ((action == "add") || (action == "update")) {
  /*****************
   * Add or Update *
   *****************/
   
  builder.$object() {
    '@search.action' "upload"
 
    String newId = "";
    // Get ID
    if (doc.id != null) {
      newId = getMD5(doc.id.getText())
    } else if (doc.fetchUrl != null) {
      newId = getMD5(doc.fetchUrl.getText())
    } else if (doc.url != null) {
      newId = getMD5(doc.url.getText())
    } else if (doc.displayUrl != null) {
      newId = getMD5(doc.displayUrl.getText())
    } else {
      newId = "ID-NOT-PROVIDED"
    }
     
    'id' newId
 
 
  // Document Source
    String[] nameDoc = doc.id.getText().split('/');
    String finalName = nameDoc[nameDoc.size()-1];
 
    name finalName
    title finalName
    description finalName    
 
    if(doc.lastModified != null) {
      modifiedDateTime doc.lastModified
      createdDateTime doc.lastModified
    }
     
    if (doc.fetchUrl != null) {
      url doc.fetchUrl
    } else if (doc.url != null) {
      url doc.url
    } else if (doc.displayUrl != null) {
      url doc.displayUrl
    } else {
      url "URL-NOT-PROVIDED"
    }
 
    // ACLs
    if (doc.acls != null) {
       builder.acls() {
         $list {
           doc.acls.getChildren().each() { val ->
             $object() {
               name val.getAttribute("name")
               access val.getAttribute("access")
               entity val.getAttribute("entity")
             }
           }
         }
       }
     
     
    //END
    }
 
    if(doc.content != null) {
      content doc.content?.getText()
    }
 
  }
} else {
  /**********
   * Delete *
   **********/
 
  builder.$object() {
    '@search.action' "delete"
 
    String delId = "";
    // Get ID
    if (doc.id != null) {
      delId = getMD5(doc.id.getText())
    } else if (doc.fetchUrl != null) {
      delId = getMD5(doc.fetchUrl.getText())
    } else if (doc.url != null) {
      delId = getMD5(doc.url.getText())
    } else if (doc.displayUrl != null) {
      delId = getMD5(doc.displayUrl.getText())
    } else {
      delId = "ID-NOT-PROVIDED"
    }
     
    'id' delId
  }
}

Once you've clicked on the Add button, it will take a moment for Aspire to download all of the necessary components (the Jar files) from the Maven repository and load them into Aspire. Once that's done, the publisher will appear in the Workflow Tree.


Info

For details on using the Workflow section, please refer to Workflow introduction.