Easy Heading Free | ||||||
---|---|---|---|---|---|---|
|
Name in script | Description | Aspire type | Init script | Process script |
---|---|---|---|---|
doc | Crawled document | AspireObject | false | true |
component | Aspire workflow component running Groovy scripts | ComponentImpl | true | true |
connection.client | REST client component for making AI calls | GenAIRestRequester | true | true |
utilities.azure.embeddings utilities.google.embeddings | Methods related to "embeddings" processing | Embeddings | true | true |
job | Job containing the crawled document | Job | false | true |
secrets | Map of secrets provided in UI | Map<String,String> | true | true |
template | Map of selected script template variables | Map<String,String> | true | true |
utilities.azure.prompts utilities.google.prompts | Methods related to "prompts" processing | Prompts | true | true |
utilities.textSplitter | Method related to text splitting | TextSplitterComponent | true | true |
variables | Map of variables provided in initialize script | Map<String,Object> | true | false |
utilities | Various helper methods | Utils | true | true |
utilities.azure.embeddings, utilities.google.embeddings |
---|
Method | Description | Syntax | Init script | Process script | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Initialize Azure | void utilities.azure.initialize(AspireObject config) | true | false | ||||||||||||||||
config:
| |||||||||||||||||||
Initialize Google Palm | void utilities.google.initialize(AspireObject config) | true | false | ||||||||||||||||
config:
| |||||||||||||||||||
Process | VectorEmbeddingResult utilities.azure.embeddings.process(List<String> splitText) VectorEmbeddingResult utilities.google.embeddings.process(List<String> splitText) VectorEmbeddingResult: TODO splitText: TODO | false | true | ||||||||||||||||
Convert response | VectorEmbeddingsResult utilities.azure.embeddings.convertResponse(String text, AspireObjectResponse response) VectorEmbeddingsResult utilities.google.embeddings.convertResponse(String text, AspireObjectResponse response) VectorEmbeddingsResult: TODO response: TODO | ||||||||||||||||||
Create POST body | AspireObject utilities.azure.embeddings.createPostBody(String text) | ||||||||||||||||||
Create Sub document | AspireObject utilities.azure.embeddings.createSubDoc(VectorEmbeddingsResult vectorEmbeddingsResult, AspireObject doc, int chunkCount) |
Example init script:
Code Block | ||
---|---|---|
| ||
import com.accenture.aspire.services.AspireObject;
utilities.textSplitter.initialize(getTextSplitterConfig("sentence"))
utilities.azure.embeddings.initialize(getEmbeddingsConfig())
def getEmbeddingsConfig() {
AspireObject returnValue = new AspireObject("config");
returnValue.add("endpoint", "${template.endpoint}");
returnValue.add("model", "${template.model}");
returnValue.add("apiVersion", "${template.apiVersion}");
returnValue.add("apiKey", "${secrets.apiKey}");
return returnValue;
}
def getTextSplitterConfig(String splitType) {
.....
} |
Example script:
Code Block | ||
---|---|---|
| ||
def sentences = utilities.textSplitter.process(doc);
embeddings = utilities.azure.embeddings.process(sentences);
doc.add(embeddings.toAspireObject()); |
Example script:
Code Block | ||
---|---|---|
| ||
import com.accenture.aspire.services.AspireException
// split field "content" and create "sentences"
def sentences = utilities.textSplitter.process(doc);
// url
endpointEmbeddings = "${template.endpoint}/openai/deployments/${template.model}/embeddings?api-version=${template.apiVersion}"
// generate and publish embeddings
sentences.eachWithIndex {currentSentence, sentencesCount ->
embeddingVector = getEmbeddingsFromSentence(endpointEmbeddings, currentSentence)
subJobAO = utilities.azure.embeddings.createSubDoc(embeddingVector, doc, sentencesCount);
utilities.createSubJob(job, subJobAO)
}
def getEmbeddingsFromSentence(endpointEmbeddings, sentence) {
response = connection.client.executePost(endpointEmbeddings, utilities.azure.embeddings.createPostBody(sentence));
def embeddings = utilities.azure.embeddings.convertResponse(sentence, response)
return embeddings
} |
Text splitter | utilities.textSplitter |
---|
Method | Description | Syntax | Init script | Process script | |||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Initialize | void utilities.textSplitter.initialize(AspireObject config) | true | false | ||||||||||||||||
config:
| |||||||||||||||||||
Process | List<String> utilities.textSplitter.process(AspireObject doc) List<String>: TODO doc: TODO | false | true |
Example init script:
Code Block | ||
---|---|---|
| ||
import com.accenture.aspire.services.AspireObject; utilities.textSplitter.initialize(getTextSplitterConfig("sentence")) def getTextSplitterConfig(String splitType) { AspireObject returnValue = new AspireObject("config"); returnValue.add("splitType", splitType); returnValue.add("fieldsToSplit", "content"); returnValue.add("customSplitRegex", "\\|+"); returnValue.add("characterThreshold", 4); return returnValue; } |
Example script:
Code Block | ||
---|---|---|
| ||
def sentences = utilities.textSplitter.process(doc); |