- Created by Johnny Vargas on Jun 27, 2018
The IBM Connections Scanner component performs full and incremental scans over a IBM Connections Site, the timestamp attribute of the Seedlist's (Activities, Blogs, Files, Forums, etc) are saved to performed the incremental crawl if necessary. Updated content is then submitted to the configured pipeline in AspireObjects attached to Jobs. As well as the URL of the changed item, the AspireObject will also contain metadata extracted from the repository. Updated content is split in to three types: add, update, and delete. Each type of content is published on a different event so that it may be handled by different Aspire pipelines
The scanner reacts to an incoming job. This job may instruct the scanner to start, stop, pause or resume. Typically the start job will contain all information required by the job to perform the crawl. However, the scanner can be configured with default values via application.xml file. When pausing or stopping, the scanner will wait until all the jobs it published have completed before itself completing.
| IBM Connections Scanner (Aspire 2) | |
|---|---|
| Factory Name | com.searchtechnologies.aspire:aspire-ibmconnections-connector |
| subType | default |
| Inputs | AspireObject from a content source submitter holding all the information required for a crawl |
| Outputs | Jobs from the crawl |
Input Job Parameters
The scanner recognizes the following parameters on the control Job:
| Element | Type | Description | |
|---|---|---|---|
| id | int | The database id from the aspire_content_sources table. | |
| @crawlId | int | The crawl id from the content database. | |
| @action | String | The control job action - start, stop, pause or resume. | |
| @actionProperties | String | Properties for the crawl - full or incremental. | |
| connectorSource/IBMServer | String | The Url of the IBM Connection server to crawl (you have to specify the protocol). | |
| connectorSource/IBMUser | String | The Username to connect with. | |
| connectorSource/IBMPassword | String | The password of the Username to connect with. | |
| connectorSource/IBMTimeStampDir | String | Path of the directory used to save the timestamp values, later required to performed an incremental crawl. If neccessary. | |
| connectorSource/Page Size | Integer | Specifies the number of entries per page to return in the crawl | |
| connectorSource/useSSL | Boolean | true if the connector is going to use authentication to connect. | |
| connectorSource/IBMSSLDir | String | Path of the certificate to the websphere trust store. If required. | |
| connectorSource/IBMSSLPassword | String | Password of the certificate. | |
| connectorSource/crawlAllApps | Boolean | true if the connector will crawl all available applications (Activities, Blogs, Bookmarks, Communities, Files, Forums, Profiles, and Wikis) | false You have to choose which applications are going to be crawled. |
Configuration
The scanner recognizes the following configuration parameters:
| Element | Type | Default | Description |
|---|---|---|---|
| IBMServer | String | The Url of the IBM Connection server to crawl (you have to specify the protocol). | |
| IBMUser | String | The Username to connect with. | |
| IBMPassword | String | The password of the Username to connect with. | |
| IBMTimeStampDir | String | Path of the directory used to save the timestamp values, later required to performed an incremental crawl. If neccessary. | |
| Page Size | Integer | Specifies the number of entries per page to return in the crawl | |
| useSSL | Boolean | false | true if the connector is going to use authentication to connect. |
| IBMSSLDir | String | Path of the certificate to the websphere trust store. If required. | |
| IBMSSLPassword | String | Password of the certificate. | |
| crawlAllApps | Boolean | false The user will be able to select only the applications that he wants to crawl. | true if the connector will crawl all available applications (Activities, Blogs, Bookmarks, Communities, Files, Forums, Profiles, and Wikis) |
Branch Configuration
This component publishes to the onAdd, onDelete and onUpdate, so a branch must be configured for each of these three events.
| Element | Type | Description |
|---|---|---|
| branches/branch/@event | string | The event to configure - onAdd, onDelete or onUpdate. |
| branches/branch/@pipelineManager | string | The name of the pipeline manager to publish to. Can be relative. |
| branches/branch/@pipeline | string | The name of the pipeline to publish to. If missing, publishes to the default pipeline for the pipeline manager. |
Example Configuration
<component name="IBMScanner" subType="default" factoryName="aspire-ibmconnections-connector">
<debug>${debug}</debug>
<snapshotDir>${IBMsnapshotDir}</snapshotDir>
<updaterComponent>../JobStatusUpdater</updaterComponent>
<branches>
<branch event="onAdd" pipelineManager="../ProcessPipelineManager"
pipeline="add-update-pipeline" allowRemote="true" batching="true"
batchSize="50" batchTimeout="60000" simultaneousBatches="2" />
<branch event="onUpdate" pipelineManager="../ProcessPipelineManager"
pipeline="add-update-pipeline" allowRemote="true" batching="true"
batchSize="50" batchTimeout="60000" simultaneousBatches="2" />
<branch event="onDelete" pipelineManager="../ProcessPipelineManager"
pipeline="post-to-search-engine-pipeline" allowRemote="true"
batching="true" batchSize="50" batchTimeout="60000"
simultaneousBatches="2" />
</branches>
</component>
- No labels
Overview
Content Tools