Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 15 Next »

Beginning with release 3.0, Aspire uses an external MongoDB instance for connectors built with the new NoSQL Connector Framework. The database is used to keep crawl metadata and allow processing and scanning to be distributed. All MongoDB configuration is done in the settings.xml file. 

Basic Example:


  <!-- noSql database provider for the 3.1 connector framework -->
  <noSQLConnectionProvider sslEnabled="false" sslInvalidHostNameAllowed="false">
    <implementation>com.searchtechnologies.aspire:aspire-mongodb-provider</implementation>
    <servers>mongodb-host:27017</servers>
  </noSQLConnecitonProvider>
Aspire will create one MongoDB database for each content source configured. When the content source is deleted, the database will be dropped. The database name will the taken from the normalised value of the content source name. Starting in Aspire 3.1, the database names will be prefixed with "aspire-" to avoid possible conflicts of name. If you wish to change the prefix, add a "namespace" to the configuration:


  <!-- noSql database provider for the 3.1 connector framework -->
  <noSQLConnectionProvider sslEnabled="false" sslInvalidHostNameAllowed="false">
    <namespace>myNamespace</namespace>
    <implementation>com.searchtechnologies.aspire:aspire-mongodb-provider</implementation>
    <servers>mongodb-host:27017</servers>
  </noSQLConnecitonProvider>


Connect to a Multi-node MongoDB Installation

To connect to a multi-node MongoDB installation, you just need to provide a comma-separated list of hostname:port of the MongoDB nodes in the cluster.

Example:

  <!-- noSql database provider for the 3.1 connector framework -->
  <noSQLConnectionProvider sslEnabled="false" sslInvalidHostNameAllowed="false">
    <implementation>com.searchtechnologies.aspire:aspire-mongodb-provider</implementation>
    <servers>mongodb-host1:27017,mongodb-host2:27017,mongodb-host3:27017,mongodb-host4:27017</servers>
  </noSQLConnecitonProvider>

Using TLS/SSL

If you need to connect to a MongoDB configured to Use TLS/SSL you need to set the following attributes into the noSQLConnectionProvider tag:

AttributeValueDescription
sslEnabledtrueEnables the ssl on the Aspire MongoDB client
sslInvalidHostNameAllowedtrue/falseDisables the hostname verification from the SSL validation


You also need to include the certificate into a java keystore, to configure it follow the instructions on Crawling via HTTPs

X.509 Authentication

Aspire 3.1 only supports authenticating to MongoDB using X.509.

The X.509 mechanism authenticates a user whose name is derived from the distinguished subject name of the X.509 certificate presented by the driver during SSL negotiation. This authentication method requires the use of SSL connections with certificate validation.

To configure it, add the following to your settings.xml file:

  <!-- noSql database provider for the 3.1 connector framework -->
  <noSQLConnectionProvider sslEnabled="true" sslInvalidHostNameAllowed="false">
    <implementation>com.searchtechnologies.aspire:aspire-mongodb-provider</implementation>
    <servers>mongodb-host:27017</servers>
    <x509username>CN=user,OU=OrgUnit,O=myOrg</x509username>
  </noSQLConnecitonProvider>

Encrypt sensitive fields in MongoDB

If you want to be extra safe and encrypt the URLs, IDs, or any other metadata stored in MongoDB, you can do by specifying the name of the fields to encrypt:


  <!-- noSql database provider for the 3.1 connector framework -->
  <noSQLConnectionProvider sslEnabled="false" sslInvalidHostNameAllowed="false">
    <implementation>com.searchtechnologies.aspire:aspire-mongodb-provider</implementation>
    <servers>mongodb-host:27017</servers>
    <encryptFields>
      <field>_id</field> <!-- Encrypts all the IDs -->
      <field>url</field> <!-- Encrypts the url fields -->
      <field>fetchUrl</field> 
      <field>parentId</field> 
    </encryptFields>
  </noSQLConnecitonProvider>
  • No labels