The Database Server Connector can be configured using the Aspire Admin UI. It requires the following entities to be created:
Credential
Connection
Connector
Seed
Create Credential
On the Aspire Admin UI go to the credentials page
All existing credentials will be listed. Click on the new button
Enter the new credential description.
Select Database Server from the Type list.
General: In Credential type select between Basic Authentication and Keberos Authentication.
Basic authentication: Fill the gaps with your user and password.
Kerberos authentication: Fill username, Keytab File(path to the Keytab file), External Jars Path(the path of the folder where other needed files are located), Hadoop Resource Files(path to hadoop files).
Create Connection
On the Aspire Admin UI go to the connections page
All existing connections will be listed. Click on the new button
Enter the new connection description.
Select Database Server from the Type list.
General:
Server URL: The connection stream of the server.
JDBC driver: The path to the driver (.jar)
Specify JDBC Driver Class: Select this option in case the driver name is non-standard.
Specify Classpath: Select this option in case adtional drivers need to be uploaded and add the path of the folder which contains the aditional drivers.
Scan Options:
Stop scan on error: The scan will be stopped as soon as an error accurs during scanning.
Prefetch size: The number of items to be loaded in memory at a time.
Index DBs and tables: To index the metadata from the databases and tables. Choose between:
Extract table row count: to include the number of rows of the table.
Add tables schema: to include the tables structure.
Use query for table metadata extraction: A specific query to extract aditional data. ex.
select data from admin_table where table_id={{table}} and database_id={{database}}
Add resultSet to table job: adds the content of the tables to the job.
Enable row extraction: index all the rows in a table.
Limited extracted rows: Specify the number of rows to be extracted. Limit(Number of rows to be extracted). Perform Sampling(randomize the extracted elements)
Please keep in mind that you necessarily need to select Index DBs and tables or Enable row extraction for the connector to work properly. Please remember that both options are exclusive, so you will not be able to use the two of them at the same time.
Create Connector Instance
For the creation of the Connector object using the Admin UI check thispage.
Create Seed
On the Aspire Admin UI go to the seeds page
All existing seed will be listed. Click on the new button
Enter the new seed description.
Select Database Server from the Type list.
Scope: Exclude/include File: can be used to filter the crawled databases and tables. Add the path to the .json file where the excluded/included items are placed.
database:List of databases to exclude/include from the crawl.
name: name or pattern of the database to exclude/include.
pattern:specifies if the name parameter is a regex pattern or not.
table: List of tables to exclude/include from the crawl.
name: name or pattern of the table to exclude/include.
database:optional - name of the database that contains the table to exclude/include. If specified only the tables contained by the database will be filtered
pattern:specifies if the name parameter is a regex pattern or not.
schema: List of schemas to exclude/include from the crawl.
name: name or pattern of the schema to exclude/include.
database:optional - name of the database that contains the schema to exclude/include. If specified only the schemas contained by the database will be filtered
pattern:specifies if the name parameter is a regex pattern or not.