Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The RDB via Snapshot Connector can be configured using the Aspire Admin UI. It requires the following entities to be created:

  • Credential
  • Connection
  • Connector
  • Seed

Bellow are the examples of how to create the Credential, Connection and the Seed. For the Connector please check this page.

Easy Heading Free
navigationTitleOn this Page
navigationExpandOptionexpand-all-by-default

Create Credential


  1. On the Aspire Admin UI go to the credentials tab 
  2. All existing credential will be list. Click on the new button 
  3. Enter the new credential description. 
  4. Select Rdb-Snapshot from the Type list.
  5. Enter User and Password. Note: The password will be automatically encrypted by Aspire.

Note: we are not going to use this credential, we will use SA instead, creating this because credential is required in connection config) 

Create Connection

  1. On the Aspire Admin UI go to the connections tab 
  2. All existing connection will be list. Click on the new button 
  3. Enter the new connection description. 
  4. Select Rdb-Snapshot from the Type list.
  5. JDBC Url -  jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId={DATABASE};OAuthServiceAcctEmail=ex-223163-prd-mktplcinteg@prd-223163-mkplace-bd-de4f8262.iam.gserviceaccount.com;OAuthType=0;OAuthPvtKeyPath=/home/dliu/projects/dmp/bigquery-lib/GCP_BQ_SA.json 

    Note - {DATABASE} value will be read from seed "Database name" field. 

  6. JDBC Driver Jar  /home/dliu/projects/dmp/baseline/assets/aspire-distribution-archetype-5.0.4-distribution/drivers/GoogleBigQueryJDBC42.jar 

  7. Specify Classpath  /home/dliu/projects/dmp/bigquery-lib 

  8. etc

The description of all "connection" fields  can be found here.

Note: The JDBC driver class can be found in attachments - bigquery-lib.zip

Create Connector


For the creation of the Connector object using the Admin UI check this page


Create Seed


  1. On the Aspire Admin UI go to the seeds tab 
  2. All existing seeds will be list. Click on the new button 
  3. Enter the new seed description. 
  4. Select Rdb-Snapshot from the Type list.
  5. Database name field: The name of the database. It will replace the marker {DATABASE} used in the field jdbcUrl of connection : prd-223163-mkplace-bd-de4f8262  (value of project id) 
  6. Full SQL:  SELECT object_type, object_name, parent_object_name, access_id, access_type, access_role, status, environment, processed_time, expiration_dttm  FROM `prd-65343-datalake-bd-88394358`.dp_resources_datamng_65343_in.dp_data_products_access  WHERE processed_date='2022-12-16' and access_type='security_group' and   status='enabled' and   environment = 'PRD' and  object_type = 'power_bi' 

  7. ID column: access_id (or TBD) 
  8. etc objectetc.

The description of all "seed" fields can be found here.