Page History
...
- ability to change version of template
- ability passing connectionId, connectorId, workflowId, policyId to seed, connection, credential
- the ability passing list of tags
- default values for properties which are not in 4.0.
- multiple Starting Points -> split to multiple seeds and connections, script read file and for each line create seed.
- ability to splitting url to connection and seed.
- ability to pass as parameter a suffix of http object description
- ability to work with relative path
- ability to use the crawl path/folder to the Seed name and host/url to the Connection name the by using an argument.
Repository
aspire-migration-component
Installation
For Developers
Python libraries used
os.path, json, urlparse, xmltodict, urllib3, zipfile, sys, datetime, logging, argparse
Step-by-step guide
Run Aspire 4.0 and Aspire 5.0
- Download zip file of connector from Aspire 4.0
- setup Aspire 5.0 url in file __init__.py, if you are using private certificates, needs to be setup also certificates
| Code Block |
|---|
URL_ASPIRE = "http://localhost:50505/aspire/_api"
# --- https with certificates
# CA_CERTS = ('client-2048.crt', 'client-2048.key')
# http = urllib3.PoolManager(cert_reqs='REQUIRED', ca_certs=CA_CERTS) |
4. Run script
| Code Block |
|---|
python __init__.py -p smb_connector-content-source.zip |
5. After few second connector is migrated to Aspire 5.0, objects "seed", "connection", "connector", "credential" created with description "migration-{date}".
You can also check the created report about migration "migration-{date}.log" which will be in the directory "reports" to see what exactly happened.
Help
We can get information about script by command
| Code Block |
|---|
python __init__.py -h |
Output
| Code Block |
|---|
usage: __init__.py [-h] [-d DESC] [-v VER] [-p PATH] [-t TAGS] [-i IDS]
Migration ASPIRE script. Migrate connector from ASPIRE version 4.0 to ASPIRE
version 5.0
options:
-h, --help show this help message and exit
-d DESC, --desc DESC Custom description of all migrated ASPIRE objects
(seed, connector, connection, credential)
-v VER, --ver VER Version of transformation matrix.
-p PATH, --path PATH Path to connector file zip.
-t TAGS, --tags TAGS Tags for seed.
-i IDS, --ids IDS Ids with should be used in seed or connection
Check log after migration. All is well that ends well. |
Ability to change version of template
Templates are in directory "migration_template"
The structure of the project looks similarily as an situation below.
-- migration
-- migration_template
-- 4.1
-- default
-- aspire-filesystem-source
-- aspire-sharepointonline-source
-- aspire-smb-source
-- type_transform_matrix.json
Connectors can have different properties related to their version 4.0, 4.1, 5.0, 5.1 etc. When we want to work with different version than "default", we need to create new directory there "4.1" everything from directory "default" needs to be copied to this new directory. Changes needs to be done in "*_transform_matrix.json" for current connector.
- Install Python 3.11
- Install Pycharm Comunity
In Pycharm go to File → Settings → Python Interpreter
Setup path to where is python
Install missing libraries urllib3, wheel, xmltodict, orjson by "+" button
Windows cmd
install Python 3.11
Update Path in environment variables:
C:\Users\#{username}\AppData\Local\Programs\Python\Python311\
C:\Users\#{username}\AppData\Local\Programs\Python\Python311\Scripts
install pip
python -m pip install --upgrade pip --trusted-host pypi.org
install virtualenv
pip install virtualenv --trusted-host pypi.org
virtualenv venv
Activate virtual env
venv\Scripts\activate
Install necessary libraries
pip install urllib3 --trusted-host pypi.org
pip install wheel --trusted-host pypi.org
pip install xmltodict --trusted-host pypi.org
pip install orjson --trusted-host pypi.org
Python libraries used
os.path, json, urlparse, xmltodict, urllib3, zipfile, sys, datetime, logging, argparse
Step-by-step guide
Run Aspire 4.0 and Aspire 5.0
- Download zip file of connector from Aspire 4.0
- setup Aspire 5.0 url in file __init__.py, if you are using private certificates, needs to be setup also certificates
| Code Block |
|---|
URL_ASPIRE = "http://localhost:50505/aspire/_api"
# --- https with certificates
# CA_CERTS = ('client-2048.crt', 'client-2048.key')
# http = urllib3.PoolManager(cert_reqs='REQUIRED', ca_certs=CA_CERTS) |
4. Run script
| Code Block |
|---|
python __init__.py -p smb_connector-content-source.zip |
5. After few second connector is migrated to Aspire 5.0, objects "seed", "connection", "connector", "credential" created with description "migration-{date}".
You can also check the created report about migration "migration-{date}.log" which will be in the directory "reports" to see what exactly happened.
Help
We can get information about script by command
| Code Block |
|---|
python __init__.py -h |
Output
| Code Block |
|---|
usage: __init__.py [-h] [-d DESC] [-v VER] [-p PATH] [-t TAGS] [-i IDS]
Migration ASPIRE script. Migrate connector from ASPIRE version 4.0 to ASPIRE
version 5.0
options:
-h, --help show this help message and exit
-d DESC, --desc DESC Custom description of all migrated ASPIRE objects
(seed, connector, connection, credential)
-v VER, --ver VER Version of transformation matrix.
-p PATH, --path PATH Path to connector file zip.
-t TAGS, --tags TAGS Tags for seed.
-i IDS, --ids IDS Ids with should be used in seed or connection
Check log after migration. All is well that ends well. |
Ability to change version of template
Templates are in directory "migration_template"
The structure of the project looks similarily as an situation below.
-- migration
-- migration_template
-- 4.1
-- default
-- aspire-filesystem-source
-- aspire-sharepointonline-source
-- aspire-smb-source
-- type_transform_matrix.json
Connectors can have different properties related to their version 4.0, 4.1, 5.0, 5.1 etc. When we want to work with different version than "default", we need to create new directory there "4.1" everything from directory "default" needs to be copied to this new directory. Changes needs to be done in "*_transform_matrix.json" for current connector.
After we are done with changes, After we are done with changes, script must be run with other parameter "-v [name of new directory]"
...
| Code Block |
|---|
{
"Seed":{
"workflows":[
"2745cfbe-0a2f-4d19-ac02-51e7c6d68898"
],
"connector":"a3ffa705-cc09-44f2-9233-ea6a5242491c",
"connection":"2f31049e-da32-45cf-af49-dbc203d00c75",
"deleteIncrementalPolicy":"e6165aa0-fb7d-490a-98da-f90032f0b886",
"throttlePolicy":"c080798a-fd43-484b-bafb-2891b555ab3e",
"routingPolicies":[
"c6ed5858-4e74-443d-a303-a33ff4099115"
]
},
"Connection":{
"credential":"aa56ec21-1459-469e-b347-c95199c2d95b",
"deleteIncrementalPolicy":"e6165aa0-fb7d-490a-98da-f90032f0b886",
"throttlePolicy":"c080798a-fd43-484b-bafb-2891b555ab3e",
"routingPolicies":[
"c6ed5858-4e74-443d-a303-a33ff4099115"
]
},
"Credential":{
"deleteIncrementalPolicythrottlePolicy":"e6165aa0-fb7d-490a-98da-f90032f0b886",
"throttlePolicy":"c080798a-fd43-484b-bafb-2891b555ab3e",
"routingPolicies":[
"c6ed5858-4e74-443d-a303-a33ff4099115"
]
},
}},
} |
This content of the file is only to show, what all of ids can be setup and in the real situation "connection" in "Seed" must be deleted if we want to also setup connection and same This content of the file is only to show, what all of ids can be setup and in the real situation "connection" in "Seed" must be deleted if we want to also setup connection and same for the situation credential in connection. Because these objects will be replaced and not created in script.
...
| Code Block |
|---|
#credential json body
{
"properties": {
"domain": "ad",
"username": "svcRS01VF4D1FS01_SMB",
"password": "encrypted:E52FA744FE5F255D982B6AEF12BCBA0F20A3E725DDD2D3D74A6FE5A70A931FCD"
},
"type": "smb",
"description": "migration smb-2023-01-26_11-58-48",
"deleteIncrementalPolicy": "e6165aa0-fb7d-490a-98da-f90032f0b886",
"throttlePolicy": "c080798a-fd43-484b-bafb-2891b555ab3e",
}
# connection "routingPolicies": [json body
{
"c6ed5858-4e74-443d-a303-a33ff4099115"
]
}
# connection json body
{
"type""type": "smb",
"description": "migration smb-2023-01-26_11-58-48",
"properties": {
"host": "smb://10.89.26.105:445/",
...
},
"credential": "babe0cc5-7223-4ed5-80c1-fa992d9a6a2f",
"deleteIncrementalPolicy": "e6165aa0-fb7d-490a-98da-f90032f0b886",
"throttlePolicy": "c080798a-fd43-484b-bafb-2891b555ab3e",
"routingPolicies": [
"c6ed5858-4e74-443d-a303-a33ff4099115"
]
}
# seed json body
{
"type": "smb",
"description": "migration smb-2023-01-26_11-58-48",
"connector": "ec658f0b-1ea1-407f-8b8c-1b5c38da09f8",
"connection": "e6cf3d33-1914-48c0-9b2f-89d45e8adcbf",
"seed": "/",
"properties": {
"seed": "/"
},
"tags": null,
"workflows": [
"2745cfbe-0a2f-4d19-ac02-51e7c6d68898"
],
"deleteIncrementalPolicy": "e6165aa0-fb7d-490a-98da-f90032f0b886",
"throttlePolicy": "c080798a-fd43-484b-bafb-2891b555ab3e",
"routingPolicies": [
"c6ed5858-4e74-443d-a303-a33ff4099115"
]
} |
...
| Code Block |
|---|
python __init.py__ -d migr -p smb_connector-content-source.zip |
Related articles
| Content by Label | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Ability to work with relative path
Migration xml files can contain relative path to other configuration files. Script can change automatically relative path to absolute path but must be run with parameter "-a [absolute path to aspire 4.0 distribution]"
| Code Block |
|---|
python __init.py__ -a C:\aspire-4.0 -p smb_connector-content-source.zip |
Ability to use the crawl path/folder to the Seed name and host/url to the Connection name the by using an argument
Script can change description to path/folder, host/url by run with parameter "-u"
| Code Block | ||
|---|---|---|
python __init.py__ -p smb_connector-content-source.zip -u | ||
| Page properties | ||
| ||
| Related issues |
Overview
Content Tools