Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

In this section, we will review new content management features and how they can help you to manage your set of content sources. 

Content Sources

In this version of Aspire, all of the management can be done in the Home Page, where we have access to all the management features, and we can see all the content sources in their new presentation. Now each content source card displays more information, so the user can identify more easily which is the content source he is looking for, what is its current state, how many jobs it has crawled and how many errors were produced. Also includes the controls for copy, delete, crawl control, activate and deactivate.

 

Controls

For the new display of content sources Aspire 2.0 adds new features and information into card like objects that represents each content source, these cards contains several functions to control the content source, the following are all the controls of the card-content source

1. Content Source Icon

Indicates the type of source the content source is crawling from.

2. Content Source Name

The display name of the content source, if the name is to long the name will appear follow by an ellipsis (e.g. ABC...), and if the mouse is over the name, a tooltip will appear with the full name. The Content Source Name is also a link to the configuration of the respective content source.

3. Content Source Status

Indicates the current status of the content source. When the status changes not only does the label change, the color of the content source card changes.

4. Time of Crawl

Indicates the length of time the crawl has been crawling, or how long it took the crawl to finish. In case the content source has never being started the label will be Not Started. If you put the mouse over the Time of Crawl control, it will show a tooltip in your local timezone, of the exact date and time the content source was started.

Image Removed

 

 

 

 

 

 

5. Type of Crawl

With this control you can choose the type of crawl you want to do by clicking on it. The default is incremental, with one click you choose Full and with two clicks you choose Test (Test is a new type of crawl to test if the configuration of the content source is the correct one, to learn how to do a Test crawl please go to Test Scan).

6. Completed Jobs

Completed jobs shows the total number of successful jobs, you can also click on Complete and open the statistics.

7. Errors

Errors will show the number of document errors if any; if there is at least one error, you can click on the number and go to the Error Page.

8. Start/Pause/Resume Button

This button starts the crawl, if the crawl is already started you can click to pause it. If the crawl is paused you can resume the crawl. The look of the button changes with the state.

9. Stop/Abort Button

This button stops the crawl if the job is already crawling or paused, it will abort the crawl if the content source if pausing, stopping or resuming.

10. Active/Deactive Button

The Active button will activate or deactivate the content source, you can only deactivate the content source if it's status is new, crawled or in error.

11. Copy

Just clicking on the copy button you can create a copy of the content source.

12. Delete

If you click the delete button a confirmation will appear, if you click OK the delete will proceed and the content source will be deleted.

States

The content sources can be in one of several states, in each state some controls change and some are disabled. In the section above we saw the controls of a content source, in this section we will see which are the controls for each state.
* The copy and delete button are always enabled

Loading

When the content source is in loading state, it means it is downloading and/or initializing the content source, and it will change to the New state. during this status the whole content source will be completely disabled.

CS-Loading.pngImage Removed 

New/Crawled

The Crawled status indicates that the crawl or stopping the content source was successful, for this state the Content Source Status (1) changes to Crawled, the Time of Crawl (2) changes to the exact time in your local time zone when the crawl was started and how long it took to reach this state. The Start/Pause/Resume Button (3) will be set to start a crawl and the Stop/Abort Button (4) will be disabled, and the Active checkbox (5) will be checked and enabled.

* The New state was the configuration of the content source card we saw above in the controls section, it has all the controls enabled except for the stop button.

CS-Crawled.pngImage Removed 

Running

The Running status indicates that a crawl is currently in progress, for this state the Content Source Status (1) changes to Running and the content source will change color to green, the Time of Crawl (2) changes to the exact time in your local time zone when the crawl was started and with each refresh it will increase the total time that it's been crawling. The Type of Crawl (3) will be set to Incremental, the Completed Jobs number (4) will start to increase and the Errors (5) will show the number of errors at that moment, if any. The Start/Pause/Resume Button and the Stop/Abort Button (6) will be set for pause and stop, both enabled. And the Active checkbox (7) will be disabled.

CS-Running.pngImage Removed 

Paused

The Paused status indicates that a crawl is currently paused, for this state the Content Source Status (1) changes to Paused and the content source will change color to blue, the Time of Crawl (2) changes to the exact time in your local time zone when the crawl was started and with each refresh it will still increase the total time that it's been crawling. The Start/Pause/Resume Button and the Stop/Abort Button (3) will be set for resume and stop, both enabled. And the Active checkbox (4) will be disabled.

CS-Paused.pngImage Removed 

Error/Failed/Aborted

The Error status indicates that a crawl finished in an unsuccessful crawl, for this state the Content Source Status (1) changes to Error (or Failed or Aborted)and the content source will change color to red (The Content Source Status will also be a link to see the cause of the unsuccessful crawl), the Time of Crawl (2) changes


Content Sources


All of the management of Content Sources can be done in the Admin UI Content Sources, where you can access all of the management features.

  • You can see all the content sources in their new presentation.
  • Each content source card displays further detailed information.
  • You can identify which is the content source you want including its current state, how many jobs it has crawled, and how many errors were produced.
  • You can access the controls for copy, delete, crawl control, activate, and deactivate.


Image Added

Anchor
Content+Source+Control
Content+Source+Control

Controls


For the display of content sources Aspire adds new features and information into card like objects that represents each content source.


Image Added

The cards contain several functions to control the content source.

  1. Content Source Name & Icon
    • The display name of the content source, if the name is to long the name will appear follow by an ellipsis (e.g. ABC...), and if the mouse is over the name, a tooltip will appear with the full name. The Content Source Name is also a link to the configuration of the respective content source.
  2. Content Source Status
    • Indicates the current status of the content source. When the status changes not only does the label change, the color of the content source card changes.
  3. Time of Crawl
    • Indicates the length of time the crawl has been crawling, or how long it took the crawl to finish. In case the content source has never being started the label will be Never Executed. If you put the mouse over the Time of Crawl control, it will show a tool-tip in your local timezone, of the exact date and time the content source was started.
  4. Jobs Done
    • Completed jobs shows the total number of successful jobs, you can also click on Complete and open the statistics.

  5. Errors

    • Errors will show the number of document errors if any; if there is at least one error, you can click on the number and go to the Error Page.

  6. Statistics

    • It will show all the information available about the current crawl, including crawl type, documents per second, start time, jobs status, etc. 

  7. Start Full Crawl

    • Starts a full crawl, but before that it will show a warning indicating all incremental indexing data will be deleted.

  8. Start Incremental Crawl

    • Starts an incremental crawl.

  9. Start Test Crawl

    • Starts a Test Crawl that will ask how many documents to crawl and how many to skip before it starts to crawl.
  10. Copy

    • Creates a new content source with the same configuration.
  11. Export

    • Downloads a zip file with the configuration of the content source that you can use to import it into a different aspire instance.
  12. Enable/Disable

    • It will enable or disable the content source. (you can only disable the content source if it's status is new, completed, error, failed or canceled)
  13. Delete

    • If you click the delete button a confirmation will appear, if you click OK the delete will proceed and the content source will be deleted.


States


The content sources can be in one of several states. In each state, some controls change and some are disabled. In the section above, we saw the controls of a content source. In this section, we will see which are the controls for each state.


New / Completed / Cancelled

The Completed / Cancelled status indicates that the crawl or stopping the content source was successful, for these states the Time of Crawl (1) changes to the exact time in your local time zone when the crawl was started and how long it took to reach this state. 

Image Added


Running

The Running status indicates that a crawl is currently in progress, for this state the content source will change color to green, the Time of Crawl changes to the exact time in your local time zone when the crawl was started and with each refresh it will increase the total time that it's been crawling. The Jobs Done number will start to increase and the Errors will show the number of errors at that moment, if any. The Pause and the Stop button will replace the start crawls buttons.

Image Added


Paused

The Paused status indicates that a crawl is currently paused, for this state the content source will change color to blue, the Time of Crawl will still increase the total time that it's been crawling. The Resume button will replace the Pause button.

Image Added


Error / Failed / Aborted

The Error status indicates that a crawl finished in an unsuccessful crawl, for these states the content source will change color to red (The Content Source Status will also be a link to see the cause of the unsuccessful crawl), the Time of Crawl will stop updating the total crawl time. The start crawls Buttons will be set again.

Info

The Failed status appears when the content source fails in the initialize phase. The Content Source Status will also be a link to see the reason of the failure

Info

The Aborted status appears when the content source was aborted by the user



Image Added

Pausing / Stopping / Resuming

This state indicates a change from one state to another, and they are the only ones in which the user can do an abort.

For these states the content source will change color to yellow, The Abort button will be set

Info
  • The Pausing status indicates that a crawl is currently trying to pause the content source
  • The Stopping status indicates that a crawl is currently trying to stop the content source
  • The Resuming status indicates that a crawl is currently trying to start the crawl again


Disabled

The Disabled status indicates that a crawl is currently disabled and it will not perform a crawl, for this state the content source will change color to gray, The start crawls buttons will be disabled.

Image Added

Grouping

it will stop updating the total crawl time. The Start/Pause/Resume Button (3) will be set for start a crawl and the Stop/Abort Button (4) will be disabled, and the Active checkbox (5) will be checked and enabled. * The Failed status appears when the content source fails in the initialize phase. The Content Source Status will also be a link to see the reason of the failure

** The Aborted status appears when the content source was aborted by the user

CS-Error-Failed.pngImage Removed 

Pausing/Stopping/Resuming (Transitive)

This state indicates a change from one state to another, and they are the only ones in which the user can do an abort.

The Pausing status indicates that a crawl is currently trying to pause the content source, for this state the Content Source Status (1) changes to Pausing (or Stopping or Resuming)and the content source will change color to yellow, The Start/Pause/Resume Button (2) will be disabled and the Stop/Abort Button (3) will be set to abort, and the Active checkbox (4) will be disabled.

* The Stopping status indicates that a crawl is currently trying to stop the content source

** The Resuming status indicates that a crawl is currently trying to start the crawl again

CS-Pausing-Transitive.pngImage Removed 

Inactive

The Inactive status indicates that a crawl is currently inactive and it will not perform a crawl, for this state the Content Source Status (1) changes to Inactive and the content source will change color to gray, The Start/Pause/Resume Button (2) will be disabled as well as the Stop/Abort Button (3), and the Active checkbox (4) will unchecked and enabled.

Grouping

Aspire UI now includes the ability to group content source, with this new feature we can group the content source in any way we want, such as by category, content source type,...

Group


A group has the same shape as a content source (card-like), but its content is different

, in

. In the image below, we

have

see all of the controls that a group will have.

Image Added

1.
  1. Group Name

    • The display name of the group, if the name is to long the name will appear follow by an ellipsis (e.g. ABC...), and if the mouse is over the name, a

tooltip
    • tool-tip will appear with the full name. The Group Name is also a link that will expand the content source, so we can see the content source inside it.

2.
  1. Number of Content Sources

    • The group will display the number of content sources that has contained.

  2. Expand

    • If clicked, it will expand the content source, so we can see the content source inside it.

3. Number of Content Sources

The group will display the number of content sources that has contained.

4.
  1. Add to Group

    • If clicked, Add to Group will open the group menu, and put the name of the group so we can only choose the content sources and click on Add Group.

  2. Content Sources Status

    • Has well as the number of content sources the group will have the status of the content sources and how many content sources inside him has that specific status

      1. Green: Stands for the Running status.

      2. Blue: Stands for the Paused status.

      3. Red: Stands for the three unsuccessful status Error, Failed and Aborted.

      4. Orange: Stands for the three transitory status Pausing, Resuming and Stopping.

      5. White

: Stands for the idle status New and Crawled.
  • Gray: Stands fot the Inactive status.
  • 5. Add to Group

    If clicked, Add to Group will open the group menu, and put the name of the group so we can only choose the content sources and click on Add Group.

    6.
        1. : Stands for the idle status New and Completed and Cancelled.

        2. Gray: Stands for the Disabled status.

    1. Ungroup

      • If clicked, Ungroup will take all the content sources inside the group and put them in the first level (root), and it will delete the group.


    Manage a Group

    This section walks through the steps necessary to create, use and dispose of a group, and how to interact with the group itself.

    Step 1: Select the

    Content Sources

    content sources

    Click

    First of all we need to click on the Group button in the Action the Action Bar, this will open the grouping menua text field will appear, and change the bottom part of all the content sources will change into just a checkbox check box that says Group. Now we can select Select the content sources we you want to group together , once this is done we can by checking the Group check box, then put the name of the group in the test text field next to the Add Group button.

    Image Added

    Tip
    *
    • Once the group is created, if you want to add another content source to the group, you can do it by click
    the
    • the Add One
    button CSM-AddOne-Button.pngImage Removed and
    •  button Image Added and repeating steps 1 and
    two
    • 2
    • If you want to cancel the creation of a group by clicking the X in the text field

    Step 2: Create the

    Group

    group

    Once you have selected the content sources and filled the name, you can click on Add Group, this will fade out and in the content source, and a group card will appear at the end of the content sources. This will be the group you just create with all the selected content sources inside it.

    Step 3: Expand the

    Group

    group

    Once we have the group created we can expand it by clicking on the Group Name or by clicking on the expand button CSM-Expand-Button.pngImage Removed Image Added, this will fade out and in the content sources and display only the content sources inside the group. Also a legend will appear in the Action Bar indicating in which group we are in now, right next to the legend is the Return turn button CSM-ReturnUngroup-Button.pngImage Removed Image Added, if weclick we click on it, it will return us to the first level (root).

    Also while we are in the expanded group we can see that the content source has another button right before the copy button, this button is the Ungroup One button CSM-ReturnUngroup-Button.pngImage Removed Image Added, it looks exactly as the Return button CSM-ReturnUngroup-Button.pngImage Removed Image Added, this button will remove the content source from the group and put it on the first level (root)

    Image Added

    Step 4: Remove the

    Group

    group

    If you want to remove a group, the only way is to ungroup the entire group by clicking the Ungroup button CSM-Ungroup-Button.pngImage Removed Image Added, this will move all the content sources from our group to the first level (root) and delete the group.

    Image RemovedImage RemovedImage RemovedImage Removed

    Image Removed


    Cookie Filters

    The cookie filters are regular filters that will be saved in a cookie so once we apply a cookie filter, this will be active until we remove it. We can access the cookie filters by clicking on the Filter button CSM-FilteringButton1.pngImage Removed, and when you activate a filter the Filter button will highlight CSM-FilteringButton2.pngImage Removed. Image Added,

    We have three ( 3 ) categories of cookie filters , all of then explained below.

    Image Added

    General

    Filters

    filters

    We have three ( 3 ) general filters:

    1. Active: (Checked by default ) If  If checked shows all the content sources that are activeInactive: active (Checked by default )
    2. Groups:  If checked shows all the groups. (Checked by default)
    3. Inactive:  If If checked shows all the content sources that are inactiveGroups: inactive (Checked by default) If checked shows all the groups.
    CSM-Filtering1.pngImage Removed

    Status

    Filters

    filters

    The status filters will has all the possible status for the content sources, also it will have the All filter (Checked by default), if any of the status filters is checked, it will unchecked the All filter and only the content sources with matching the matching checked status will be displayed. If the All filter is checked again, it will unchecked all the status filters.

    CSM-Filtering2.pngImage RemovedImage Added

    Connector

    Filters

    filters

    The connector filters will be build according to the types of content sources we have available for the aspire account, but it will always will have the the All filter (Checked by default), if any of the connector filters is checked, it will unchecked the All filter and only the content sources with matching the matching checked connector types will be displayed. If the All filter is checked again, it will unchecked all the connector filters.

    CSM-Filtering3.pngImage RemovedImage Added

    Time

    Filters

    filters

    The time filters can be applied for start and end time of the crawl.

    • Start Time: (Unchecked by default) Compares the time given with the start time of the content source, if the time given is after or the exact time of the content source, the content source will be display, if we check the Start Time filter but we don't give a start time, the filter won't be applied. Also if the content source doesn't have a start time, the content source won't be displayed.'t be displayed.
      • To set the date and time of the filter click on the calendar button (2)
      • To enable the filter check the filter (1)
         
    • End Time: (Unchecked by default) Compares the time given with the end time of the content source, if the time given is before or the exact time of the content source, the content source will be display, if we check the End Time filter but we don't give a start time, the filter won't be applied. Also if the content source doesn't have a end time, the content source won't be displayed.
      • To set the date and time of the filter click on the calendar button (2)
      • To enable the filter check the filter (1)

    Image Added 

    Import
    Anchor
    Import

    CSM-Filtering4.pngImage Removed CSM-Filtering5.pngImage Removed

    Import


    In Aspire Admin UI 2.0 , we can import a content source zip file and load the content source directly to our management page. For this we just need to click on the Import button in the Action Bar, this will open a browse window where wecan search for our content source zip file, select it and click open. Automatically the content source will be load into the management page.

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

    To import a content source do the following steps:

    1. Click on Import in the Action Bar
    2. Use the browse window to find and select the zip file
    3. Click on Open to import 

    Image Added

    Once you have clicked on open, the content source will appear in the screen as loading with a warning icon (1), which means the source icon is missing.

    Image Added

    After it has loaded the content source will appear as New, and the source icon (1) will show up 

    Image Added

    *
    Info

    The import can only be successful if the zip file contains all

    four (

    4

    )

    necessary files, and this files are correctly formatted.


    ** This is functionality is only enable for some browsers, at the moment, please see Browser Compatibility.

     

    Add Source


    With Add Source we can choose , select the type of content source we you want to do by choosing the connector. The Add Source menu has three (3) main sections:.

     Image Added

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

     

    1. Add Source
      • Access the Source Menu from the Add Source button
    2. Legacy Connectors
      • These connector will be identified with a LEGACY label, which means these connector haven't been updated to use the new connector framework.
    3. New Connectors
      • These connectors have been updated to use the new connector framework. 
    4. . Artifact Id
      • Indicates the maven Artifact Id of the connector.
    5. Legacy Label
      • The Legacy label will indicate which connectors are not updated to use the new connector framework.
    6. Custom
    7. Official Connectors: Is the lost of all the connector created by Search Technologies for Aspire. Just by clicking on the connector we want to use, we will be redirect to the configuration page. The Official connector may change according to your connector entitlements.
    8. Custom Connectors:
        • It will open a menu to install a custom connector from maven coordinates or a set of config files.
      • Refresh Sources: 
        • With the Refresh button we can update the list of the connectors available for us.
    Note

    The Official connector may change according to your connector entitlements.

    Custom Connector

    By clicking on the Custom Connectors button, we will Click Custom to open a window where we you can choose between two methods to install a custom connector , repository and configuration files, both . Both show as toggle buttons on at the top of the window.

    Repository

    The repository method is always the default

    one

    ,

    with

    With this option

    we

    , you can download the custom connector from a

    maven

    Maven repository. To install the custom connector

    with need to Group ID:

    , fill the following fields

    :

    .


    1. Repository Tab
      1. Indicates the method currently being use to add a new connector
    2. Group Id
      • The groupId of the maven artifact
      • e.g. com.searchtechnologies.aspire
    1. Artifact
    ID:
    1. Id
      • The
    id
      • artifactId of the
    artifact representing the connector
      • maven artifact representing the connector 
      • e.g. app-custom-connector
    1. Version
    :
    1. (Optional)
      • If the version of the artifact isn't specify, Aspire will use the same version as it.
    2. OK Button
      • Click to load the connector. This may take a few seconds.
    3. Cancel
      • You can close the window by either clicking on the X or clicking on Cancel



    Image Added

     

    *
    Info

    All the connectors added using this method will be added to

    the

    the Add Source

    menu

     menu.

    Warning
    **

    It is not recommended to use an older version of a connector is a new version is available.

    Configuration Files

    Before accessing the configuration file method, an alert will

    be show indicating

    indicate that the

    connector

    connectors added using this method

    are

    will not

    going to  

    be included in the Add Source menu.

     

     

     

     

     


    Image Added


    The configuration files method requires

    to have

    both the application file and

    dxf

    DXF file in the Aspire server. To install a custom connector using this method

    we just have to

    , specify the direction of the application file.


    1. FileTab
      1. Indicates the method currently being use to add a new connector
    2. File Path
      • Path to the xml File
      • e.g. config/application.xml
    3. OK Button
      • Click to load the connector. This may take a few seconds.
    4. Cancel
      • You can close the window by either clicking on the X or clicking on Cancel

    Image Added

    Note
    • The dxf file must be call as the application xml file with the dxf suffix (e.g. application-dxf.xml)
    • And it must be in the same folder as the application.xml
    Warning

    If the dxf file doesn't have the new valid format for connectors, it won't be possible to configure the connector.

    application file.

     

     

     

     

     

     

     

     

    * If the dxf file doesn't have the new valid format for connectors, it won't be possible to configure the connector.

    Loaded Custom Connectors

    In case that a custom connector was added without an icon, and alert icon will appear before the connector's name.

     

     

     

     

    If the added custom connector doesn't exist, the alert icon will be appear before the content source name and the icon option will be disabled.

     

    CSM-Custom-Connector2.pngImage Removed