The biggest change in Aspire 3.2 is related to the way
the
connectors work
, they
. They now use an external database (MongoDB) to hold all of the crawling informationsuch as document urls, status, statistics, snapshots (for incrementals), logs, etc.
The idea behind this change is to allow
This allows the connectors to work distributed from
its very
the architectural design.
Now all
All of the connectors run under the same principles, using the same logic, so that each connector is more like a Repository Access Provider
so we
. We keep them as simple as possible, rather than a complex (multi-threaded) crawling application
; so the
. The complexity of distributed crawling and multi-threading relies on the Connector Framework.
What's next?
Children Display
Responsibilities that the Connector developers
have to
implement:
Scan
the
the repository document containers to discover new documents to process