Language processing requires linguistic resources. This includes:
- Dictionaries of word variations (e.g. for lemmatizers)
- Dictionaries of names, places, products, etc.
- The pipeline configuration itself
The Language Processing Toolkit has a "Resource Management" system for reading and using these resources for language processing.
Goals:
- Separated storage layer
- Allow for resources to be stored in files or different database systems
- Isolate storage details from pipeline functionality
- Change providers without changing pipeline configuration
Resource Providers
Resources
Overview
Content Tools