Page History

...

A resource provider provides access to a specific set of resources . This can be a from a particular storage technology. For example, a file system directory of resource files or a collection of tables.

Available Resource Provider

...

Implementations

• FileSystem Resource Provider
• MongoDB Resource Provider
• Elasticsearch Resource Provider (EXPECTED - UNDER CONSTRUCTION)

Note that only the FileSystem implementation is provided as part of the core system. Others are provided in separate Jar files which specifically included on the class path when needed.

Locating the Resource Provider Class

...

Types of Resources

Currently, there are two three types of resources defined:

...

• Blob
  • This resource contains a simple blob of binary data.
  • The typical use is to hold a JSON file, such as the pipeline configuration file.
• JSON Map
  • This resource contains a key/value map, where the key is a string and the value is a JSON record.
• String[] Map
  • This resource contains a key/value map, where the key is a string and the value is an array of strings.

A resource provider must provide methods to read and write all three types of resources.

Most resources are read-only, but some resources may need to be created by the pipeline stage which uses them. For example, to create a dictionary index.

...

Resource Providers Configuration

...