Table of Contents |
---|
The Aspire Web HDFS publisher was created and tested using version XX.
Before installing the Web HDFS publisher, make sure that:
In order to access Web HDFS an user account with sufficient privileges must be supplied. It is recommended the account be the site administrator.
...
...
The WebHDFS feature must be enabled in order to use this publisher.
...
Granting READ permissions is a must since the connector won't be able to get any data if the Path to be crawled is restricted.
...
For Kerberized Clusters, a delegation token is required in order to crawl any path within the HDFS. To obtain this token you must:
Run:
Code Block |
---|
$ kinit <your-user-principal>
$ curl -i --negotiate -u : "http://<host>:<port>/webhdfs/v1/?op=GETDELEGATIONTOKEN"
...
{"Token":{"urlString":"<A-VERY-LONG-TOKEN>"}} |
...