Introduction

Box connector will crawl content from a Box repository. The connector will retrieve the supported elements using the RESTful API (Content API Basics 2.0 version), for authentication will use Box API (that uses OAuth 2). Using JWT configuration.

Environment and Access Requirements

Repository Support

The Box connector supports crawling the following the repositories:

User Account Requirements

To access Box repository, a user account with sufficient privileges must be supplied.

Box Application

To access Box APIs, you'll need to create an Box Application (Client Id and Client Secret)

Authentication 

The connector supports two types of authentication using a Box Application's Client Id and Client Secret (Client Credential) or using Json Web Token (JWT).

Client Credential Grant 

This server-side aithentication does not require end-user interaction and, if granted the proper privileges, can be used to act on behalf of any user in an enterprise, this is important to get all the content extracted. Identity is validated using the application's client ID and client secret.

See Box https://developer.box.com/guides/authentication/oauth2/oauth2-setup/ for the steps on how configure Custom App, 

Please, select Server Authentication (Client Credentials Grant) as authentication method. 

For Application Scopes, select 'Read all files and folders stored in Box', 'Manage user', 'Manage groups', 'Manage enterprise properties'.

JWT

Server-side authentication using JSON Web Tokens (JWT) does not require end-user interaction and, if granted the proper privileges, can be used to act on behalf of any user in an enterprise, this is important to get all the content extracted. Identity is validated using a JWT assertion and public/private keypair.

Create the Azure AD application

1. See Microsoft's Use portal to create an Azure Active Directory application and service principal that can access resources for the steps on how to create an Application ID properly and its key (Client Secret). Make sure to write down your Application Key at the time of creation. It will not be shown again after you exit the portal. 

Assign permissions to the application

1. Log into the Azure Management Portal.
2. Click on the “Azure Active Directory” option.
3. Select the “App registrations” option and then select your client application.

4. Select "API permissions" > "Add a permission" > "Microsoft APIs".

5. Select the “Microsoft Graph” option.
6. On Application Permissions, select the following:
   • Read files in all site collections (Files.Read.All)
   • Read and write files in all site collections (Files.ReadWrite.All)
   • Read all users’ full profiles (User.Read.All)
   • Read directory data (Directory.Read.All)
   • Read all groups (Group.Read.All)
   • Read and write items in all site collections (Sites.ReadWrite.All)
   • Read items in all site collections (Sites.Read.All)
7. Click on “Save”.
8. Click on “Grant admin consent for . . . ”
   • Click on “Yes” when prompted

Framework and Connector Features

Framework Features

Connector Features

Some of the features of the Box connector include:

• Ability to perform either full or incremental crawling (so that only new/updated documents are indexed), using stream_position.
• Possibility of exclude a folder or a set of folders and their content.
• Possibility of exclude or include elements (folders or files) by file name or folder name using regular expression (regex patterns)
• Metadata extraction
• it is search engine independent
• Runs from any machine with access to the given Box account
• Fetches access control lists (ACLs)

Content Crawled

The Box connector retrieves several types of objects:

Incremental Crawl

For incremental crawls, the connector will use the event stream position value, so only the changes reported from that time value to the current moment will be crawl. 

These are the list of events supported:

• ITEM_CREATE
• ITEM_UPLOAD
• COMMENT_CREATE
• ITEM_MOVE
• ITEM_COPY
• TASK_ASSIGNMENT_CREATE
• ITEM_TRASH
• ITEM_RENAME
• ITEM MODIFY
• TASK CREATE
• ITEM UNDELETED VIA TRASH

Identity Crawl

Folder colaborators, group memberships, and users are part of the ACLs and Identity Fetcher values which are part of the security of the content.