If you are to crawl a web site you will need to make sure you have the following points covered:
- The Aspire Server running the crawl must have access to the seed(s) URL (configured in the content source configuration).
- Check for any credentials needed to access the sites to be crawled. (Basic, Digest, HTTP forms and NTLM are supported)
Overview
Content Tools