Page History
Table of Contents |
---|
General
Why is does an incremental crawl last as long as a full crawl?
The RDB via Snapshots Connector performs incremental crawls based on snapshot files, which are meant to match the exact documents that have been indexed by the connector to the search engine. On an incremental crawl the connector fully crawls the RDBMS content the same way as a full crawl, but it only indexes the modified, new or deleted documents during that crawl.
Common Errors
Connection errors
If you can't get a connection to the database, you'll see an exception similar to this:
Couldn't
get
an
RDB
connection
Code Block |
---|
com.searchtechnologies.aspire.services.AspireException: Unable to open connection to jdbc:postgresql://localhost/RDBMS_Test_DB (aspire_user) (component='/My_RDB_Source/RDBMSRDB', componentFactory='aspire-multiple-rdb')
at com.searchtechnologies.aspire.components.rdb.MultiRDBConnectionPool$ConnectionPool.createConnection(MultiRDBConnectionPool.java:535)
at com.searchtechnologies.aspire.components.rdb.MultiRDBConnectionPool$ConnectionPool.getConnection(MultiRDBConnectionPool.java:477)
at com.searchtechnologies.aspire.components.rdb.MultiRDBConnectionPool$ConnectionPool.access$000(MultiRDBConnectionPool.java:387)
at com.searchtechnologies.aspire.components.rdb.MultiRDBConnectionPool.getConnection(MultiRDBConnectionPool.java:131)
at com.searchtechnologies.aspire.components.rdbconnector.RDBScanner.doCrawl(RDBScanner.java:420)
at com.searchtechnologies.aspire.scanner.AbstractPushScanner.performScan(AbstractPushScanner.java:30)
at com.searchtechnologies.aspire.scanner.AbstractScanner.scanProcess(AbstractScanner.java:701)
at com.searchtechnologies.aspire.scanner.AbstractScanner.process(AbstractScanner.java:323)
at com.searchtechnologies.aspire.application.JobHandlerImpl.runNested(JobHandlerImpl.java:158)
at com.searchtechnologies.aspire.application.JobHandlerImpl.run(JobHandlerImpl.java:80)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:207)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:64)
at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:136)
at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:29)
at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:21)
at org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:31)
at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
at org.postgresql.Driver.makeConnection(Driver.java:410)
at org.postgresql.Driver.connect(Driver.java:280)
at com.searchtechnologies.aspire.components.rdb.MultiRDBConnectionPool$ConnectionPool.createConnection(MultiRDBConnectionPool.java:525)
... 12 more
Caused by: java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.connect0(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at org.postgresql.core.PGStream.<init>(PGStream.java:60)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:101)
... 21 more |
SQL Errors
Errors in the SQL configured for the job will produce exceptions such as:
Code Block |
---|
2014-02-10T19:54:07Z ERROR: Error executing select sql for full crawl (SELECT "id", "name", "company", "email", birthday FROM Test_table WHERE "id" <= 1000 AND MOD(id,10) = 9 )
org.postgresql.util.PSQLException: ERROR: relation "test_table" does not exist
Position: 57
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:560)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:417)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:302)
at com.searchtechnologies.aspire.components.rdbconnector.RDBScanner.executeSelectSQL(RDBScanner.java:57)
at com.searchtechnologies.aspire.components.rdbconnector.RDBScanner.doCrawl(RDBScanner.java:466)
at com.searchtechnologies.aspire.scanner.AbstractPushScanner.performScan(AbstractPushScanner.java:30)
at com.searchtechnologies.aspire.scanner.AbstractScanner.scanProcess(AbstractScanner.java:701)
at com.searchtechnologies.aspire.scanner.AbstractScanner.process(AbstractScanner.java:323)
at com.searchtechnologies.aspire.application.JobHandlerImpl.runNested(JobHandlerImpl.java:158)
at com.searchtechnologies.aspire.application.JobHandlerImpl.run(JobHandlerImpl.java:80)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source) |
Overview
Content Tools