Connector could not be started for the following reason: failed to start connector

Answer

The error Connector could not be started for the following reason: failed to start connector is usually generated when the connector JVM fails to start.

When performing a Test-it, be sure to check the "Recrawl" option. Test-it tries very hard to use the crawler's cache. When Test-it uses the cache, it doesn't try to spin up the connector. Thus, in your case, if it is using the cache, then you won't see the same problem you see when trying to actually run a crawl or refresh.

Common causes

Below are some common causes of JVM startup problems:

Assigning too large of a maximum heap size to the JVM

On 32-bit platforms, the JVM will fail to start if you try to set the heap size larger than about 1.5 GiB, irregardless of how much physical memory is available. On all platforms, if you try to assign a heap size larger than the available physical memory, the JVM will fail to start. Double check to make sure the maximum heap size assigned to the JVM is smaller than the maximum amount of available physical memory.

This really only applies if you've changed the maximum heap size for the connector. For most connectors, you have to override and edit the seed function to change the maximum heap size. The default is 160 MiB, so make sure you have at least that much available physical memory.

Not enough disk space

You are correct in that a lack of disk space could cause the connector to not start-up properly. The connector requires a certain amount of temporary space available in order to run.

Incorrect permissions on the filesystem

Velocity needs write access to several directories in order to function properly. In this case, the JVM/connector needs write access to the locations it creates temporary files:

• The top-level Vivisimo tmp directory


• The search collection's crawl directory

Please note that Test-it uses a different crawl directory than a real crawl/refresh uses. If your problem is a permissions related issue, then this might explain why Test-it works but full crawl does not (Velocity has permissions to the Test-it crawl directory, but not to the search collection crawl directory).

Additionally, Velocity needs full read permissions on the connector files found in lib/java and lib/java/plugins. In particular, check the permissions on the connector plugin the crawl is using.

Other troubleshooting tips

If checking the above three common issues doesn't reveal the problem, then there are a few other things you can look at to try to find the root cause:

Check the connector-fatal-error log file

Sometimes startup errors are logged to the connector-fatal-error file found in the root of the crawl directory. The file is located in the collection's storage directory. If the
customer has not customized the search collection storage location, this should
be located at:

/data/search-collectio ns///crawl1/connector-fat al-error-

where:

;
• is a hash based on the collection name


• is the name of the collection


• is the protocol of the connector (e.g., smb, sharepoint, imap)

Turn on connector bootstrap logging

With startup problems, it's sometimes helpful to turn on connector bootstrap logging. If nothing else, this helps to determine if the problem is with the JVM or if the problem is with the connector startup process.

Specifically:

1. Enable connector bootstrap logging as described in this Overflow question.


2. Attempt to perform a crawl (or a test-it if you can reproduce the issue using test-it)


3. Look at the /tmp/viv_connector-log file.

If the file does not exist or is empty, then the problem is with the JVM itself. If the file is not empty, then the problem is probably related to the connector startup process. Look through the log file for errors (particularly lines marked with ERROR or FATAL).

Inspect the "standard out" temporary files

Sometimes, a JVM crash or startup failure is written to standard-out, and sometimes, standard-out is captured in a temporary file in /tmp.

1. Att empt to perform a crawl (or a test-it if you can reproduce the issue using test-it)


2. Sort the files in the /tmp by last modified date.


3. Starting with the most recent files, look at each file that starts with viv_, but is NOT:
   
   • A connector log file (viv_connector-*)
   
   
   • A shared pipe file (viv_conv_pipe_something), or any other file with the word "pipe" in it)

With any luck, you'll find a file that contains an error message relevant to the problem on hand.

Open a support case

Of course, if all else fails, contact Vivisimo support. Be sure to provide support with:

• Velocity version


• Platform (64-bit vs 32-bit)


• Operating System and full version (e.g. Windows Server 2003 SP2)


• Server specs (disk space, physical memory size, etc)


• The names (and possibly versions) of the connector(s) being used


• A list of troubleshooting steps you've already taken


• Any and all log files generated during your initial troubleshooting