# Advanced Configuration for Connector Manager v2.4.4 and Later #

Connector Manager version 2.4.4 moves most of the configuration
options that a Connector administrator may wish to set out of the
web application bean definition file and into a separate application
properties file.

This change makes the upgrade process much less painful for those
administrators have have tailored the Connector Manager deployment
to their enterprise.  The web application bean definition file is
redeployed during the upgrade process, overwriting any modifications
that an administrator may have made.   However, the web application
properties file is not overwritten during an upgrade, so setting
common configuration properties in that file preserves, preserves
those customizations through future upgrades.

It is strongly suggested that the administrator only set advanced
properties whose values differ from the default values mentioned
below.  This allows Google to tune the defaults in later releases
to the benefit of all who had not explicitly overridden them.


## Setting or Modifying Advanced Configuration Properties ##

To set or modify Connector Manager Advanced Configuration
Properties, you must edit its `applicationContext.properties` file.
This [Java Properties](http://java.sun.com/j2se/1.5.0/docs/api/java/util/Properties.html) file
is a plain-text file in `ISO-8859-1` character encoding,
and must remain so when modified.  The syntax for setting
property values must conform the Java Properties specification.
The Connector Manager supports only plain-text properties files,
not XML-formatted properties files.

  1. Shutdown the Connector's Tomcat server.<br><br>
<ol><li>Make a backup copy of the file:<br><code>$TOMCAT_HOME/webapps/connector-manager/WEB-INF/applicationContext.properties</code><br><br>
</li><li>Edit the file:<br><code>$TOMCAT_HOME/webapps/connector-manager/WEB-INF/applicationContext.properties</code><br><br>
</li><li>Make the necessary modifications (see below) and save the file.<br><br>
</li><li>Restart the Connector's Tomcat server.<br><br>
</li><li>Examine the logs in <code>$TOMCAT_HOME/logs</code>, looking for any errors that may have been generated by mis-configured properties.</li></ol>

<b>Note:</b> <code>$TOMCAT_HOME</code> represents the Apache Tomcat installation directory. For Connectors installed using the Google Connector Installer (GCI), this would be the Tomcat directory in the Connector Installation.<br>
<br>
<br>
<h2>Feed Connection Properties</h2>

<h3>gsa.feed.protocol</h3>
The <code>gsa.feed.protocol</code> property specifies the URL protocol for<br>
the feed host on the GSA. The supported values are <code>http</code> and<br>
<code>https</code>.<br>
For example:<br>
<pre><code>gsa.feed.protocol=http<br>
</code></pre>

<i>Since:</i> 2.8.0<br>
<br>
<h3>gsa.feed.host</h3>
The <code>gsa.feed.host</code> property specifies the host IP address for the<br>
feed host on the Google Search Appliance.<br>
For example:<br>
<pre><code>gsa.feed.host=172.24.2.0<br>
</code></pre>

<h3>gsa.feed.port</h3>
The <code>gsa.feed.port</code> property specifies the HTTP host port for the<br>
feed host on the GSA.<br>
For example:<br>
<pre><code>gsa.feed.port=19900<br>
</code></pre>

<h3>gsa.feed.securePort</h3>
The <code>gsa.feed.securePort</code> property specifies the HTTPS host port<br>
for the feed host on the GSA. This port will be used if the<br>
<code>gsa.feed.host</code> property is set to <code>https</code>.<br>
For example:<br>
<pre><code>gsa.feed.securePort=19902<br>
</code></pre>

<i>Since:</i> 2.8.0<br>
<br>
<h3>gsa.feed.validateCertificate</h3>
The <code>gsa.feed.validateCertificate</code> property specifies whether to<br>
validate the GSA certificate when sending SSL feeds. If the GSA<br>
certificate is installed in the Tomcat keystore, this should be<br>
set to <code>true</code>, otherwise it must be set to <code>false</code>.<br>
For example:<br>
<pre><code>gsa.feed.validateCertificate=false<br>
</code></pre>

<i>Since:</i> 2.8.0<br>
<br>
<h3>manager.locked</h3>
The <code>manager.locked</code> property is used to lock out the Admin Servlet<br>
and prevent it from making further changes to the Feed Connection properties.<br>
If it is set to <code>true</code> or missing the Servlet will<br>
not be allowed to update the Feed Connection properties.<br>
<br>
<b>NOTE:</b>This property will automatically be changed to <code>true</code> upon<br>
successful update of the <code>gsa.feed.host</code> and <code>gsa.feed.port</code> when registering<br>
a Connector Manager with a Google Search Appliance.  Therefore, once the<br>
Feed Connection properties are successfully updated by the Admin Servlet,<br>
subsequent updates will be locked out until the flag is manually<br>
reset to <code>false</code>.  For more information, see<br>
<a href='ChangeGSA.md'>Changing the GSA Feed Host</a>.<br>
<br>
<pre><code>manager.locked=false<br>
</code></pre>


<h2>Feed Logging Properties</h2>

<h3>feedLoggingLevel</h3>
The <code>feedLoggingLevel</code> property controls the logging of the feed<br>
record to a log file.  The log record will contain the feed XML<br>
without the content data.  Set this property to <code>ALL</code> to enable feed<br>
logging, <code>OFF</code> to disable.  Customers and developers can use this<br>
functionality to observe the feed record and metadata information<br>
the connector manager sends to the Google Search Appliance.<br>
The feed log contains most of the information feed to the Search<br>
Appliance, but does not log the Document content.<br>
For example:<br>
<pre><code>feedLoggingLevel=ALL<br>
</code></pre>

<h3>feed.logging.FileHandler.pattern</h3>
The <code>feed.logging.FileHandler.pattern</code> property specifies the<br>
location and naming convention used when generating feed logs.<br>
The feed log filename pattern follows the<br>
<a href='http://java.sun.com/j2se/1.5.0/docs/api/java/util/logging/FileHandler.html'>java.util.logging.FileHandler</a>
rules.  The default pattern places feed logs is the <code>logs</code> directory of<br>
the Tomcat installation for the Connector Manager, in files named<br>
<code>google-connectors.feed*.log</code>.<br>
For example:<br>
<pre><code>feed.logging.FileHandler.pattern=/var/logs/connectors/acme-connectors.feed%g.log<br>
</code></pre>

<h3>feed.logging.FileHandler.limit</h3>
The <code>feed.logging.FileHandler.limit</code> property specifies an approximate<br>
maximum size, in bytes, to any one feed log file before creating a new<br>
feed log file. If this is zero, then there is no limit.  The default limit is 50MB.<br>
For example:<br>
<pre><code>feed.logging.FileHandler.limit=0<br>
</code></pre>

<h3>feed.logging.FileHandler.count</h3>
The <code>feed.logging.FileHandler.count</code> property specifies how many feed<br>
log files to cycle through.  No more than this number of feed logs will be<br>
maintained, with older logs being discarded as needes.  The default feed<br>
log count is 10.<br>
For example:<br>
<pre><code>feed.logging.FileHandler.limit=30<br>
</code></pre>

<h3>teedFeedFile</h3>
If you set the <code>teedFeedFile</code> property to the name of an existing<br>
file, whenever the connector manager feeds content to the Search Appliance,<br>
it will write a duplicate copy of the feed XML to the file specified by<br>
the <code>teedFeedFile</code> property.  Google Search Appliance customers and<br>
third-party developers can use this functionality to observe the content<br>
the connector manager sends to the Search Appliance and reproduce any<br>
issue which may arise.<br>
For example:<br>
<pre><code>teedFeedFile=/tmp/connector/CMTeedFeedFile<br>
</code></pre>
<b>NOTE:</b> The <code>teedFeedFile</code> will contain all feed data sent to the<br>
Search Appliance, including document content and metadata.<br>
The <code>teedFeedFile</code> can therefore grow quite large very quickly.<br>
<br>
<h2>Feed Content Properties</h2>

<h3>feed.timezone</h3>
The <code>feed.timezone</code> property defines the default time zone used<br>
for Date metadata values for Documents.  A <code>null</code> or empty string<br>
indicates that the local time zone of the machine running the<br>
Connector Manager should be used.  The default feed time zone<br>
is local time zone of the Connector Manager.  Standard <a href='http://java.sun.com/j2se/1.5.0/docs/api/java/util/TimeZone.html'>Java TimeZone</a>
identifiers may be used.  For example:<br>
<pre><code>  feed.timezone=America/Los_Angeles<br>
</code></pre>
If a standard <code>TimeZone</code> identifier is unavailable, then a custom<br>
<code>TimeZone</code> identifier can be constructed as +/-hours<code>[</code>minutes<code>]</code> offset<br>
from GMT.  For example:<br>
<pre><code>  feed.timezone=GMT+10    # GMT + 10 hours<br>
  feed.timezone=GMT+0630  # GMT + 6 hours, 30 minutes<br>
  feed.timezone=GMT-0800  # GMT - 8 hours, 0 minutes<br>
</code></pre>

<h3>feed.file.size</h3>
The <code>feed.file.size</code> property sets the target size, in bytes, of<br>
an accumulated feed file. The Connector Manager tries to collect<br>
many feed Documents into a single feed file to improve the<br>
efficiency of sending data to the Google Search Appliance.<br>
Specifying too small a value may result in many small feeds which<br>
might overrun the GSA's feed processor.  However, specifying too<br>
large a feed size reduces concurrency and may result in OutOfMemory<br>
errors in the Java VM, especially if using multiple Connector instances.<br>
The default target feed size is 10MB, which will typically hold<br>
50-100 fed documents.<br>
<pre><code>feed.file.size=10485760<br>
</code></pre>

<h3>feed.document.size.limit</h3>
The <code>feed.document.size.limit</code> property defines the maximum<br>
allowed size in bytes of a Document's content.  Documents whose<br>
content exceeds this size will still have metadata indexed,<br>
however the content itself will not be fed.  The default value is<br>
30MB, the maximum file size accepted by the Google Search Appliance.<br>
<pre><code>feed.document.size.limit=31457280<br>
</code></pre>

<h3>feed.backlog.floor, feed.backlog.ceiling, feed.backlog.interval</h3>
The Feed Backlog properties are used to throttle back the<br>
document feed if the Search Appliance has fallen behind processing<br>
outstanding feed items.  The Connector Manager periodically polls the Search Appliance,<br>
fetching the count of unprocessed feed items (the backlog count).<br>
If the backlog count exceeds the ceiling value, feeding is paused.<br>
Once the backlog count drops back down below the floor value, feeding<br>
resumes.<br>
<pre><code># Stop feeding the GSA if its backlog exceeds this value.<br>
feed.backlog.ceiling=10000<br>
# Resume feeding the GSA if its backlog falls below this value.<br>
feed.backlog.floor=1000<br>
# How often to check for feed backlog (in seconds).<br>
feed.backlog.interval=900<br>
</code></pre>

<h2>Traversal Properties</h2>

<h3>traversal.batch.size</h3>
The <code>traversal.batch.size</code> property defines the optimal number<br>
of items to return in each repository traversal batch.  The batch<br>
size represents the size of the roll-back that occurs during a<br>
failure condition.  Batch sizes that are too small may incur<br>
excessive processing overhead.  Batch sizes that are too large<br>
may produce OutOfMemory conditions within a Connector or result<br>
in early termination of the batch if processing time exceeds the<br>
<code>travesal.time.limit</code>.  The default traversal batch size is 500 items.<br>
For example:<br>
<pre><code>traversal.batch.size=1000<br>
</code></pre>

<h3>traversal.poll.interval</h3>
The <code>traversal.poll.interval</code> property defines the number of<br>
seconds to wait after a traversal of the repository finds no new<br>
content before looking again.  Short intervals allow new content<br>
to be readily available for search, at the cost of increased<br>
repository access.  Long intervals add latency before new<br>
content becomes available for search.  By default, the Connector<br>
Manager waits 5 minutes (300 seconds) before retraversing the<br>
repository if no new content was found on the last traversal.<br>
For example:<br>
<pre><code>traversal.poll.interval=900<br>
</code></pre>

<h3>traversal.time.limit</h3>
The <code>traversal.time.limit</code> property defines the number of<br>
seconds a traversal batch should run before gracefully exiting.<br>
Traversals that exceed this time period risk cancelation.<br>
The default time limit is 30 minutes (1800 seconds).<br>
For example:<br>
<pre><code>traversal.time.limit=3600<br>
</code></pre>

<h3>traversal.enabled</h3>
The <code>traversal.enabled</code> property is used to enable or disable<br>
Traversals and Feeds for all connector instances in this<br>
Connector Manager.  Disabling Traversal would be desirable if<br>
configuring a Connector Manager deployment that only authorizes<br>
search results.  Traversals are enabled by default.<br>
<pre><code>traversal.enabled=false<br>
</code></pre>

<i>Since:</i> 2.6.0<br>
<br>
<br>
<h2>Miscellaneous Properties</h2>

<h3>config.change.detect.interval</h3>
The <code>config.change.detect.interval</code> property specifies how often<br>
(in seconds) to look for asynchronous configuration changes.<br>
Values <= 0 imply never.  For stand-alone deployments, long<br>
intervals or never are probably sufficient.  For clustered<br>
deployments with a shared configuration store, 60 to 300 seconds<br>
is probably sufficient.  The default configuration change<br>
detection interval is 0 (never).<br>
<pre><code>config.change.detect.interval=60<br>
</code></pre>

<i>Since:</i> 2.8.0