-
Notifications
You must be signed in to change notification settings - Fork 18
wxslucene
Lucene and all its dependencies
- wxslucene.jar
- wxsutils.jar
- ogclient.jar or objectgrid.jar
- wxsutils.jar
- wxslucene.jar
- objectgrid.jar and all its dependencies
The objectgrid.xml and deployment.xml files in the test/resources should be used with appropriate modifications for partition counts and replication strategy. The only file requiring modification is the deployment.xml. The only things here to modify are the partition count as well as replication strategy.
Each directory and its file contents are stored in a separate map with a prefix of ChunkMap.
The Lucene plugin is configured using two property files that should be on the classpath. The wxsutils.properties file is used to connect to a grid and the wxslucene.properties file configures the directory plugin.
The wxsutils.properties file in test/resources should be configured to attach to the grid in which to store the indexes. There is no need to add maps and so on for additional indexes. The map definitions are constant. I may break each directory to its own maps purely so that metrics on map sizes etc can be easily made specific to a particular directory.
The wxslucene.properties file in test/resources allows the directory to be customized. The main things to modify here if required are the block size and the compression. We recommend async_put to always be on, it’s MUCH faster when copying disk based directories in to a grid based directory, possible 6-10x faster. The compression can significantly reduce the memory required for the index and reduces the network bandwidth required between the lucene JVMs and the grid. There is a CPU cost and this cost needs to be weighed against the advantages. Index files are broken in to chunks or blocks of a fixed size when stored in the grid. The blockSize parameter specifies how big those blocks are. 4k seems a common value for these kinds of system.
Both these property files need to be in a folder on the lucene JVM classpath for them to be found.
Example command line to start a WXS grid container would be:
startOgServer.sh server01 -objectgridFile xml/objectgrid.xml -deploymentPolicyFile xml/deployment.xml -jvmArgs -d64 -Xmx9G -Dcom.sun.management.jmxremote -XX:+AggressiveOpts -verbose:gc -Xloggc:logs/server01.gc.log -XX:-PrintGCDetails -XX:-PrintGCTimeStamps -cp jars/wxsutils-1.3-SNAPSHOT.jar:jars/wxslucene-1.0-SNAPSHOT.jar