Sphinx setup guide

Sphinx (http://sphinxsearch.com/) is used by Camdram as the backend for the autocomplete and full text search features. It uses SphinxRealtimeBundle, which is part of the Camdram organisation on Github but is a separate repository (because it could potentially have uses outside of Camdram) - https://github.com/camdram/sphinx-realtime-bundle. It hooks into Camdram in two ways:

• There's a listener which listens into all Doctirne insert/update/delete queries, and syncs the changes to the Sphinx data store.
• The autocomplete feature calls Sphinx to get its results.

The Sphinx docs talk about two different ways of using Sphinx. We exclusively use the 'real-time' mode.

search_provider

This is a key in parameters.yml which dictates which search backend to use. At this point in time, there are two possible values:

• doctrine performs search queries directly on the database. Is slow and produces poorly-ranked results, but does not require Sphinx to be installed. This is the default for new installs
• sphinx performs search queries using Sphinx. This is the value of parameters.yml in production.

Note that at the moment, search_provider only affects the backend called when the autocomplete feature is used; it does not turn off the Doctrine listener which updates the Sphinx data store, which causes errors to be generated when actions are performed that write to the database if Sphinx is not running. For now, this feature can separately be turned off by adding enabled: false to the acts_sphinx_realtime: section in config.yml

Setting up Sphinx

Camdram requires Sphinx 2.1, which at the time of writing is greater than the version in most Linux repositories. Packages for a range of distros can be downloaded from http://sphinxsearch.com/downloads/release/.

SphinxRealtimeBundle generates its own config file by running acts:sphinx:config, based on the information in the acts_sphinx_realtime section of config.yml. The config file is output to app/config/sphinx.default.cfg. Sphinx expects to see its config file at /etc/sphinxsearch/sphinx.conf (on Ubuntu), so you can either copy this file there or create a symlink.

The default config file is set up to store its data in app/data/. This requires this folder to exist, and to be writeable by both yourself and www (in a similar way to app/cache and app/logs).

Sphinx must then be restarted by running sudo service sphinxsearch restart (depending on your distro, the service might be called 'searchd' instead). You may get errors about write access to certain files/folders for its logs and lock file, which should be fixed by either creating the files or editing the generated config file so that the paths point to the right place on your computer.

'Populate' command

If the Sphinx data store gets out of sync with the DB (e.g. if you've just imported a dump of the database using phpMyAdmin or something), then you can run acts:sphinx:populate, which automatically clears the datastore and re-populates it from the database.

If you intend to run a command which touches every relevant row in the DB (e.g. camdram:entites:slugs), then there's no need to run this command because the Doctrine listeners should pick up any changes.

Filters

Certain entity types have a 'is_indexable_callback:' defined in config.yml. This refers to a function on the entity which can be called to determine whether the object should be stored in the search index. E.g. for the Show entity, the function returns false if the show has not been authorised. NB: there is currently a bug which means that the 'populate' command ignores these filters.