-
-
Notifications
You must be signed in to change notification settings - Fork 19
Indexing API
Google group thread.
Most frameworks come with a variety of configuration files in various formats. Indexing those files to add support for framework specific behavior is currently not possible using the PDT/DLTK infrastructure (the DLKT H2 indexer only treats IModelElements as indexable - which .xml or .json files are not in a PHP project).
To overcome this limitation i propose the following API to allow indexing of non-sourcefiles in PDT projects based on apache lucene.
The following extension point allows to register BuildParticipants which receive configuration files to index:
<schema targetNamespace="com.dubture.indexing.core" xmlns="http://www.w3.org/2001/XMLSchema">
<annotation>
<appinfo>
<meta.schema plugin="com.dubture.indexing.core" id="buildParticipant" name="Lucene Build Indexing Participant"/>
</appinfo>
<documentation>
[Enter description of this extension point.]
</documentation>
</annotation>
<element name="extension">
<annotation>
<appinfo>
<meta.element />
</appinfo>
</annotation>
<complexType>
<sequence>
<element ref="participant" minOccurs="1" maxOccurs="unbounded"/>
</sequence>
<attribute name="point" type="string" use="required">
<annotation>
<documentation>
</documentation>
</annotation>
</attribute>
<attribute name="id" type="string">
<annotation>
<documentation>
</documentation>
</annotation>
</attribute>
<attribute name="name" type="string">
<annotation>
<documentation>
</documentation>
<appinfo>
<meta.attribute translatable="true"/>
</appinfo>
</annotation>
</attribute>
</complexType>
</element>
<element name="participant">
<complexType>
<attribute name="nature_id" type="string" use="required">
<annotation>
<documentation>
</documentation>
</annotation>
</attribute>
<attribute name="file_extensions" type="string" use="required">
<annotation>
<documentation>
Space separated list of file extensions registered for this visitor.
</documentation>
</annotation>
</attribute>
<attribute name="visitor" type="string" use="required">
<annotation>
<documentation>
The Implementation of the IndexingVisitor for the participant.
</documentation>
<appinfo>
<meta.attribute kind="java" basedOn=":com.dubture.indexing.core.index.IndexingVisitor"/>
</appinfo>
</annotation>
</attribute>
<attribute name="name" type="string">
<annotation>
<documentation>
The human readable name of the participant
</documentation>
</annotation>
</attribute>
</complexType>
</element>
</schema>
Extenders should implement a BuildParticipant with following interface:
/**
* IndexingVisitor interface for buildParticipant extension implementation.
*
* @author Robert Gruendler <[email protected]>
*
*/
public interface IndexingVisitor {
/**
* Set the indexingrequestor.
*
* @param requestor
* @return {@link IndexingVisitor}
*/
IndexingVisitor setRequestor(IIndexingRequestor requestor);
/**
* Get the resource the visitor is operating on.
*
* @return {@link IResource}
*/
IResource getResource();
/**
* Sets the resource the visitor is operating on.
*
* @param resource
* @return {@link IndexingVisitor}
*/
IndexingVisitor setResource(IResource resource);
/**
* The transformed POJO.
*
* @param object
* can be safely cast to your implemetation object.
*/
void visit(Object object);
/**
* A resource is about to be deleted.
*
* @param file
* the resource being deleted
*/
void resourceDeleted(IFile file);
}
An example implementation of the extension point looks like this (taken from the composer plugin):
<extension point="com.dubture.indexing.core.buildParticipant">
<participant
file_extensions="json"
name="ComposerBuildParticipant"
nature_id="com.dubture.composer.core.composerNature"
visitor="com.dubture.composer.core.visitor.ComposerVisitor">
</participant>
</extension>
With an example implementation of the visit() method which receives to object being indexed:
@Override
public void visit(Object objectToIndex)
{
// ... type checking etc ommitted for simplicity
PackageInterface pHPPackage = (PackageInterface) objectToIndex;
ReferenceInfo info = new ReferenceInfo("composer.json.reference",pHPPackage.getName());
requestor.addReference(info);
}
In this case the objectToIndex is already deserialized into the correct Java type (PackageInterface) from a json file. This is done by implementing the JsonIndexingVisitor interface (which extends IndexingVisitor):
@Override
public Gson getBuilder()
{
return new GsonBuilder()
.registerTypeAdapter(License.class, new LicenseDeserializer())
.setFieldNamingStrategy(new ComposerFieldNamingStrategy())
.create();
}This is a simple API which enables extenders to easily index configuration files from frameworks.
The indexing feature provides a SearchEngine API to query the index for previous indexed references:
/**
* some internal method from an extender plugin to search for indexed items by path
*/
public List<EclipsePHPPackage> getPackages(IPath path)
{
List<ReferenceInfo> references;
List<EclipsePHPPackage> packages = new ArrayList<EclipsePHPPackage>();
SearchEngine engine = SeachEngine.getInstance();
references = engine.findReferences(path, ComposerVisitor.REFERENCE_ID);
for (ReferenceInfo info : references) {
// retrieve additional metadata from the indexed document
String meta = info.getMetadata();
PHPPackage json = gson.fromJson(meta, PHPPackage.class);
EclipsePHPPackage pHPPackage = new EclipsePHPPackage(json);
packages.add(pHPPackage);
}
return packages;
}The underlying index is implemented using apache lucene. Every reference to a domain object is indexed as a lucene document with the following structure:
- path: the absolute path to the item being index
- filename: the name of the file where the item being indexed is found
- type: the type of the reference (should be a unique identifier over all index-implementors, use your java namespace as a base)
- referencename: the name of the reference
- metadata: an optional string where users can store additional metadata to retrieve later on when querying the index