Entity Stores

The Entity Stores layer is designed to access data as entities with attributes and links. You should have a transaction to create, modify, read and query data. Transactions are quite similar to those on the Environments layer, though the Entity Store API is much richer in terms of quering data. API as well as implementation live in the jetbrains.exodus.entitystore package.

PersistentEntityStore
Transactions
Entities
Properties
Links
Blobs
Search Queries
    Iterating over All Entities of Specified Type
    EntityIterable
    Searching by Property Value
    Searching in Range of Property Values
    Traversing Links
    SelectDistinct and SelectManyDistinct
    Binary Operations
    Searching for Entities Having Property, Link, Blob
    Sorting
    Other Goodies

###PersistentEntityStore

To open or create an entity store, you have to create an instance of PersistentEntityStore with the help of the PersistentEntityStores utility class:

PersistentEntityStore entityStore = PersistentEntityStores.newInstance("/home/me/.myAppData");

PersistentEntityStore works over Environment, so the above method implicitly creates Environment with the same location. PersistentEntityStore has a name, several entity stores with different names can be created over an Environment. If you don't specify one, default name is used.

PersistentEntityStores has different methods to create an instance of PersistentEntityStore. In addition to underlying Environment, you can specify BlobValut and PersistentEntityStoreConfig. BlobVault is a base class describing interface to binary large objects (BLOBs) used internally by implementation of PersistentEntityStore. If you don't specify BlobVault on creation of PersistentEntityStore, instance of the FileSystemBlobVault class is used. If you don't specify PersistentEntityStoreConfig on creation of PersistentEntityStore, PersistentEntityStoreConfig.DEFAULT is used.

Like ContextualEnvironment, PersistentEntityStore is always aware of transaction started in current thread. Method getCurrentTransaction() returns transaction started in current thread or null if there is no such one.

After you finished working with PersistentEntityStore, call close() method.

###Transactions

Entity store transactions are quite similar to the Environment layer transactions. To manually start transaction use beginTransaction():

final StoreTransaction txn = store.beginTransaction(); 

or beginReadonlyTransaction():

final StoreTransaction txn = store.beginReadonlyTransaction();

Attempt to modify data in read-only transaction will fail with ReadonlyTransactionException.

Any transaction should be finished, i.e. aborted or committed. Transaction can also be flushed or reverted. Methods commit() and flush() return true if they succeed. If any of them returns false the database version mismatch has happened. In that case, there are two possibilities: to abort the transaction and finish and just continue. Unsuccessful flush implicitly reverts transaction and moves it to the latest (newest) database snapshot, so database operations can be repeated against it:

StoreTransaction txn = beginTransaction();
try {
    do {
        // do something
        // if txn has already been aborted in user code
        if (txn != getCurrentTransaction()) {
            txn = null;
            break;
        }
    } while (!txn.flush());
} finally {
    // if txn has not already been aborted in execute()
    if (txn != null) {
        txn.abort();
    }
}

If you don't care of such spinning and don't want to control result of flush() and commit(), you can just use executeInTransaction(), executeInReadonlyTransaction(), computeInTransaction() and computeInReadonlyTransaction() methods.

###Entities

Entities can have properties and blobs, and can be linked. Each property, blob or link is identified by its name. Although entity properties expected to be Comparable, only Java primitives types, Strings and ComparableSet values can be used by default. Use method PersistentEntityStore.registerCustomPropertyType() to define your own property type.

Imagine that your application must include a user management system. All further samples imply that you have accessible StoreTransaction txn. Let's create a new user:

final Entity user = txn.newEntity("User");

Each Entity has string entity type and its unique id which is described by EntityId:

final String type = user.getType();
final EntityId id = user.getId();

Entity id may be used as a part of URL or in any other way for further loading the entity:

final Entity user = txn.getEntity(id);

###Properties

Let's create user with specified loginName, fullName, email and password:

final Entity user = txn.newEntity("User");
user.setProperty("login", loginName);
user.setProperty("fullName", fullName);
user.setProperty("email", email);
final String salt = MessageDigestUtil.sha256(Double.valueOf(Math.random()).toString());
user.setProperty("salt", salt);
user.setProperty("password", MessageDigestUtil.sha256(salt + password));

In order to save password ciphered, MessageDigestUtil class from utils module is used.

###Links

Probably the user management system should be able to save some additional information about user: age, bio, avatar, etc. It's reasonable not to save this information directly in a User entity, but to create a UserProfile one and link it with the user:

final Entity userProfile = txn.newEntity("UserProfile");
userProfile.setLink("user", user);
user.setLink("userProfile", userProfile);
userProfile.setProperty("age", age);

Reading profile of a user:

final Entity userProfile = user.getLink("userProfile");
if (userProfile != null) {
    // read properties of userProfile
}

Method setLink() sets the new link and overrides previous one. It is also possible to add a new link not affecting already added links. Suppose users can be logged in with the help of different AuthModules: LDAP, OpenId, etc. It makes sense to create an entity for each such auth module and to link it with the user:

final Entity authModule = txn.newEntity("AuthModule");
authModule.setProperty("type", "LDAP");
user.addLink("authModule", authModule);
authModule.setLink("user", user);

Iterating over all user's auth modules:

for (Entity authModule: user.getLinks("authModule)) {
    // read properties of authModule
}

It's also possible to delete particular auth module:

user.deleteLink("authModule", authModule);

or delete all available auth modules:

user.deleteLinks("authModule);

###Blobs

Some properties cannot be expressed as Strings or primitive types, or its values are too large. E.g. it's better to save large strings (like user's biography) in a blob string instead of just property. For raw binary data (images, media, etc.) use blobs:

userProfile.setBlobString("bio", bio);
userProfile.setBlob("avatar", file);

Blob string is very similar to property, but it cannot be used in Search Queries. To read blob string, use Entity.getBlobString() method.

Value of a blob can be set as java.io.InputStream or java.io.File. The second method is preferable when setting blob from a file. To read blob, use Entity.getBlob() method. You don't have and you should not close the input stream returned by the method. Concurrent access to a single blob within single transaction is not possible.

###Queries

StoreTransaction contains a lot of methods to query, sort and filter entities. All of them return an instance of EntityIterable.

#####Iterating over All Entities of Specified Type

Let's iterate over all users and print their full names:

final EntityIterable allUsers = txn.getAll("User);
for (Entity user: allUsers) {
    System.out.println(user.getProperty("fullName"));
}

As you can see, EntityIterate is Iterable<Entity>.

#####EntityIterable

EntityIterate allows to lazily iterate over entities. EntityIterable is valid only against particular database snapshot, so finishing transaction or moving it to the newest snapshot (flush(), revert()) breaks iteration. If you need to flush current transaction during iteration over an EntityIterable, you have to load manually the entire entity iterable to a list and then iterate over the list.

You can find out the size of EntityIterable without iterating:

final long userCount = txn.getAll("User).size();

Though method size() performs faster than iteration, it can be quite slow for some iterables. Xodus does internally a lot of caching, so sometimes size of an EntityIterable can be computed quite quickly. You can check if it can using count() method:

final long userCount = txn.getAll("User").count();
if (userCount >= 0) {
    // result for txn.getAll("User") is cached, so user count is known
}

Method count() checks if the result (sequence of entity ids) is cached for the EntityIterable and returns quickly its size if it is. If the result is not cached method count() returns -1.

In addition to size() and count() which always return actual value (if not -1), there are eventually consistent methods getRoughCount() and getRoughSize(). If result for the EntityIterable is cached, these methods return the same value as count() and size() do. If the result is not cached, Xodus can internally cache the value of last known size of the EntityIterable. If last known size is cached, getRoughCount() and getRoughSize() return it. Otherwise, getRoughCount() returns -1 and getRoughSize() returns the value of size().

Use method isEmpty() to check if an EntityIterable is empty. In most cases, it is faster than getting size(), and it is quite immediate if the EntityIterable's result cached.

#####Finding Entities by Property Value

To login user with provided credentials (loginName and password), at first you must find all users with specified loginName:

final EntityIterable candidates = txn.find("User", "login", loginName);

Then you have to iterate over candidates and check is password matches:

Entity loggedInUser = null;
for (Entity candidate: candidates) {
    final String salt = candidate.getProperty("salt");
    if (MessageDigestUtil.sha256(salt + password).equals(candidate.getProperty("password"))) {
        loggedInUser = candidate;
        break;
    }
}

return loggedInUser; 

If you want to login users with email also, calculate candidates as follows:

final EntityIterable candidates = txn.find("User", "login", loginName).union(txn.find("User", "email", email));

To find user profiles of users with specified age:

final EntityIterable little15Profiles = txn.find("UserProfile", "age", 15);

Please note that search by string property values is case-insensitive.

#####Searching in Range of Property Values

Searching for user profiles of users with age in range [17-23] inclusively:

final EntityIterable studentProfiles = txn.find("UserProfile", "age", 17, 23);

Another case of range search is searching for entities with string property starting with a value:

final EntityIterable userWithFullNameStartingWith_a = txn.findStartingWith("User", "fullName", "a");

Please note that search by string property values is case-insensitive.

#####Traversing Links

One method for traversing links is already mentioned above: Entity.getLinks(). It is considered as query because it returns EntityIterable. It allows to iterate over outgoing links of an entity with specified name.

It is also possible to find incoming links. E.g., let's search for user who uses particular auth module:

final EntityIterable ldapUsers = txn.findLinks("User", ldapAuthModule, "authModule");
final EntityIterator ldapUsersIt = ldapUsers.iterator();
return ldapUsersIt.hasNext() ? ldapUsersIt.next() : null; 

#####Select and SelectMany

Searching for users with age in range [17-23] inclusively:

final EntityIterable studentProfiles = txn.find("UserProfile", "age", 17, 23);
final EntityIterable students = studentProfiles.selectDistinct("user");

Getting all auth modules of users with age in range [17-23] inclusively:

final EntityIterable studentProfiles = txn.find("UserProfile", "age", 17, 23);
final EntityIterable students = studentProfiles.selectDistinct("user");
final EntityIterable studentAuthModules = students.selectManyDistinct("authModule");

selectDistinct operation should be chosen if corresponding link is single, i.e. if it is set using setLink() method. If the link is multiple, i.e. if it is set using addLink() method, selectManyDistinct should be used. Results of both selectDistinct and selectManyDistinct operations never contain duplicate entities. In addition, result of selectManyDistinct can contain null. E.g., if there is a user with no auth module.

#####Binary operations

There are four binary operations defined for EntityIterable: intersect(), union(), minus() and concat(). For all of them, instance is a left operand, and parameter is a right operand.

Let's search for users whose login and fullName start with "xodus" (case-insensitively):

final EntityIterable xodusUsers = txn.findStartingWith("User", "login", "xodus").intersect(txn.findStartingWith("User", "fullName", "xodus"));

Users whose login or fullName start with "xodus":

final EntityIterable xodusUsers = txn.findStartingWith("User", "login", "xodus").union(txn.findStartingWith("User", "fullName", "xodus"));

Users whose login and not fullName start with "xodus":

final EntityIterable xodusUsers = txn.findStartingWith("User", "login", "xodus").minus(txn.findStartingWith("User", "fullName", "xodus"));

There is no suitable sample for concat() operation, it just concatenates results of two entity iterables.

Result of binary operation (EntityIterable) itself can be an operand of binary operation. So you can construct query tree of arbitrary height.

#####Searching for Entities Having Property, Link, Blob

Method StoreTransaction.findWithProp returns entities of specified type that have property with specified name. There are also methods StoreTransaction.findWithBlob and StoreTransaction.findWithLink.

E.g., if we allow user to have no full name, i.e. her property fullName can be null, then probably we might wish to get users with or without full name using findWithProp:

final EntityIterable usersWithFullName = txn.findWithProp("User", "fullName");
final EntityIterable usersWithoutFullName = txn.getAll("User").minus(txn.findWithProp("User", "fullName"));

Getting user profiles with avatars using findWithBlob:

final EntityIterable userProfilesWithAvatar = txn.findWithBlob("UserProfile", "avatar");

Method findWithBlob is also applicable to blob strings:

final EntityIterable userProfilesWithBio = txn.findWithBlob("UserProfile", "bio");

Getting users with auth modules:

final EntityIterable usersWithAuthModules = txn.findWithLink("User", "authModule");

#####Sorting

Sorting all user by login property:

final EntityIterable sortedUsersAscending = txn.sort("User", "login", true);
final EntityIterable sortedUsersDescending = txn.sort("User", "login", false);

Sorting all users that have LDAP authentication by login property:

// at first, find all LDAP auth modules
final EntityIterable ldapModules = txn.find("AuthModule", "type", "ldap"); // case-insensitive!
// then select users
final EntityIterable ldapUsers = ldapModules.selectDistinct("user");
// finally, sort them
final EntityIterable sortedLdapUsers = txn.sort("User", "login", ldapUsers, true);

Sorting can be stable. E.g., getting users sorted by login in ascending order and by fullName in descending (users with the same login name will be sorted by full name in descending order):

final EntityIterable sortedUsers = txn.sort("User", "login", txn.sort("User", "fullName", false), true);

You can implement your custom sorting algorithms. EntityIterable.reverse() can be used for this. Wrap your sort results EntityIterable with EntityIterable.asSortResult(). This will allow sorting engine to recognize sort result and to use stable sorting algorithm. If source is not a sort result the engine uses non-stable sorting algorithm which is faster in general.

#####Other Goodies