ToMany considers changes applied even if put fails

[GautaA]

ObjectBox version (are you using the latest version?): 3.5.0
Reproducibility: Every time
Device: Multiple Android devices
OS: Android

Hi, I'm wondering whether the issue I'm experiencing is a intended behavior or a bug.

Our working flow goes like this:
We have a model with a lot of different fields but two most important ones are ToMany relations with list of specific events. Single object that contains all of the data that we are adding to the database, and the said object gets updated till the data gets uploaded to our server, and the cleared and deleted from the database.

Under specific circumstances the database gets full and we have a need to extend it.

We do so by caching the exception, closing the database, increasing the maxSizeInKByte value and rebuilding it. The problem I have noticed is it that when Objectbox tries to add data after the database has extended it won't add the events (that are stored in the Lists and are connected to the main object by ToMany relations) that happened in those last few seconds between us closing the database and rebuilding it. Or to be more precise the data that was supposed to be added but wasn't successfully because of the exception will never be added, even though it's still in the object itself, and the same said object always is sent to be added.

I'm guessing this has something to do with caching or some similar mechanism, here is what else I tried:

1. Deleting all files from the database and re-adding the object -> results in only data after extending the database being added (which confirms in someway that the data that Objectbox added before isn't getting re-added for some reason, even though database is empty and the object still has the data)
2. Serializing and deserializing the object so it has a completely new memory address, still the same result as in 1.
3. Playing around with ids of ToMany objects, methods like reset(), setTarget() and so on, nothing has worked so far.

Please if you could point me into the right direction and tell me what I'm doing wrong it would be greatly appreciated.

[greenrobot-team]

Thanks for your question and detailed research and explanation. I guess this can be caused by the internal change tracking of ToMany, e.g. see

objectbox-java/objectbox-java/src/main/java/io/objectbox/relation/ToMany.java

Lines 792 to 853 in b17ac52

	public void internalApplyToDb(Cursor<?> sourceCursor, Cursor<TARGET> targetCursor) {
	TARGET[] toRemoveFromDb;
	TARGET[] toPut;
	TARGET[] addedStandalone = null;
	List<TARGET> removedStandalone = null;
	boolean isStandaloneRelation = relationInfo.relationId != 0;
	IdGetter<TARGET> targetIdGetter = relationInfo.targetInfo.getIdGetter();
	synchronized (this) {
	if (isStandaloneRelation) {
	for (TARGET target : entitiesAdded.keySet()) {
	if (targetIdGetter.getId(target) == 0) {
	entitiesToPut.add(target);
	}
	}
	if (removeFromTargetBox) {
	entitiesToRemoveFromDb.addAll(entitiesRemoved.keySet());
	}
	if (!entitiesAdded.isEmpty()) {
	addedStandalone = (TARGET[]) entitiesAdded.keySet().toArray();
	entitiesAdded.clear();
	}
	if (!entitiesRemoved.isEmpty()) {
	removedStandalone = new ArrayList<>(entitiesRemoved.keySet());
	entitiesRemoved.clear();
	}
	}
	toRemoveFromDb = entitiesToRemoveFromDb.isEmpty() ? null : (TARGET[]) entitiesToRemoveFromDb.toArray();
	entitiesToRemoveFromDb.clear();
	toPut = entitiesToPut.isEmpty() ? null : (TARGET[]) entitiesToPut.toArray();
	entitiesToPut.clear();
	}
	if (toRemoveFromDb != null) {
	for (TARGET target : toRemoveFromDb) {
	long id = targetIdGetter.getId(target);
	if (id != 0) {
	targetCursor.deleteEntity(id);
	}
	}
	}
	if (toPut != null) {
	for (TARGET target : toPut) {
	targetCursor.put(target);
	}
	}
	if (isStandaloneRelation) {
	long entityId = relationInfo.sourceInfo.getIdGetter().getId(entity);
	if (entityId == 0) {
	throw new IllegalStateException("Source entity has no ID (should have been put before)");
	}
	if (removedStandalone != null) {
	removeStandaloneRelations(sourceCursor, entityId, removedStandalone, targetIdGetter);
	}
	if (addedStandalone != null) {
	addStandaloneRelations(sourceCursor, entityId, addedStandalone, targetIdGetter);
	}
	}
	}

The entitiesAdded and entitiesRemoved maps are cleared before the actual put is happening. So if it fails, the ToMany will still consider all changes to be processed.

The workaround would be to re-add the new objects to the ToMany. Or get the owning object from the database to get the current state of the ToMany, then add any missing objects back.

Let me know if this is the issue. Having a quick look at the code we could change ToMany to only clear the tracked changes after all puts have successfully completed.

[GautaA]

Hi, thank you for the response.

Not sure if I'm understanding you correctly but what you suggested is:

After the database is extended, fetch the current "main" object from the database, re-add the ToMany objects to it and re-put it to the database?

I've done that but for some reason it's still only containing the "new" events.
Let me know if I'm missing something.

[greenrobot-team]

Maybe another option: put the event objects into their box first, only then add them to the ToMany and save the owning object. E.g. something like:

store.box<Event>().put(events)
collector.events.addAll(events)
store.box<Collector>().put(collector)

[GautaA]

Good morning, the second work around together with resetting event IDs to 0 before re-adding them worked. Thank you.
I think that confirms it that it's the bug you mentioned above.