iOS - Replicator on background thread and Couchbase lock-up on boot

[carrotcomputer]

I've got an iOS App which requires periodic replication to update data on client (couchbase-lite) from server (a remote couchdb instance). The database is about 2GB and each "update" includes about 50 revisions/new documents.

However, every time I send a POST using the REST API to the replicator, it completely locks up couchbase and I am unable to query anything else from it (For example, trying to GET /_active_tasks will not return anything for at least a couple of minutes).

Am I right in assuming that the replicator does not run on a separate thread to the main "manager"? If not, what kind of task would be required to do this. My objective c knowledge is basic but can understand and follow most "advanced" topics.

Excuse me if my terminology is a bit of. Not new to Couch but certainly new to doing it on mobile!

Thanks in advance

[Crapulax]

Hello,

50 documents is not a lot. Basically the http calls are asynchrone so they should not locked your application. I can figure out two issues that my explain above behavior:

• Your application is doing an heavy computation on new document received. You might deactivate the computation if you still get the issue.
• The documents you replicate are very big and you have exhausted the number of sockets which can be opened in parallel by the browser (around 5 if I am correct - depends on the browser too). You might check this with the inspector on networks tab and see what is going on
• Looking to network inspector can be a good starting point to understand what is going on here

hope it helps

[carrotcomputer]

Thanks for your reply @Crapulax

The application doesn't do anything with the received data until requested (im aware of the browser limitations too). Each doc is very small also.

Since I made this issue, my biggest problem seems to be that the database needs a few minutes to "start up" on boot where during that time, the Couchbase Lite thread maxes out at 100%.

No requests can be made to it during that time and because of this, my application either locks up completely or sits there waiting for a response and I cant report progress because requests to _active_tasks wont respond!

Is there any way to enable some form of logging so I can track what is going on in Couchbase Lite?

Also, going back to my original question - is there any way to have the replicator run on a different thread?

When my database has no data, the replicator maxes out the Couchbase thread which again stops the application from accepting any further requests.

[Crapulax]

If you are on Android you can check the device monitor or logcat for the Java logs. You will see logs from the DB there and might figure out what is going on. How many documents do you have on the mobile once it is initalised? I recall seeing in above logs huge computations on views when we had hundred thousands docs.
If you are on IOS there might be a way to look to such logs but I do not know how.

And to answer your question on background thread, I do not know :)

[carrotcomputer]

Just under 500k documents with update seq of 1378660 and it's on iOS.

Thanks for your help so far. Have you ever had any projects with a similar level of documents to mine?

[Crapulax]

Hello,

I was seeing this loading time issue with your amount of documents. If you cannot found the way to get the couchbase lite logs you can do this.

• Use a debugger on Mac and randomy pause your IOS program when it is frozen and check what the active task is doing
• Drop your views and check if issue persist, try to isolate the 'offending' view

I just also recall a regression on couchbase lite phonegp we have faced on IOS in 1.2.0 which was solved with 1.2.1 making couchbase lite looping for ever time to time. I am unable to retreive the issue description. If you use 1.2.0 check with 1.2.1 it worths it...

All the best,
Steve.

[carrotcomputer]

@Crapulax - Update on my progress. I managed to turn on Couchbase Logging (using this https://github.com/couchbase/couchbase-lite-ios/wiki/Logging).

The replication (from an empty database) starts off okay in the background. _active_tasks will not respond to any requests, but my main application is still usable.

But as it continues to eat more and more RAM, the application will eventually "freeze" whilst still replicating data. Eventually it either runs out of memory and crashes (2m documents, 2.8m update seq, 5.9GB) or completes and releases memory (100k documents, 800k update seq, 4.1GB) but swells to 1GB RAM usage during the process.

Three questions:

1. Is having 2 million documents on device (iPad Air 2 or iPad Pro 12") unrealistic?

2. Should I be able to poll _active_tasks whilst replication is happening using the REST API?

3. Why might I be getting this "memory leak"?

[Crapulax]

Hello Spark,

This seems to be huge numbers for a a mobile/Ipad.
There are heavy RAM limitations on such devices. Most recent devices have 1GB RAM (Iphone 6+ for instance), newest like Iphone6S have 2GB.
To be frank, i do not think it makes sense to have so many documents. I do not know your use case but probably that only a subset is required.

As a general way to see things you can work into two directions:
1- group data in small logical packages which get replicated when user requires them (using channels)
2- get some of the computations requiring a lot of data done on backend side and send back the result

Hope it helps,
Steve.