Change to defaulting to ordering async txns after single operations

README.md

@@ -308,11 +308,11 @@ While caching can improve performance, LMDB itself is extremely fast, and for sm
 If you are using caching with a database that has versions enabled, you should use the `getEntry` method to get the `value` and `version`, as `getLastVersion` will not be reliable (only returns the version when the data is accessed from the database).
 
 ### Asynchronous Transaction Ordering
-Asynchronous single operations (`put` and `remove`) are executed in the order they were called, relative to each other. Likewise, asynchronous transaction callbacks (`transactionAsync` and `childTransaction`) are also executed in order relative to other asynchronous transaction callbacks. However, by default all queued asynchronous transaction callbacks are executed _before_ all queued asynchronous single operations. But, you can enable strict ordering so that asynchronous transactions executed in order _with_ the asynchronous single operations, by enabling the `strictAsyncOrder` option (setting to `true`).
+Asynchronous single operations (`put` and `remove`) are executed in the order they were called, relative to each other. Likewise, asynchronous transaction callbacks (`transactionAsync` and `childTransaction`) are also executed in order relative to other asynchronous transaction callbacks. However, by default all queued asynchronous transaction callbacks are executed _after_ all queued asynchronous single operations. But, you can enable strict ordering so that asynchronous transactions executed in order _with_ the asynchronous single operations, by setting the `asyncTransactionOrder` property to 'strict'.
 
-However, `strictAsyncOrder` comes with a couple of caveats. First, because lmdb-store executes asynchronous single operations on a separate transaction thread, but asynchronous transaction callbacks must execute on the main JS thread, if there is a lot of frequent switching back and forth between single operations and callbacks, this can significantly reduce performance since it requires substantial thread switching and event queuing.
+However, strict ordering comes with a couple of caveats. First, because lmdb-store executes asynchronous single operations on a separate transaction thread, but asynchronous transaction callbacks must execute on the main JS thread, if there is a lot of frequent switching back and forth between single operations and callbacks, this can significantly reduce performance since it requires substantial thread switching and event queuing.
 
-Second, if there are asynchronous operations that have been performed, and asynchronous transaction callbacks that waiting to be called, and a synchronous transaction is executed (`transactionSync`), this must interrupt and split the current asynchronous transaction batch, so the synchronous transaction can be executed (the synchronous transaction can not block to wait for the asynchronous if there are outstanding callbacks to execute as part of that async transaction, as that would result in a deadlock). This can potentially create an exception to the general rule that all asynchronous operations that are performed in one event turn will be part of the same transaction. Of course each single asynchronous transaction callback is still guaranteed to execute in a single atomic transaction (and calls to `transactionSync` _during_ a asynchronous transaction callback are simply executed as part of the current transaction).
+Second, if there are asynchronous operations that have been performed, and asynchronous transaction callbacks that are waiting to be called, and a synchronous transaction is executed (`transactionSync`), this must interrupt and split the current asynchronous transaction batch, so the synchronous transaction can be executed (the synchronous transaction can not block to wait for the asynchronous if there are outstanding callbacks to execute as part of that async transaction, as that would result in a deadlock). This can potentially create an exception to the general rule that all asynchronous operations that are performed in one event turn will be part of the same transaction. Of course, each single asynchronous transaction callback is still guaranteed to execute in a single atomic transaction (and calls to `transactionSync` _during_ a asynchronous transaction callback are simply executed as part of the current transaction). With the default ordering of 'after', it is possible for the async transactions to be performed in a separate transaction than the single operations if executed. Setting the ordering to 'before' ensures they are always in the same transaction.
 
 ### Store Options
 The open method can be used to create the main database/environment with the following signature:

index.d.ts

@@ -104,7 +104,7 @@ declare namespace lmdb {
 		**/
 		getRange(options: RangeOptions): ArrayLikeIterable<{ key: K, value: V, version?: number }>
 		/**
-		* @deprecated Since v1.3.0, this will be replaced with the functionality of asyncTransaction in a future release
+		* @deprecated Since v1.3.0, this will be replaced with the functionality of transactionAsync in a future release
 		**/
 		transaction<T>(action: () => T): T
 		/**
@@ -225,6 +225,7 @@ declare namespace lmdb {
 		maxDbs?: number
 		/** Set a longer delay (in milliseconds) to wait longer before committing writes to increase the number of writes per transaction (higher latency, but more efficient) **/
 		commitDelay?: number
+		asyncTransactionOrder?: 'after' | 'before' | 'strict'
 		mapSize?: number
 		pageSize?: number
 		remapChunks?: boolean

index.js

@@ -38,6 +38,7 @@ function open(path, options) {
 	let committingWrites
 	let scheduledTransactions
 	let scheduledOperations
+	let asyncTransactionAfter = true, asyncTransactionStrictOrder
 	let transactionWarned
 	let readTxn, writeTxn, pendingBatch, currentCommit, runNextBatch, readTxnRenewed, cursorTxns = []
 	let renewId = 1
@@ -64,6 +65,12 @@ function open(path, options) {
 		mapSize: remapChunks ? 0x10000000000000 :
 			0x20000, // Otherwise we start small with 128KB
 	}, options)
+	if (options.asyncTransactionOrder == 'before')
+		asyncTransactionAfter = false
+	else if (options.asyncTransactionOrder == 'strict') {
+		asyncTransactionStrictOrder = true
+		asyncTransactionAfter = false
+	}
 	if (!fs.existsSync(options.noSubdir ? dirname(path) : path))
 		mkdirpSync(options.noSubdir ? dirname(path) : path)
 	if (options.compression) {
@@ -216,9 +223,10 @@ function open(path, options) {
 				return callback()
 			}
 			let lastOperation
-			let inOrder = this.strictAsyncOrder
+			let after, strictOrder
 			if (scheduledOperations) {
-				lastOperation = scheduledOperations[inOrder ? scheduledOperations.length - 1 : 0]
+				lastOperation = asyncTransactionAfter ? scheduledOperations.appendAsyncTxn :
+					scheduledOperations[asyncTransactionStrictOrder ? scheduledOperations.length - 1 : 0]
 			} else {
 				scheduledOperations = []
 				scheduledOperations.bytes = 0
@@ -230,9 +238,11 @@ function open(path, options) {
 				transactionSet = scheduledTransactions[transactionSetIndex]
 			} else {
 				// for now we signify transactions as a true
-				if (inOrder)
+				if (asyncTransactionAfter) // by default we add a flag to put transactions after other operations
+					scheduledOperations.appendAsyncTxn = true
+				else if (asyncTransactionStrictOrder)
 					scheduledOperations.push(true)
-				else // put all the async transaction at the beginning by default
+				else // in before mode, we put all the async transaction at the beginning
 					scheduledOperations.unshift(true)
 				if (!scheduledTransactions) {
 					scheduledTransactions = []
@@ -796,6 +806,9 @@ function open(path, options) {
 							// operations to perform, collect them as an array and start doing them
 							let operations = scheduledOperations || []
 							let transactions = scheduledTransactions
+							if (operations.appendAsyncTxn) {
+								operations.push(true)
+							}
 							scheduledOperations = null
 							scheduledTransactions = null
 							const writeBatch = () => {

package.json

@@ -41,7 +41,7 @@
     "weak-lru-cache": "^0.4.1"
   },
   "optionalDependencies": {
-    "msgpackr": "^1.2.10"
+    "msgpackr": "^1.3.2"
   },
   "devDependencies": {
     "@types/node": "latest",

test/index.test.js

@@ -48,7 +48,7 @@ describe('lmdb-store', function() {
         name: 'mydb3',
         create: true,
         useVersions: true,
-        strictAsyncOrder: true,
+        //asyncTransactionOrder: 'strict',
         compression: {
           threshold: 256,
         },
@@ -386,32 +386,6 @@ describe('lmdb-store', function() {
         should.equal(db.get('key3'), 'test-async-child-txn');
       })
     });
-    it('async transaction with interrupting sync transaction in order', async function() {
-      db.strictAsyncOrder = true
-      let order = []
-      let ranSyncTxn
-      db.transactionAsync(() => {
-        order.push('a1');
-        db.put('async1', 'test');
-        if (!ranSyncTxn) {
-          ranSyncTxn = true;
-          setImmediate(() => db.transactionSync(() => {
-            order.push('s1');
-            db.put('inside-sync', 'test');
-          }));
-        }
-      });
-      db.put('outside-txn', 'test');
-      await db.transactionAsync(() => {
-        order.push('a2');
-        db.put('async2', 'test');
-      });
-      order.should.deep.equal(['a1', 's1', 'a2']);
-      should.equal(db.get('async1'), 'test');
-      should.equal(db.get('outside-txn'), 'test');
-      should.equal(db.get('inside-sync'), 'test');
-      should.equal(db.get('async2'), 'test');
-    });
     it('async transaction with interrupting sync transaction default order', async function() {
       db.strictAsyncOrder = false
       let order = []