Skip to content

Commit 5f9a95f

Browse files
committed
Major refactor to introduce safer patterns
- **Added new concurrency model system** - Created `AndroidxSqliteConcurrencyModel` with multiple concurrency strategies - Added support for single reader/writer, multiple readers, and multiple readers with single writer patterns - Enhanced WAL mode support with configurable reader connection counts - **Refactored driver architecture** - Split `AndroidxSqliteDriver` into focused components: - `AndroidxSqliteExecutingDriver` - handles SQL execution - `AndroidxSqliteDriverHolder` - manages schema initialization and lifecycle - `AndroidxSqliteConfigurableDriver` - provides driver configuration - Improved separation of concerns and testability - **Enhanced SQL handling and safety** - Added `AndroidxSqliteSpecialCase` enum for special SQL operations - Created `AndroidxSqliteUtils` for SQL parsing and analysis - Improved journal mode setting with dedicated connection handling - Enhanced foreign key constraint validation - **Improved connection pool management** - Refactored `ConnectionPool` with better concurrency control - Added safer connection acquisition/release patterns - Enhanced transaction handling with proper connection isolation - **Expanded test coverage** - Added comprehensive `ConnectionPoolTest` suite - Added `AndroidxSqliteUtilsTest` for SQL parsing validation - Updated existing tests to work with new architecture - **Documentation improvements** - Updated README with new concurrency model documentation - Added detailed API documentation for new components
1 parent 230c8c2 commit 5f9a95f

19 files changed

+2122
-709
lines changed

README.md

Lines changed: 141 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -110,39 +110,166 @@ have been introduced during the migration.
110110
111111
## Connection Pooling
112112

113-
By default, one connection will be used for both reading and writing, and only one thread can acquire that connection
114-
at a time. If you have WAL enabled, you could (and should) set the amount of pooled reader connections that will be used:
113+
SQLite supports several concurrency models that can significantly impact your application's performance. This driver
114+
provides flexible connection pooling through the `AndroidxSqliteConcurrencyModel` interface.
115+
116+
### Available Concurrency Models
117+
118+
#### 1. SingleReaderWriter
119+
120+
The simplest model with one connection handling all operations:
115121

116122
```kotlin
117-
AndroidxSqliteDriver(
118-
...,
119-
readerConnections = 4,
120-
...,
123+
AndroidxSqliteConfiguration(
124+
concurrencyModel = AndroidxSqliteConcurrencyModel.SingleReaderWriter
121125
)
122126
```
123127

124-
On Android you can defer to the system to determine how many reader connections there should be<sup>[1]</sup>:
128+
**Best for:**
129+
130+
- Simple applications with minimal database usage
131+
- Testing and development
132+
- When memory usage is a primary concern
133+
- Single-threaded applications
134+
135+
#### 2. MultipleReaders
136+
137+
Dedicated reader connections for read-only access:
138+
139+
```kotlin
140+
AndroidxSqliteConfiguration(
141+
concurrencyModel = AndroidxSqliteConcurrencyModel.MultipleReaders(
142+
readerCount = 3 // Number of concurrent reader connections
143+
)
144+
)
145+
```
146+
147+
**Best for:**
148+
149+
- Read-only applications (analytics dashboards, reporting tools)
150+
- Data visualization and content browsing applications
151+
- Scenarios where all writes happen externally (data imports, ETL processes)
152+
- Applications that only query pre-populated databases
153+
154+
**Important:** This model is designed for **read-only access**. No write operations (INSERT, UPDATE, DELETE) should be
155+
performed. If you need write capabilities, use `MultipleReadersSingleWriter` in WAL mode instead.
156+
157+
#### 3. MultipleReadersSingleWriter (Recommended)
158+
159+
The most flexible model that adapts based on journal mode:
160+
161+
```kotlin
162+
AndroidxSqliteConfiguration(
163+
concurrencyModel = AndroidxSqliteConcurrencyModel.MultipleReadersSingleWriter(
164+
isWal = true, // Enable WAL mode for true concurrency
165+
walCount = 4, // Reader connections when WAL is enabled
166+
nonWalCount = 0 // Reader connections when WAL is disabled
167+
)
168+
)
169+
```
170+
171+
**Best for:**
172+
173+
- Most production applications
174+
- Mixed read/write workloads
175+
- When you want to leverage WAL mode benefits
176+
- Applications requiring optimal performance
177+
178+
### WAL Mode Benefits
179+
180+
- **True Concurrency**: Readers and writers don't block each other
181+
- **Better Performance**: Concurrent operations improve throughput
182+
- **Consistency**: ACID properties are maintained (when `PRAGMA synchronous = FULL` is used)
183+
- **Scalability**: Handles higher concurrent load
184+
185+
### Choosing Reader Connection Count
186+
187+
The optimal number of reader connections depends on your use case:
188+
189+
```kotlin
190+
// Conservative (default)
191+
AndroidxSqliteConcurrencyModel.MultipleReadersSingleWriter(
192+
isWal = true,
193+
walCount = 4,
194+
nonWalCount = 0,
195+
)
196+
197+
// High-concurrency applications
198+
AndroidxSqliteConcurrencyModel.MultipleReadersSingleWriter(
199+
isWal = true,
200+
walCount = 8
201+
)
202+
203+
// Memory-conscious applications
204+
AndroidxSqliteConcurrencyModel.MultipleReadersSingleWriter(
205+
isWal = true,
206+
walCount = 2
207+
)
208+
```
209+
210+
### Platform-Specific Configuration
211+
212+
On Android, you can use system-determined connection pool sizes:
125213

126214
```kotlin
127215
// Based on SQLiteGlobal.getWALConnectionPoolSize()
128-
fun getWALConnectionPoolSize() {
216+
fun getWALConnectionPoolSize(): Int {
129217
val resources = Resources.getSystem()
130-
val resId =
131-
resources.getIdentifier("db_connection_pool_size", "integer", "android")
218+
val resId = resources.getIdentifier("db_connection_pool_size", "integer", "android")
132219
return if (resId != 0) {
133220
resources.getInteger(resId)
134221
} else {
135-
2
222+
2 // Fallback default
136223
}
137224
}
225+
226+
AndroidxSqliteConfiguration(
227+
concurrencyModel = AndroidxSqliteConcurrencyModel.MultipleReadersSingleWriter(
228+
isWal = true,
229+
walCount = getWALConnectionPoolSize(),
230+
nonWalCount = 0,
231+
)
232+
)
138233
```
139234

140-
See [WAL & Dispatchers] for more information about how to configure dispatchers to use for reads and writes.
235+
### Performance Considerations
236+
237+
| Model | Memory Usage | Read Concurrency | Write Capability | Best Use Case |
238+
|---------------------------------------|--------------|------------------|------------------|--------------------|
239+
| SingleReaderWriter | Lowest | None | Full | Simple apps |
240+
| MultipleReaders | Medium | Excellent | None (read-only) | Read-only apps |
241+
| MultipleReadersSingleWriter (WAL) | Higher | Excellent | Full | Production |
242+
| MultipleReadersSingleWriter (non-WAL) | Medium | Limited | Full | Legacy/constrained |
243+
244+
### Special Database Types
141245

142246
> [!NOTE]
143-
> In-Memory and temporary databases will always use 0 reader connections i.e. there will be a single connection
247+
> In-Memory and temporary databases automatically use `SingleReaderWriter` model regardless of configuration, as
248+
> connection pooling provides no benefit for these database types.
249+
250+
### Journal Mode
251+
252+
If `PRAGMA journal_mode = ...` is used, the connection pool will:
253+
254+
1. Acquire the writer connection
255+
2. Acquire all reader connections
256+
3. Close all reader connections
257+
4. Run the `PRAGMA` statement
258+
5. Recreate the reader connections
259+
260+
This ensures all connections use the same journal mode and prevents inconsistencies.
261+
262+
### Best Practices
263+
264+
1. **Start with defaults**: Uses `MultipleReadersSingleWriter` in WAL mode
265+
2. **Monitor performance**: Profile your specific workload to determine optimal reader count
266+
3. **Consider memory**: Each connection has overhead - balance performance vs memory usage
267+
4. **Test thoroughly**: Verify your concurrency model works under expected load
268+
5. **Platform differences**: Android may have different optimal settings than JVM/Native
269+
270+
See [WAL & Dispatchers] for more information about how to configure dispatchers to use for reads and writes.
144271

145-
[1]: https://blog.p-y.wtf/parallelism-with-android-sqlite#heading-secondary-connections
146272
[AndroidX Kotlin Multiplatform SQLite]: https://developer.android.com/kotlin/multiplatform/sqlite
147273
[SQLDelight]: https://github.com/sqldelight/sqldelight
148274
[WAL & Dispatchers]: https://blog.p-y.wtf/parallelism-with-android-sqlite#heading-wal-amp-dispatchers
275+
[Write-Ahead Logging]: https://sqlite.org/wal.html

integration/src/commonTest/kotlin/com/eygraber/sqldelight/androidx/driver/integration/AndroidxSqliteConcurrencyIntegrationTest.kt

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package com.eygraber.sqldelight.androidx.driver.integration
22

33
import app.cash.sqldelight.coroutines.asFlow
4+
import com.eygraber.sqldelight.androidx.driver.AndroidxSqliteConcurrencyModel.MultipleReadersSingleWriter
45
import com.eygraber.sqldelight.androidx.driver.AndroidxSqliteConfiguration
56
import com.eygraber.sqldelight.androidx.driver.AndroidxSqliteDatabaseType
67
import kotlinx.coroutines.delay
@@ -23,7 +24,10 @@ class AndroidxSqliteConcurrencyIntegrationTest : AndroidxSqliteIntegrationTest()
2324
// having 2 readers instead of the default 4 makes it more
2425
// likely to have concurrent readers using the same cached statement
2526
configuration = AndroidxSqliteConfiguration(
26-
readerConnectionsCount = 2,
27+
concurrencyModel = MultipleReadersSingleWriter(
28+
isWal = true,
29+
walCount = 2,
30+
),
2731
)
2832

2933
launch {

integration/src/commonTest/kotlin/com/eygraber/sqldelight/androidx/driver/integration/AndroidxSqliteIntegrationTest.kt

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,8 @@ abstract class AndroidxSqliteIntegrationTest {
2222

2323
@OptIn(ExperimentalCoroutinesApi::class, DelicateCoroutinesApi::class)
2424
private fun readDispatcher(): CoroutineDispatcher? = when {
25-
configuration.readerConnectionsCount >= 1 -> newFixedThreadPoolContext(
26-
nThreads = configuration.readerConnectionsCount,
25+
configuration.concurrencyModel.readerCount >= 1 -> newFixedThreadPoolContext(
26+
nThreads = configuration.concurrencyModel.readerCount,
2727
name = "db-reads",
2828
)
2929
else -> null
Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
package com.eygraber.sqldelight.androidx.driver
2+
3+
/**
4+
* Defines the concurrency model for SQLite database connections, controlling how many
5+
* reader and writer connections are maintained in the connection pool.
6+
*
7+
* SQLite supports different concurrency models depending on the journal mode and application needs:
8+
* - Single connection for simple use cases
9+
* - Multiple readers with WAL (Write-Ahead Logging) for better read concurrency
10+
* - Configurable reader counts for fine-tuned performance
11+
*
12+
* @property readerCount The number of reader connections to maintain in the pool
13+
*/
14+
public sealed interface AndroidxSqliteConcurrencyModel {
15+
public val readerCount: Int
16+
17+
/**
18+
* Single connection model - one connection handles both reads and writes.
19+
*
20+
* This is the simplest and most conservative approach, suitable for:
21+
* - Applications with low concurrency requirements
22+
* - Simple database operations
23+
* - Testing scenarios
24+
* - When database contention is not a concern
25+
*
26+
* **Performance characteristics:**
27+
* - Lowest memory overhead
28+
* - No connection pooling complexity
29+
* - Sequential read/write operations only
30+
* - Suitable for single-threaded or low-concurrency scenarios
31+
*/
32+
public data object SingleReaderWriter : AndroidxSqliteConcurrencyModel {
33+
override val readerCount: Int = 0
34+
}
35+
36+
/**
37+
* Multiple readers model - allows concurrent read operations only.
38+
*
39+
* This model creates a pool of dedicated reader connections for read-only access.
40+
* **No write operations should be performed** when using this model.
41+
*
42+
* **Use cases:**
43+
* - Read-only applications (analytics dashboards, reporting tools)
44+
* - Data visualization and content browsing applications
45+
* - Scenarios where all writes happen externally (e.g., data imports)
46+
* - Applications that only query pre-populated databases
47+
*
48+
* **Performance characteristics:**
49+
* - Excellent read concurrency
50+
* - Higher memory overhead due to connection pooling
51+
* - No write capability - reads only
52+
* - Optimal for read-heavy workloads with no database modifications
53+
*
54+
* **Important:** This model is designed for read-only access. If your application
55+
* needs to perform any write operations (INSERT, UPDATE, DELETE, schema changes),
56+
* use `MultipleReadersSingleWriter` in WAL mode instead.
57+
*
58+
* @param readerCount Number of reader connections to maintain (typically 2-8)
59+
*/
60+
public data class MultipleReaders(
61+
override val readerCount: Int,
62+
) : AndroidxSqliteConcurrencyModel
63+
64+
/**
65+
* Multiple readers with single writer model - optimized for different journal modes.
66+
*
67+
* This is the most flexible model that adapts its behavior based on whether
68+
* Write-Ahead Logging (WAL) mode is enabled:
69+
*
70+
* **WAL Mode (isWal = true):**
71+
* - Enables true concurrent reads and writes
72+
* - Readers don't block writers and vice versa
73+
* - Best performance for mixed read/write workloads
74+
* - Uses `walCount` reader connections
75+
*
76+
* **Non-WAL Mode (isWal = false):**
77+
* - Falls back to traditional SQLite locking
78+
* - Reads and writes are still serialized
79+
* - Uses `nonWalCount` reader connections (typically 0)
80+
*
81+
* **Recommended configuration:**
82+
* ```kotlin
83+
* // For WAL mode
84+
* MultipleReadersSingleWriter(
85+
* isWal = true,
86+
* walCount = 4 // Good default for most applications
87+
* )
88+
*
89+
* // For non-WAL mode
90+
* MultipleReadersSingleWriter(
91+
* isWal = false,
92+
* nonWalCount = 0 // Single connection is often sufficient
93+
* )
94+
* ```
95+
*
96+
* @param isWal Whether WAL (Write-Ahead Logging) journal mode is enabled
97+
* @param nonWalCount Number of reader connections when WAL is disabled (default: 0)
98+
* @param walCount Number of reader connections when WAL is enabled (default: 4)
99+
*/
100+
public data class MultipleReadersSingleWriter(
101+
public val isWal: Boolean,
102+
public val nonWalCount: Int = 0,
103+
public val walCount: Int = 4,
104+
) : AndroidxSqliteConcurrencyModel {
105+
override val readerCount: Int = when {
106+
isWal -> walCount
107+
else -> nonWalCount
108+
}
109+
}
110+
}

library/src/commonMain/kotlin/com/eygraber/sqldelight/androidx/driver/AndroidxSqliteConfigurableDriver.kt

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,21 +2,23 @@ package com.eygraber.sqldelight.androidx.driver
22

33
import app.cash.sqldelight.db.QueryResult
44
import app.cash.sqldelight.db.SqlCursor
5+
import app.cash.sqldelight.db.SqlDriver
56
import app.cash.sqldelight.db.SqlPreparedStatement
67

78
public class AndroidxSqliteConfigurableDriver(
8-
private val driver: AndroidxSqliteDriver,
9+
private val driver: SqlDriver,
910
) {
1011
public fun setForeignKeyConstraintsEnabled(isForeignKeyConstraintsEnabled: Boolean) {
11-
driver.setForeignKeyConstraintsEnabled(isForeignKeyConstraintsEnabled)
12+
val foreignKey = if(isForeignKeyConstraintsEnabled) "ON" else "OFF"
13+
executePragma("foreign_keys = $foreignKey")
1214
}
1315

1416
public fun setJournalMode(journalMode: SqliteJournalMode) {
15-
driver.setJournalMode(journalMode)
17+
executePragma("journal_mode = ${journalMode.value}")
1618
}
1719

1820
public fun setSync(sync: SqliteSync) {
19-
driver.setSync(sync)
21+
executePragma("synchronous = ${sync.value}")
2022
}
2123

2224
public fun executePragma(
@@ -27,10 +29,10 @@ public class AndroidxSqliteConfigurableDriver(
2729
driver.execute(null, "PRAGMA $pragma;", parameters, binders)
2830
}
2931

30-
public fun <T> executePragmaQuery(
32+
public fun <R> executePragmaQuery(
3133
pragma: String,
32-
mapper: (SqlCursor) -> QueryResult<T>,
34+
mapper: (SqlCursor) -> QueryResult<R>,
3335
parameters: Int = 0,
3436
binders: (SqlPreparedStatement.() -> Unit)? = null,
35-
): QueryResult.Value<T> = driver.executeQuery(null, "PRAGMA $pragma;", mapper, parameters, binders)
37+
): QueryResult<R> = driver.executeQuery(null, "PRAGMA $pragma;", mapper, parameters, binders)
3638
}

0 commit comments

Comments
 (0)