Skip to content

Commit

Permalink
feat: Database.new supports extensions: option parameter
Browse files Browse the repository at this point in the history
This commit updates the Database documentation with lots of examples
and explanation, but to sum up:

   Database.new(":memory:", extensions: ["SQLean::Crypto"])

which allows injection of extensions in a Rails database config:

    development:
      adapter: sqlite3
      database: storage/development.sqlite3
      extensions:
        - SQLean::Crypto
  • Loading branch information
flavorjones committed Nov 25, 2024
1 parent ceb76cc commit c7d625a
Show file tree
Hide file tree
Showing 2 changed files with 204 additions and 53 deletions.
147 changes: 110 additions & 37 deletions lib/sqlite3/database.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,10 @@
require "sqlite3/fork_safety"

module SQLite3
# The Database class encapsulates a single connection to a SQLite3 database.
# Its usage is very straightforward:
# == Overview
#
# The Database class encapsulates a single connection to a SQLite3 database. Here's a
# straightforward example of usage:
#
# require 'sqlite3'
#
Expand All @@ -19,28 +21,72 @@ module SQLite3
# end
# end
#
# It wraps the lower-level methods provided by the selected driver, and
# includes the Pragmas module for access to various pragma convenience
# methods.
# It wraps the lower-level methods provided by the selected driver, and includes the Pragmas
# module for access to various pragma convenience methods.
#
# The Database class provides type translation services as well, by which
# the SQLite3 data types (which are all represented as strings) may be
# converted into their corresponding types (as defined in the schemas
# for their tables). This translation only occurs when querying data from
# The Database class provides type translation services as well, by which the SQLite3 data types
# (which are all represented as strings) may be converted into their corresponding types (as
# defined in the schemas for their tables). This translation only occurs when querying data from
# the database--insertions and updates are all still typeless.
#
# Furthermore, the Database class has been designed to work well with the
# ArrayFields module from Ara Howard. If you require the ArrayFields
# module before performing a query, and if you have not enabled results as
# hashes, then the results will all be indexible by field name.
# Furthermore, the Database class has been designed to work well with the ArrayFields module from
# Ara Howard. If you require the ArrayFields module before performing a query, and if you have not
# enabled results as hashes, then the results will all be indexible by field name.
#
# == Thread safety
#
# When SQLite3.threadsafe? returns true, it is safe to share instances of the database class
# among threads without adding specific locking. Other object instances may require applications
# to provide their own locks if they are to be shared among threads. Please see the README.md for
# more information.
#
# == SQLite Extensions
#
# SQLite3::Database supports the universe of {sqlite
# extensions}[https://www.sqlite.org/loadext.html]. It's possible to load an extension into an
# existing Database object using the #load_extension method and passing a filesystem path:
#
# db = SQLite3::Database.new(":memory:")
# db.enable_load_extension(true)
# db.load_extension("/path/to/extension")
#
# As of v2.4.0, it's also possible to pass an object that responds to
# +#sqlite_extension_path+. This documentation will refer to the supported interface as
# +_ExtensionSpecifier+, which can be expressed in RBS syntax as:
#
# interface _ExtensionSpecifier
# def sqlite_extension_path: () → String
# end
#
# So, for example, if you are using the {sqlean gem}[https://github.com/flavorjones/sqlean-ruby]
# which provides modules that implement this interface, you can pass the module directly:
#
# db = SQLite3::Database.new(":memory:")
# db.enable_load_extension(true)
# db.load_extension(SQLean::Crypto)
#
# It's also possible in v2.4.0+ to load extensions via the SQLite3::Database constructor by using
# the +extensions:+ keyword argument to pass an array of strings or extension specifiers:
#
# db = SQLite3::Database.new(":memory:", extensions: ["/path/to/extension", SQLean::Crypto])
#
# Note that the constructor will implicitly call #enable_load_extension if the +extensions:+
# keyword argument is present.
#
# Finally, it's also possible to load an extension by passing the _name_ of a constant that
# implements the +_ExtensionSpecifier+ interface:
#
# Thread safety:
# db = SQLite3::Database.new(":memory:", extensions: ["SQLean::Crypto"])
#
# Handling the name of a constant like this is what enables injection of an extension in a Rails
# application's `config/database.yml` file, like this:
#
# development:
# adapter: sqlite3
# database: storage/development.sqlite3
# extensions:
# - SQLean::Crypto
#
# When `SQLite3.threadsafe?` returns true, it is safe to share instances of
# the database class among threads without adding specific locking. Other
# object instances may require applications to provide their own locks if
# they are to be shared among threads. Please see the README.md for more
# information.
class Database
attr_reader :collations

Expand Down Expand Up @@ -76,23 +122,25 @@ def quote(string)
# as hashes or not. By default, rows are returned as arrays.
attr_accessor :results_as_hash

# call-seq: SQLite3::Database.new(file, options = {})
# call-seq:
# SQLite3::Database.new(file, options = {})
#
# Create a new Database object that opens the given file.
#
# Supported permissions +options+:
# - the default mode is <tt>READWRITE | CREATE</tt>
# - +:readonly+: boolean (default false), true to set the mode to +READONLY+
# - +:readwrite+: boolean (default false), true to set the mode to +READWRITE+
# - +:flags+: set the mode to a combination of SQLite3::Constants::Open flags.
# - +readonly:+ boolean (default false), true to set the mode to +READONLY+
# - +readwrite:+ boolean (default false), true to set the mode to +READWRITE+
# - +flags:+ set the mode to a combination of SQLite3::Constants::Open flags.
#
# Supported encoding +options+:
# - +:utf16+: boolean (default false), is the filename's encoding UTF-16 (only needed if the filename encoding is not UTF_16LE or BE)
# - +utf16:+ +boolish+ (default false), is the filename's encoding UTF-16 (only needed if the filename encoding is not UTF_16LE or BE)
#
# Other supported +options+:
# - +:strict+: boolean (default false), disallow the use of double-quoted string literals (see https://www.sqlite.org/quirks.html#double_quoted_string_literals_are_accepted)
# - +:results_as_hash+: boolean (default false), return rows as hashes instead of arrays
# - +:default_transaction_mode+: one of +:deferred+ (default), +:immediate+, or +:exclusive+. If a mode is not specified in a call to #transaction, this will be the default transaction mode.
# - +strict:+ +boolish+ (default false), disallow the use of double-quoted string literals (see https://www.sqlite.org/quirks.html#double_quoted_string_literals_are_accepted)
# - +results_as_hash:+ +boolish+ (default false), return rows as hashes instead of arrays
# - +default_transaction_mode:+ one of +:deferred+ (default), +:immediate+, or +:exclusive+. If a mode is not specified in a call to #transaction, this will be the default transaction mode.
# - +extensions:+ <tt>Array[String | _ExtensionSpecifier]</tt> SQLite extensions to load into the database. See Database@SQLite+Extensions for more information.
#
def initialize file, options = {}, zvfs = nil
mode = Constants::Open::READWRITE | Constants::Open::CREATE
Expand Down Expand Up @@ -135,6 +183,8 @@ def initialize file, options = {}, zvfs = nil
@readonly = mode & Constants::Open::READONLY != 0
@default_transaction_mode = options[:default_transaction_mode] || :deferred

marshal_extensions(options[:extensions])

ForkSafety.track(self)

if block_given?
Expand Down Expand Up @@ -659,36 +709,59 @@ def busy_handler_timeout=(milliseconds)
end

# call-seq:
# load_extension(extension_specifier) -> Database
# load_extension(extension_specifier) -> self
#
# Loads an SQLite extension library from the named file. Extension loading must be enabled using
# Database#enable_load_extension prior to using this method.
# #enable_load_extension prior to using this method.
#
# See also: https://www.sqlite.org/loadext.html
# See also: Database@SQLite+Extensions
#
# [Parameters]
# - extension_specifier (String | +_ExtensionSpecifier+) If a String, it is the filesystem path
# - +extension_specifier+: (String | +_ExtensionSpecifier+) If a String, it is the filesystem path
# to the sqlite extension file. If an object that responds to #sqlite_extension_path, the
# return value of that method is used as the filesystem path to the sqlite extension file.
#
# +_ExtensionSpecifier+ describes the following interface:
# [Example] Using a filesystem path:
#
# interface _ExtensionSpecifier
# def sqlite_extension_path: () → String
# end
# db.load_extension("/path/to/my_extension.so")
#
# For example, the +sqlean+ ruby gem offers a set of classes that implement this interface,
# allowing a calling convention like:
# [Example] Using the {sqlean gem}[https://github.com/flavorjones/sqlean-ruby]:
#
# db.load_extension(SQLean::VSV)
#
def load_extension(extension_specifier)
if extension_specifier.respond_to?(:sqlite_extension_path)
extension_specifier = extension_specifier.sqlite_extension_path
elsif !extension_specifier.is_a?(String)
raise TypeError, "extension_specifier #{extension_specifier.inspect} is not a String or a valid extension specifier object"
end
load_extension_internal(extension_specifier)
end

def marshal_extensions(extensions) # :nodoc:
return if extensions.nil?
raise TypeError, "extensions must be an Array" unless extensions.is_a?(Array)
return if extensions.empty?

enable_load_extension(true)

extensions.each do |extension|
# marshall the extension into an object if it's the name of a constant that responds to
# `#sqlite_extension_path`
if extension.is_a?(String)
begin
extension_spec = Object.const_get(extension)
if extension_spec.respond_to?(:sqlite_extension_path)
extension = extension_spec
end
rescue NameError
end
end

load_extension(extension)
end
end

# A helper class for dealing with custom functions (see #create_function,
# #create_aggregate, and #create_aggregate_handler). It encapsulates the
# opaque function object that represents the current invocation. It also
Expand Down
110 changes: 94 additions & 16 deletions test/test_database.rb
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,12 @@
require "pathname"

module SQLite3
class FakeExtensionSpecifier
def self.sqlite_extension_path
"/path/to/extension"
end
end

class TestDatabase < SQLite3::TestCase
attr_reader :db

Expand All @@ -15,6 +21,17 @@ def teardown
@db.close unless @db.closed?
end

def mock_database_load_extension_internal(db)
class << db
attr_reader :load_extension_internal_path

def load_extension_internal(path)
@load_extension_internal_path ||= []
@load_extension_internal_path << path
end
end
end

def test_custom_function_encoding
@db.execute("CREATE TABLE
sourceTable(
Expand Down Expand Up @@ -650,37 +667,98 @@ def test_strict_mode
assert_match(/no such column: "?nope"?/, error.message)
end

def test_load_extension_with_nonstring_argument
db = SQLite3::Database.new(":memory:")
def test_load_extension_error_with_nonexistent_path
skip("extensions are not enabled") unless db.respond_to?(:load_extension)
db.enable_load_extension(true)

assert_raises(SQLite3::Exception) { db.load_extension("/path/to/extension") }
end

def test_load_extension_error_with_invalid_argument
skip("extensions are not enabled") unless db.respond_to?(:load_extension)

assert_raises(TypeError) { db.load_extension(1) }
assert_raises(TypeError) { db.load_extension(Pathname.new("foo.so")) }
assert_raises(TypeError) { db.load_extension(Pathname.new("foo")) }
end

def test_load_extension_with_an_extension_descriptor
extension_descriptor = Class.new do
def sqlite_extension_path
"path/to/extension"
end
end.new
mock_database_load_extension_internal(db)

db.load_extension(FakeExtensionSpecifier)

assert_equal(["/path/to/extension"], db.load_extension_internal_path)
end

def test_marshal_extensions_with_extensions_calls_enable_load_extension
mock_database_load_extension_internal(db)
class << db
attr_reader :load_extension_internal_path
attr_reader :enable_load_extension_called

def load_extension_internal(path)
@load_extension_internal_path = path
def enable_load_extension(...)
@enable_load_extension_called = true
end
end

db.load_extension(extension_descriptor)
db.marshal_extensions(nil)
refute(db.enable_load_extension_called)

db.marshal_extensions([])
refute(db.enable_load_extension_called)

assert_equal("path/to/extension", db.load_extension_internal_path)
db.marshal_extensions([FakeExtensionSpecifier])
assert(db.enable_load_extension_called)
end

def test_load_extension_error
db = SQLite3::Database.new(":memory:")
assert_raises(SQLite3::Exception) { db.load_extension("path/to/foo.so") }
def test_marshal_extensions_object_is_an_extension_specifier
mock_database_load_extension_internal(db)

db.marshal_extensions([FakeExtensionSpecifier])
assert_equal(["/path/to/extension"], db.load_extension_internal_path)

db.load_extension_internal_path.clear # reset

db.marshal_extensions(["SQLite3::FakeExtensionSpecifier"])
assert_equal(["/path/to/extension"], db.load_extension_internal_path)

db.load_extension_internal_path.clear # reset

db.marshal_extensions(["CannotBeResolved"])
assert_equal(["CannotBeResolved"], db.load_extension_internal_path)
end

def test_marshal_extensions_object_not_an_extension_specifier
mock_database_load_extension_internal(db)

db.marshal_extensions(["CannotBeResolved"])
assert_equal(["CannotBeResolved"], db.load_extension_internal_path)

assert_raises(TypeError) { db.marshal_extensions([Class.new]) }

assert_raises(TypeError) { db.marshal_extensions(FakeExtensionSpecifier) }
end

def test_initialize_with_extensions_calls_marshal_extensions
# ephemeral class to capture arguments passed to marshal_extensions
klass = Class.new(SQLite3::Database) do
attr :marshal_extensions_called, :marshal_extensions_arg

def marshal_extensions(extensions)
@marshal_extensions_called = true
@marshal_extensions_arg = extensions
end
end

db = klass.new(":memory:")
assert(db.marshal_extensions_called)
assert_nil(db.marshal_extensions_arg)

db = klass.new(":memory:", extensions: [])
assert(db.marshal_extensions_called)
assert_empty(db.marshal_extensions_arg)

db = klass.new(":memory:", extensions: ["path/to/ext1", "path/to/ext2", "ClassName"])
assert(db.marshal_extensions_called)
assert_equal(["path/to/ext1", "path/to/ext2", "ClassName"], db.marshal_extensions_arg)
end

def test_raw_float_infinity
Expand Down

0 comments on commit c7d625a

Please sign in to comment.