Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#1392 : Add support for SQLite VFS and open mode flags #1393

Draft
wants to merge 4 commits into
base: dev
Choose a base branch
from

Conversation

ancientjpeg
Copy link

@ancientjpeg ancientjpeg commented Feb 1, 2025

Addresses #1392

Changelog

  • Add vfs_mode enum class to simplify the sqlite_orm API for using different VFS systems, as compared to the raw string passed in sqlite3.h
  • Add open_mode enum class to support the open flags passed by sqlite3_open_v2. Only two values have been added so far, but this addition makes it very easy to add more in the future.
  • Add platform definitions to detect operating system
    • Necessary for exposing platform-specific VFS types
  • Add tests

More Info

https://www.sqlite.org/vfs.html - VFS options and description
https://www.sqlite.org/c3ref/open.html - open flag options

@fnc12
Copy link
Owner

fnc12 commented Feb 1, 2025

hey @ancientjpeg . Thank for this PR. Please work in dev branch not master

@ancientjpeg ancientjpeg force-pushed the feature/1392-support-sqlite-vfs branch from 13c0562 to 42d717b Compare February 1, 2025 08:57
@ancientjpeg ancientjpeg changed the base branch from master to dev February 1, 2025 08:57
@ancientjpeg
Copy link
Author

Just rebased. Thanks and apologies again.

@ancientjpeg ancientjpeg changed the title #1392 : Add support for SQLite VFS #1392 : Add support for SQLite VFS and open mode flags Feb 3, 2025
Copy link
Author

@ancientjpeg ancientjpeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fnc12 I've added enums to allow for full control over what open mode/VFS options users actually have access to. I'm happy with the state this PR is in, but I have some preliminary evidence that using some VFS modes like unix-dotfile may interfere with the ability to set certain journal_modes. This is detectable by just checking journal_mode after you've set it, but if you'd like me to brainstorm a layer of protection for that I'd be happy to.

@ancientjpeg ancientjpeg marked this pull request as ready for review February 4, 2025 19:56
Copy link
Collaborator

@trueqbit trueqbit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ancientjpeg for your contribution! Here come my 2 cents...

@ancientjpeg ancientjpeg marked this pull request as draft February 5, 2025 17:32
@ancientjpeg
Copy link
Author

ancientjpeg commented Feb 5, 2025

@trueqbit Still working through your changes, but something occurred to me while refactoring. I had some of those unnecessary sqlite_orm:: qualifiers on the mode types because I actually got a couple compile errors complaining about ambiguous symbols. Would it be alright if I appended _t to these flag types? I.e.,

enum class vfs_mode_t;

Then it'd be a lot clearer to re-use the same verbiage for defining variables, e.g.

vfs_mode_t vfs_mode;
open_mode_t open_mode;

struct storage_options {
  vfs_mode_t vfs_mode;
  open_mode_t open_mode;
};

Let me know what you think of that.

Edit: I ended up implementing this because GCC complains of ambiguity in cases such as const open_mode open_mode. I prefer the look of const open_mode_t open_mode to const open_mode open_flags as I feel the latter might sow confusion, but I'm happy to change it if it feels more in line with the rest of the lib.

@ancientjpeg ancientjpeg force-pushed the feature/1392-support-sqlite-vfs branch from b96a21f to 319c614 Compare February 5, 2025 21:29
@trueqbit
Copy link
Collaborator

trueqbit commented Feb 5, 2025

@trueqbit Still working through your changes, but something occurred to me while refactoring. I had some of those unnecessary sqlite_orm:: qualifiers on the mode types because I actually got a couple compile errors complaining about ambiguous symbols. Would it be alright if I appended _t to these flag types? I.e.,

enum class vfs_mode_t;

Then it'd be a lot clearer to re-use the same verbiage for defining variables, e.g.

vfs_mode_t vfs_mode;
open_mode_t open_mode;

struct storage_options {
  vfs_mode_t vfs_mode;
  open_mode_t open_mode;
};

Let me know what you think of that.

Hmm, that's a funny problem as both are public :)
However, I do not recommend changing the enum type's name. Firstly, it is not recommended in the coding guidelines, and we do not use it in sqlite_orm with one exception.

I personally like camel case for (member) variables, but I agree that's debatable.
A good choice in any case for the variable name is vfsOption, modeOption or vfs_option, mode_option as "option" conveys that fact that a specific thing is selected .

While we are at it, I also suggest renaming vfs_mode -> vfs_object, as SQLite calls the feature Virtual File System Objects.

@ancientjpeg
Copy link
Author

ancientjpeg commented Feb 6, 2025

@trueqbit Great point about using option in the variable name, I think I'll go with that. I'll also change the members and parameters to camelCase if that's the preferred styling for this library—I had just assumed snake_case was the go-to given the external API is style that way, but now that I look closer I see a lot of parameter names are camelCase.

I'll be away from my development system until next week, at which time I'll wrap up these changes. Thanks for the feedback!

Comment on lines 645 to 649
*/
sqlite_orm::vfs_object vfs_object() const {
return this->connection->options.vfs_option;
}
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Namespacing here seems to be required to prevent errors in GCC. Mirrors existing code:

sqlite_orm::journal_mode journal_mode()

@ancientjpeg ancientjpeg marked this pull request as ready for review February 14, 2025 17:48
@ancientjpeg ancientjpeg requested a review from trueqbit February 14, 2025 20:17
@ancientjpeg ancientjpeg marked this pull request as draft February 19, 2025 20:54
@ancientjpeg
Copy link
Author

Re-drafting to address merge conflicts.

@trueqbit
Copy link
Collaborator

Re-drafting to address merge conflicts.

Hi @ancientjpeg,

I must apologize for the inconvenience. Independently of you, I have developed a way to pass various storage options to make_storage(), in particular connection control options. The important change for you is that there is now a connection_control aggregate that you extend with your new options.

Also, I would like to point out to you in advance that an enum to represent a VFS is most likely too restrictive as it is not a fixed set of values. But we can discuss that later.

@ancientjpeg ancientjpeg force-pushed the feature/1392-support-sqlite-vfs branch from da943af to 0f34242 Compare February 21, 2025 00:00
add test skeleton

add open flags

rename file

move test file, finish test code, add to cmake

fixup clang error

add another make_storage overload

remove overload

fixup

rename to v2, fixup tests

fixup copy ctor

remove int flags option

add platform definitions

add vfs file

rename tests

update tests and gen code

rename to vfs_t

implement vfs enum

remove default arg for connection holder

clarify test log

add vfs getter and default vfs def

rename vfs_default, add public getter

swap from different method name to overload

cleanup macros and fix linux macros

more vfs cleanup and test fixup

re-run gen

remove unnecessary pack indexing check

add open_mode.h

add to header

fixup win macro error

use enum value instead of constexpr

regenerate to include vfs

move test file

convert to constexpr, set up open_mode tests

fully set up open mode in ctors

rename vfs_t to vfs_mode

begin renaming open_mode

rename open_mode

enum scoping fixups

add readonly; need to attempt fix

fix readonly call

remove usage of tmpnam

rename mac macro to apple

rename file

remove pointless line

update gen

run tests for mem as well

reword test variable

fixup test on Windows

fix gcc err

run gen

fixup

add return path to silence MSVC warn

fixup pragma once placements

remove inline

no nested namespaces

cleanup serializers

add include

change to serialize type

remove unneeded namespacing

just call it all open_mode

implement storage_options

remove auto

inject options into connection

add _t suffix

large batch of name fixes

separate static tests

remove unused comments

remove unused include

revert _t suffix

rename member vars

rename to vfs_object

fix weird typo

more naming fixups

fix namespacing

attempt to fix MSVC crash

real string fix
@ancientjpeg ancientjpeg force-pushed the feature/1392-support-sqlite-vfs branch from 0f34242 to 2cf9c47 Compare February 21, 2025 00:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants