CXX-3236 add mongocxx v1 declarations

[eramongodb]

Summary

Resolves CXX-3236. Followup to #1462 and companion PR to #1469 and #1470. Contains cherry-pick of proposed changes in #1469 (the "cpx: cxx-abi-v1-instance" commit).

This is 5 out of an estimated 7 major PRs which in total are expected to resolve CXX-2745 and CXX-3320.

• Top-Level Dependencies (CXX-3239)
• bsoncxx v1 Interfaces (CXX-3232)
• bsoncxx v1 Implementations (CXX-3233)
• bsoncxx v_noabi Refactor (CXX-3234)
• mongocxx v1 Interfaces (CXX-3236) (<-- you are here)
• mongocxx v1 Implementations (CXX-3237)
• mongocxx v_noabi Refactor (CXX-3238)

Related tickets affected by the changes in this PR (applicable only to the v1 API) include (not exhaustive, but tried my best):

• CXX-849: "Audit API usage of int32_t"
• CXX-1046: "Hide members of options classes behind private implementation"
• CXX-1203: "Audit methods for noexcept and constexpr" (no constexpr)
• CXX-1229: "Fix incorrect constness in several places"
• CXX-1245 "Guarantee thread safety for const methods in mongocxx API"
• CXX-1524: "Audit CXX usage of libmongoc APIs"
• CXX-1546: "Return document::view instead of document::view_or_value from options getters"
• CXX-1617: "Permit options::client::ssl_opts without "ssl=true" in URI"
• CXX-1801: "stop deleting the rvalue overload of pool::entry::operator->()"
• CXX-1826: "Reduce namespace nesting"
• CXX-1836: "Nest options inside of their associated classes"
• CXX-1827: "Reduce friction between view and value types"
• CXX-1828: "Add constraints for templated functions"
• CXX-1844: "Error hierarchy uncorrelated with error domains"
• CXX-2038: "Change URI option getters to return optional views where possible"
• CXX-2266: "Use bsoncxx::string::view_or_value in gridfs::bucket"
• CXX-2270: "Audit string arguments"
• CXX-2367: "Review #include hygiene in header files"
• CXX-2376: "Revisit C++ client exceptions"
• CXX-3257: "Remove deprecated mongocxx::v_noabi::options::index::storage_options"

Relative Changelog (v_noabi -> v1)

• Added
  • is_open() to v1::gridfs::downloader and v1::gridfs::uploader to extend behavior of operator bool().
  • flush() to v1::gridfs::uploader (exposing v_noabi::gridfs::uploader::flush_chunks()) + documenting the existence of underlying buffers for better control over its behavior.
  • chunks_written() to v1::gridfs::uploader (useful information).
  • Add .upserted_ids() for v1::update_many_result for consistency with v1::bulk_write::result (vs. v1::update_one_result which uses the old .upserted_id()).
  • Add k_unknown to v1::write_concern for symmetry with v1::read_concern.
  • Add copyability to v1::indexes, v1::pipeline, v1::bulk_write::single, etc.
  • v1::bulk_write::result::raw() to allow access to the raw server response.
  • v1::bulk_write::insert_one::value is made public.
  • Allow rvalue invocation of v1::pool::entry::operator->, v1::client::database(), etc. (by removing deleted overloads and ref-qualifiers; premature pessimization).
• Changed
  • v_noabi::*::view_or_value is replaced with either view or value (std::string for v_noabi::string::view_or_value).
    • For parameters:
      • If the value is (eventually) owned by mongocxx, value.
      • If the value is (eventually) passed to or owned by mongoc, view.
      • Some string parameters known to require null-terminated strings are defined in terms of a char const* to allow avoiding forced allocations. (We could really use a cstring_view...)
    • For return values:
      • Always view, with a few exceptions for "simple" classes wrapping the underlying value with no additional invariants (e.g. v1::bulk_write::insert_one).
      • Always replace Optional<view_or_value> -> Optional<View> (view_or_value implies internal ownership).
  • Do not return T const& or Optional<T> const&: return T or Optional<T> instead.
    • Avoid returning references to internal representation (e.g. Optional<T> const&) which severely restrict implementation freedom.
    • The only functions which return a reference are special member functions, error category functions, and setters (method chaining), plus the following exceptions:
      • v1::client_session::client(): return the associated client (must be a mongocxx reference).
      • v1::bulk_write::single::get_*(): straightforward union-like API.
  • Remove const qualifier from member functions and parameters which are not logically const.
    • v1::collection::list_indexes() uses mongoc_collection_t*.
    • v1::database parameters in v1::client_encryption use mongoc_database_t*.
    • v1::client database operations use mongoc_client_t*.
    • Several stray top-level const in return types are removed.
  • Consistently return T& for setters (method-chaining) where various functions used to return void.
  • v1::server_error
    • Invariant: a raw server error document is always available (not optional).
    • When a server error code is not accompanied by a raw server document (rare), throw as mongocxx::v1::exception with .code() == v1::source_errc::server.
    • Fix documentation for server vs. client error codes (.code() is always the server error code, .client_code() is the optional client error code, for consistency with the .raw() invariant).
  • v1::event::*
    • Use Ro0 instead of Ro5 for "view-only" trivially-copyable classes.
      • All void* are labeled with a comment indicating the underlying type.
      • v1::events::server_description is the sole exception due to v1::events::topology_description::servers(), see "Removed" entry below.
      • The "impl" constructors are made private and included in this PR to indicate the default constructor(s) are deliberately disabled.
    • server_heartbeat_failed::message() return type: string -> string_view.
  • v1::read_concern: ignore unsupported values instead of throwing an exception.
  • v1::read_preference: treat invalid read modes as k_primary instead of an error (same behavior as mongoc_read_prefs_get_mode()).
  • v1::write_concern
    • Treat .tag(nullptr) as k_default instead of an error.
    • Avoid .acknowledge_level() errors by converting unsupported values to k_unknown and document "not k_unknown" as a precondition for .to_document() (treat as if unset).
  • Ensure and document v1::gridfs::uploader flushes its buffer on destruction (it didn't already do this?!).
  • Unconditionally not-null T* parameters are made T& instead for input/output streams to v1::gridfs::bucket.
  • Return T* instead of Optional<T*> for .key_vault_*() in v1::auto_encryption and v1::client_encryption.
  • Change v1::search_indexes::create_index() return type to be consistent with v1::indexes::create_one().
  • Replace the deprecated .comment() with .comment_option() in v1::find_options for consistency with other option classes.
  • Basic template parameter constraints for v1::collection's bulk write API.
  • Int32 -> Int64 for bulk write API result values and some v1::pipeline fields.
  • id_map for v1::bulk_write::result and related API:
    • Change key type from Size -> Int64 for keys of id_map (e.g. v1::bulk_write::result) for better spec accuracy.
    • Change value type from document::element to types::view to avoid unnecessary proxy type (element key is always "_id"; useless info) + consistency with other "result" API.
  • Make v1::bulk_write::* operation single-argument constructors explicit.
  • Rename v1::client_session::options() -> .opts() to avoid conflict with ::options.
  • Change with_transaction_cb parameter type from T* (unconditional not-null) to T&.
• Removed
  • v_noabi::events::server_description::is_master() (deprecated by CXX-2262, CXX-2263 #841).
  • v_noabi::events::topology_description::server_descriptions: move ownership of underlying mongoc_server_description_t objects to v1::events::server_description and avoid the awkward intermediate class. The array itself (not its elements) is deallocated after the std::vector<v1::events::server_description> is fully constructed.
  • v_noabi::gridfs::chunks_and_bytes_offset: implementation detail not used by any public API.
  • v_noabi::result::gridfs::upload ctor is made internal for v1::gridfs::upload_result.
  • ordered() for v1::insert_one_options (specific to v1::insert_many_options).
  • Removed deprecated v_noabi::options::index::base_storage_options and related API.
  • Removed deprecated v_noabi::read_preference::hedge().
  • Deprecated concern+preference accessors for v1::client.
  • Deprecated .ssl() API.
  • Remove deprecated v_noabi::options::index::haystack*() API.
  • Remove unused v_noabi::options::index::version(). (Was this meant to be "textIndexVersion"?)
• Deferred/Rejected/Todo
  • API consistency across various "convertible to a BSON document" classes.
    • Required changes are far beyond the scope of this PR.
    • Conflicting design philosophies and goals over the years, e.g. CXX-763 vs. CXX-1359 vs. CXX-1833 (non-exhaustive list).
    • Inconsistent presence vs. absence of equality comparison operators is left unchanged.
  • Specification conformance for exception class types that should derive from v1::server_error (e.g. WriteConcernError, BulkWriteError, etc.).
    • Prior exception class types were not conforming either (did not provide necessary fields with error-specific information).
    • v1::server_error::raw() should continue to suffice as a workaround for obtaining error-specific fields manually.
  • Improving constraint validation of APM event handlers (e.g. noexcept) via template parameter constraints (e.g. like bsoncxx::v1::document::value's deleter API).
  • v1::events::*: .duration() and .round_trip_time(): changing the return type from std::int64_t to std::chrono::microseconds.
  • v1::events::server_heartbeat_failed: support returning or throwing a mongocxx::v1::exception for the bson_error_t?
  • v1::events::server_description does not conform to specification for ServerDescription.
  • v1::tls underspecification and lack of support for all mongoc_ssl_opt_t fields.
  • Many issues with v1::collection: missing access to raw server response (e.g. .rename()), missing support for additional command options (e.g. .rename()), missing session overloads (e.g. .estimated_document_count()), awkward range templates, use of bulk write operations instead of dedicated database commands (e.g. .insert_one() using bulk write "insertMany" instead of the insert database command?).
  • Many issues with indexes vs. search_indexes API symmetry and specification compliance (exacerbated by the model vs. command-specific options vs. common index options situation; v1::search_indexes::model should probably be split into SearchIndexModel and SearchIndexOptions).
  • Missing support for various fields, options, and commands.
  • Validation and error handling that should be handled by mongoc, but is not yet being done (e.g. the collation && !is_acknowledged condition for .find_and_modify() and etc.).
  • Fields which may be specified in multiple ways and override one another (underspecified).

General Notes

• Minimize noisy, redundant, and superfluous documentation as much as possible.
  • Avoid silly \param or \returns when behavior is plainly obvious from the declaration (see: Slack). Only include when at least one parameter or the return value requires documentation of special values and behaviors that is not apparent from the types alone. Prefer \par Preconditions for partial parameter documentation.
    • e.g. Optional<T> return type for a "field" is "obviously" empty when "unset".
    • e.g. an argument of 0 being equivalent to "unset" for an Int field is not "obvious", thus it is documented.
    • e.g. v1::client::start_session documents: "The client session object MUST be passed as the first argument to all operations that are intended to be executed in the context of a session." -> avoid needing countless repetition of \param session paragraphs. The overload complexity should be revisited and addressed at a later time... (CXX-1835)
  • Use "Equivalent to ..." whenever able to avoid repetitive documention when \copydoc is not applicable.
    • Note: () is omitted given \ref func_name when more than one overload may apply.
    • Use \{ ... \} Doxygen groups when able (helped by absence of \param).
    • Take advantage of Markdown code blocks: "a code block is worth a thousand words".
  • Avoid documenting error codes coming from external libraries (e.g. mongoc) in detail.
    • e.g. "\throws mongocxx::v1::exception for all other runtime errors".
    • Most users only use ex.what(); encourage programmatic inspection via v1::source_errc and v1::type_errc instead.
    • Errors specific to mongocxx library behavior are defined using component-specific errc (as in bsoncxx), of which there are currently only 9 in total (all other errors are deferred to mongoc):
      • instance (multiple object detection: CXX-3320 migrate instance and logger to mongocxx::v1 #1469)
      • server_api (translate false returned by mongoc API for invalid versions into an exception)
      • uri (server_selection_try_once() setter throws instead of returning a boolean)
      • pool (wait queue timeout for .acquire())
      • collection (narrowing conversion from std::chrono::duration Int64 into a UInt32 for mongoc API compatibility)
      • indexes (index name "*" is not permitted for .drop_one(), no relevant mongoc API to defer to)
      • v1::gridfs::bucket (GridFS file metadata validation + database commands)
      • v1::gridfs::downloader (runtime GridFS file metadata validation and underlying stream state)
      • v1::gridfs::uploader (runtime GridFS file metadata validation and underlying stream state)
  • Define and document "supported fields" instead of "data members" with explicit reference to BSON document field names when applicable for reference and searchability.
    • Ensure parameter names and/or documented "supported fields" match external documentation and specification.

• Defer validation and error handling to mongoc whenever possible.
  • Reduce mongocxx specific behaviors and redundant (re)implementation of behaviors handled by mongoc.
  • Avoid the need for many exception cases and isolate "may throw an exception" points to the actual point-of-use (i.e. executing a database command).

• Aggressively use forward headers to minimize transitive dependencies as much as possible. (CXX-2367)
  • e.g. v1/database.hpp no longer includes v1/collection.hpp and all of its transitive dependencies. 🎉
  • Note: void fn(T) and T fn() do not require T to be a complete type (even when T = Optional<U>!).
  • Unable when default arguments are used. 😢 An effort is made to avoid default arguments, but not feasible at scale for some API (i.e. collection member functions)...
  • Somewhat forces Include What You Use (IWYU) in downstream code by providing the most minimal declarations necessary per component.
• Clarify and fixup API for iterator types (similar to bsoncxx::v1::document::view).
  • Make end iterators equivalent to default-constructed iterator + iterator end() const even when iterator begin() (not-const).
  • Avoid confusing "exhausted" terminology for cursor iterators (not to be confused with "exhaust cursors").
• Clarify and document discrepancies between specification and mongoc-specific behavior (e.g. bulk write result documents).
• Identify and avoid unnecessary pointer semantics by changing T* -> T& when unconditionally not-null, or by changing Optional<T*> -> T* (double-null).

Review Focus Items

• All classes and member functions should have a reference to documentation or specification describing the behavior and/or purpose of the given class or function.
  • This includes supported "fields" as parameters or data members (e.g. filter).
  • Documentation for member functions defer to the parent class (e.g. client_encryption) or the parameter (class) type (e.g. write_concern).

• GridFS download/upload buffer behavior was previously undocumented/underspecified. Check that the new documentation is accurate.

• Check that documented preconditions and postconditions are correct and consistent.

Questions for Review

• Should std::unique_ptr<impl> _impl be made void* _impl instead (even when a class impl is used) to leave open the possibility of replacing impl with a mongoc equivalent or vise versa as an ABI compatible change? Or is this "premature optimization" and using std::unique_ptr<T> is sufficient? Applied: now using void* _impl; for all mongocxx classes.
• Regardless of the question above, should classes which can currently be implemented entirely in terms of a mongoc_*_t type use void* _impl; // mongoc_*_t, e.g. to avoid double-indirection? (Initial changes propose that only the view-like v1::events::* classes do this.) Applied: annotated with underlying type when applicable.
• Is providing char const* + std::string + string_view set of overloads premature optimization? Should they be simplified to just a single string_view overload with an internal std::string allocation? Removed extra overloads: simple string_view only.
• Is removing the deprecated v1::read_preference::hedge() premature? (CXX-3241) Reverted: removal is premature.
• Is removing the deprecated readConcern + writeConcern + readPreference accessors in v1::client premature? (CXX-1939) Applied: removal is acceptable.
• Is removing the deprecated geoHaystack API for v1::indexes premature? (CXX-1978) Applied: removal is acceptable.
• Should mongoc_cursor_clone() be implemented by mongocxx by making cursors copyable? Or should mongoc_cursor_clone() be deprecated given its confusing behavior (re-execute the underlying query)? Applied: may add later upon request.
• Should v1::change_stream::batch_size() support std::uint32_t for consistency with mongoc_cursor_set_batch_size()? Or does mongoc_cursor_set_batch_size() need to be updated to support std::int64_t instead? Neither: spec expects Int32.

[kevinAlbs]

Posting initial comments.

• Should std::unique_ptr<impl> _impl be made void* _impl instead (even when a class impl is used) to leave open the possibility of replacing impl with a mongoc equivalent or vise versa as an ABI compatible change? Or is this "premature optimization" and using std::unique_ptr<T> is sufficient?

I am slightly in favor of switching to void *, but also OK with the current proposal. I expect saving one pointer dereference may not make an observable performance difference in most cases (especially when much of mongocxx API includes network calls), but this may help to future-proof the ABI.

• Regardless of the question above, should classes which can currently be implemented entirely in terms of a mongoc_*_t type use void* _impl; // mongoc_*_t, e.g. to avoid double-indirection? (Initial changes propose that only the view-like v1::events::* classes do this.)

Similar: Slightly in favor, but OK with current proposal.

• Is providing char const* + std::string + string_view set of overloads premature optimization? Should they be simplified to just a single string_view overload with an internal std::string allocation?

IIUC this only appears to impact four methods: read_concern::acknowledge_string(), server_error::has_error_label(), uri::uri(), and write_concern::tag.

Some libbson API was added accept a length argument (e.g. bson_iter_init_find_w_len). If libmongoc adds functions accepting a string length (suggested in comment), that may avoid a need for the char const* / std::string overloads. Filed CDRIVER-6128 to propose this API in the C driver.

IMO: I am slightly in favor of reducing the overloads since this could be eventually improved by libmongoc additions.

• Is removing the deprecated v1::read_preference::hedge() premature? (CXX-3241)

I am in favor of adding to v1. Hedged reads was only deprecated in server 8.0. And only deprecated in the C++ driver v4.1.

• Is removing the deprecated readConcern + writeConcern + readPreference accessors in v1::client premature? (CXX-1939)
• Is removing the deprecated geoHaystack API for v1::indexes premature? (CXX-1978)

I am in favor of keepint these out. CXX-1939 deprecated 5 years ago, and CXX-1978 deprecated 8 years ago, I expect those have low usage. If that assumption is wrong, they can be added to v1 on request.

• Should mongoc_cursor_clone() be implemented by mongocxx by making cursors copyable? Or should mongoc_cursor_clone() be deprecated given its confusing behavior (re-execute the underlying query)?

I also think mongoc_cursor_clone behavior is surprising (I would expect "clone" means to clone the current state). mongoc_cursor_clone documents the behavior. And this appears to be the requested behavior in CDRIVER-204 for a language binding. I would rather not continue the pattern in C++ driver unless there is a known need. But if there are users wanting the C API, I think it is fine as-is.

• Should v1::change_stream::batch_size() support std::uint32_t for consistency with mongoc_cursor_set_batch_size()? Or does mongoc_cursor_set_batch_size() need to be updated to support std::int64_t instead?

The spec suggests both should be std::int32_t. I expect mongoc_cursor_set_batch_size should instead be int32_t to match the spec.

[eramongodb]

I am slightly in favor of reducing the [NTBS] overloads since this could be eventually improved by libmongoc additions.

Done. This means we are accepting a performance penalty (due to unconditional std::string allocations) until mongoc improves support for length-based string API.

I am in favor of adding [.hedge()] to v1. Hedged reads was only deprecated in server 8.0.

Done. v1::read_preference also has deprecated .hedge() accessors.

I would rather not continue the [mongoc_cursor_clone()] pattern in C++ driver unless there is a known need.

Will leave v1::cursor as-is then. Both v1::cursor and v_noabi::cursor will remain non-copyable.

I am slightly in favor of switching to void * ... this may help to future-proof the ABI.

Done. All _impl data members are now annotated with the underlying type (class impl; is assumed and implied by the presence of the declaration).

The following classes are defined entirely in terms of the mongoc API:

• v1::events::* (all event types are fully backed by mongoc equivalents)
• v1::client_encryption -> mongoc_client_encryption_t
• v1::client_session::options -> mongoc_session_opt_t
• v1::read_concern -> mongoc_read_concern_t
• v1::read_preference -> mongoc_read_prefs_t
• v1::transaction -> mongoc_transaction_opt_t
• v1::uri -> mongoc_uri_t
• v1::write_concern -> mongoc_write_concern_t

The following classes are defined entirely in terms of other mongocxx API:

• v1::change_stream::iterator -> v1::change_stream (shared state iterators)
• v1::cursor::iterator -> v1::cursor (shared state iterators)

The following classes still use class impl; despite an apparent mongoc equivalent being available:

• v1::apm (mongoc_apm_callbacks_t)
  • Needs to support std::function<T> getters.
• v1::auto_encryption (mongoc_auto_encryption_opts_t)
  • Needs to support .key_vault_client() -> v1::client* and .key_vault_pool() -> v1::pool*.
  • Missing mongoc_auto_encryption_opts_get_*() for getters.
• v1::change_stream (mongoc_change_stream_t)
  • Needs to support shared state for iterators (i.e. the event document).
• v1::client_encryption::options (mongoc_client_encryption_opts_t)
  • Needs to support mongocxx getters (no mongoc_client_encryption_opts_get_*()).
• v1::client_session (mongoc_client_session_t)
  • Needs to support .client() -> v1::client const& and .options() -> v1::client_session::options const&.
• v1::client (mongoc_client_t)
  • Needs to extend the lifetime of v1::apm.
• v1::collection (mongoc_collection_t)
  • Needs to store a reference to the associated v1::client object (possibly avoidable if we use mongoc_collection_create_indexes_with_opts() in v1::indexes::create_many()?).
• v1::cursor (mongoc_cursor_t)
  • Needs to support shared state for iterators (i.e. the result document).
• v1::database (mongoc_database_t)
  • Needs to store a reference to the associated v1::client object.
  • Missing mongoc_database_command_simple_with_server_id().
• v1::encrypt (mongoc_client_encryption_encrypt_opts_t)
  • Missing mongoc_client_encryption_encrypt_opts_get_*() for getters.
• v1::indexes::model (mongoc_index_model_t)
  • Missing mongoc_index_model_get_*() for getters.
• v1::range (mongoc_client_encryption_encrypt_range_opts_t)
  • Missing mongoc_client_encryption_encrypt_range_opts_get_*() for getters.
• v1::pool (mongoc_client_pool_t)
  • Needs to extend the lifetime of v1::apm.

[eramongodb]

Applied the decision made in #1487 to v1::server_error (Ro5 is still needed due to the _impl data member).