diff --git a/docs/user-guide/databases.rst b/docs/user-guide/databases.rst index a3a4419d..9065eacb 100644 --- a/docs/user-guide/databases.rst +++ b/docs/user-guide/databases.rst @@ -5,7 +5,7 @@ SQL Databases Felis can create the corresponding database objects from a schema using the command line tool or the Python API. This includes the schema itself and all of its tables, columns, indexes, and constraints. The DDL to perform these actions can either be executed automatically or written to a file for later use. -An existing database can be used or a new one can be created, depending on the options provided. +An existing database can be used or a new one created, depending on the options provided. When creating a new database rather than updating an existing one, the schema will be instantiated using ``CREATE DATABASE`` in MySQL and ``CREATE SCHEMA`` in PostgreSQL. The user must have the necessary permissions to create databases in the target database server for this to @@ -43,7 +43,7 @@ A specific database dialect such as MySQL can be selected in dry run mode using felis create --engine-url mysql:// schema.yaml This will generate the SQL in MySQL format and print it to the console but will not actually create or update -the database. +a database. The dry run mode may also be explicitly enabled using the ``--dry-run`` option: @@ -69,15 +69,15 @@ In order to create a persistent database, the ``--engine-url`` must be set to a The URL format follows `SQLAlchemy engine conventions `_: ``dialect+driver://username:password@host:port/database``. -Each of the parameters is explained below: +The database URL has the following parameters: - ``dialect``: The name of the database backend, such as ``sqlite``, ``mysql``, or ``postgresql``. The default is ``sqlite``. - ``driver``: The name of the DBAPI to use, such as ``mysqlconnector``, or ``psycopg2``. This is optional and the default driver for the dialect will be used if not specified. - ``username``: The username to use when connecting to the database. - ``password``: The password to use when connecting to the database. -- ``host``: The host to connect to. Typically, this should be set to ``localhost`` if the database is running on the same machine. -- ``port``: The port to connect to. This will use the default port for the dialect if not specified. -- ``database``: The name of the database to create. For MySQL, this should be left blank. For PostgreSQL, this should be the name of the database in which the schema will be created. +- ``host``: The host for connection. Typically, this should be set to ``localhost`` if the database is running on the same machine. +- ``port``: The port for connection. This will use the default port for the dialect if not specified. +- ``database``: The name of the database to create. For MySQL, this should typically be left blank, and Felis will use the name of the schema for the database. For a PostgreSQL connection, this should be the name of the database in which the schema will be created. The database URL may also be set using the ``FELIS_ENGINE_URL`` environment variable, in which case the ``--engine-url`` option can be omitted. @@ -85,7 +85,7 @@ The database URL may also be set using the ``FELIS_ENGINE_URL`` environment vari MySQL ^^^^^ -To create a MySQL database from a schema file, the command would look like this: +To create a MySQL database from a schema file, the command would look similar to the following: .. code-block:: bash @@ -104,8 +104,8 @@ dialect: felis create --engine-url postgresql+psycopg2://username:password@localhost/database schema.yaml Felis can be used to create the schema, but it cannot create the database itself, which must be included as -part of the URL. -This must be done beforehand using the ``CREATE DATABASE`` command in the PostgreSQL client. +part of the URL; or the command will fail. +The database must be created beforehand using the ``CREATE DATABASE`` command in the PostgreSQL client. SQLite ^^^^^^ @@ -116,49 +116,53 @@ To persist a SQLite database, first create an empty database on disk as follows: sqlite3 /tmp/my.db "VACUUM;" -The database can then be created from a schema file: +Installation of SQLite is not covered; please refer to the `SQLite documentation `_ for more information. + +The database objects can then be instantiated from a schema file: .. code-block:: bash felis create --engine-url sqlite:////tmp/my.db schema.yaml -After it has been created, uou may open the database file with a SQLite client to inspect the schema as -follows: +After it has been created, you may open the database file with a SQLite client to inspect the schema: .. code-block:: bash sqlite3 /tmp/my.db -To show the instantiated tables from the SQLite client, use the following command: +To show the tables which were instantiated, use the following command from within the SQLite client: .. .tables -SQLite will ignore the name of the schema, as it does not support named schemas or databases internally. +SQLite will ignore the name of the schema, as it does not support named schemas or databases. Creating a New Database ----------------------- Felis can also be used to create the database itself, rather than use an existing one, by using the -``--create-if-not-exists`` option: +``--create-if-not-exists`` option. + +Here is an example of creating a new MySQL database: .. code-block:: bash felis create --engine-url mysql+mysqlconnector://username:password@localhost --create-if-not-exists schema.yaml -Felis can also drop an existing database first and recreate it: +Felis can also drop an existing database first and then recreate it: .. code-block:: bash felis create --engine-url mysql+mysqlconnector://username:password@localhost --drop-if-exists schema.yaml -The commands to create or drop databases will require that the database user has the necessary permissions. +The commands to create or drop databases will require that the database user has the necessary permissions on +the server. Using a Different Schema Name ----------------------------- -Typically, the name of the schema in the database will be the same as the ``name`` field in the YAML file, but +The name of the schema in the database will by default be the same as the ``name`` field in the YAML file, but this can be overridden using the ``--schema-name`` option: .. code-block:: bash @@ -174,8 +178,8 @@ The Python API can also be used to create a database from a schema. First, the schema object should be loaded from a YAML file, following the instructions in :ref:`validating-with-python-api`. -Once the schema object has been created after being successfuly validated, the builder can be used to create -the SQLAlchemy metadata object: +Once the schema object has been successfuly created, the builder can be used to create the SQLAlchemy metadata +object: .. code-block:: python @@ -183,7 +187,7 @@ the SQLAlchemy metadata object: metadata = MetaDataBuilder(schema).build() The metadata object can be used to create the database using standard SQLAlchemy commands. -For example, to create an in-memory SQLite database: +For example, the following command will create an in-memory SQLite database in Python: .. code-block:: python @@ -191,8 +195,6 @@ For example, to create an in-memory SQLite database: engine = create_engine("sqlite:///:memory:") metadata.create_all(engine) -The above command will just create an in-memory SQLite database and will not persist the database to disk. - To create a MySQL database, the engine URL should be changed to something like this: .. code-block:: python @@ -203,7 +205,7 @@ To create a MySQL database, the engine URL should be changed to something like t The database will then be created on the MySQL server at ``localhost``. Felis also provides the `DatabaseContext class <../dev/internals/felis.db.utils.DatabaseContext.html>`_ which -can be used to create a database from a schema. +supports creation of the database or schema itself: .. code-block:: python @@ -213,5 +215,5 @@ can be used to create a database from a schema. ctx.create_all() An advantage of using this class is that it can automatically handle the creation of the database if it does -not already exist with the ``create_if_not_exists`` method or drop and recreate the database with the -``drop_and_create`` method. +not already exist with the ``create_if_not_exists`` method; an existing database may also be dropped and +recreated using the ``drop_and_create`` method. diff --git a/docs/user-guide/datatypes.rst b/docs/user-guide/datatypes.rst index 5f7981bd..77ae7040 100644 --- a/docs/user-guide/datatypes.rst +++ b/docs/user-guide/datatypes.rst @@ -2,12 +2,12 @@ Data Types ########## -Felis has an internal type system that is used to define the data types of columns in a schema. -These are mapped to the appropriate SQL data types when the schema is created in a database and the -corresponding `VOTable `__ data type when writing information to +Felis provides an internal type system that is used to define the data types of columns in a schema. +These data types are mapped to the appropriate SQL types when instantiating a database and the corresponding +`VOTable `__ data types when writing information to `TAP_SCHEMA `_. -The following values are supported for the +The following values are supported by the `datatype <../dev/internals/felis.datamodel.Column.html#felis.datamodel.Column.datatype>`_ field of a column object: diff --git a/docs/user-guide/examples.rst b/docs/user-guide/examples.rst index 097e15dc..c745855c 100644 --- a/docs/user-guide/examples.rst +++ b/docs/user-guide/examples.rst @@ -2,8 +2,10 @@ Examples ######## -The `SDM Schemas github repository `_ contains Felis schema files used by the Rubin Observatory, which will be used as examples to illustrate the features of Felis. -The following is an excerpt from the `DP0.2 DC2 schema `_: +The `SDM Schemas github repository `_ contains Felis schema files used by +the Rubin Observatory, which will be used as examples to illustrate the features of Felis. +The following is an excerpt from the +`DP0.2 DC2 schema `_: .. code-block:: yaml :linenos: @@ -33,22 +35,27 @@ The following is an excerpt from the `DP0.2 DC2 schema `_ class provides a full list of available fields, including TAP and VOTable-specific metadata. +A table is comprised of one or more columns which must must have a name, id, and datatype, and an optional +(but highly recommended to include) description. +The `Column <../dev/internals/felis.datamodel.Column.html>`_ class provides a full list of available fields, +including TAP and VOTable-specific metadata. -The second exerted column is coord_ra, which is a measurement field including units of measurement. - -Both fields shown here have an `IVOA UCD `_ field, which is a "vocabulary for describing astrononomical data quantities," describing the semantics of the fields. +Both fields shown here have an `IVOA UCD `_ field, +which is a "vocabulary for describing astrononomical data quantities," describing the semantics of the fields. +The first column in the Object table is ``objectId``, which is a long integer field defining a unique identifier +for records in the table. +The ``meta.id`` word in the column's UCD flags the field semantically as an identifier. +The second exerted column is ``coord_ra``, which is a measurement field including units of measurement. Felis also supports table constraints, such as foreign keys. -The DP0.2 DC2 schema includes a foreign key constraint on the ccdVisitId field of the Source table, defined as follows: +The `DP0.2 DC2 schema `_ includes a foreign +key constraint on the ``ccdVisitId`` field of the ``Source`` table, defined as follows: .. code-block:: yaml :linenos: @@ -63,6 +70,7 @@ The DP0.2 DC2 schema includes a foreign key constraint on the ccdVisitId field o referencedColumns: - "#CcdVisit.ccdVisitId" -The ccdVisitId field in the Source table is linked to the ccdVisitId field in the CcdVisit table. +The ``ccdVisitId`` field in the ``Source`` table is linked to the ``ccdVisitId`` field in the ``CcdVisit`` +table. -Felis supports many additional features. Refer to the `model documentation `_ for a complete list. +Felis schemas support many additional features. Refer to the `model documentation `_ for a complete list. diff --git a/docs/user-guide/installation.rst b/docs/user-guide/installation.rst index 0eeeeb38..ed2a3e24 100644 --- a/docs/user-guide/installation.rst +++ b/docs/user-guide/installation.rst @@ -3,31 +3,32 @@ Installation ############ Felis is deployed to PyPI on a weekly release schedule. + To install the latest release, use the following command: .. code-block:: bash pip install lsst-felis -To install a specific release, use a command similar to the following: +A specific release can be installed using a command similar to the following: .. code-block:: bash pip install 'lsst-felis==27.2024.2700' -This is just an example. If you want to install a different version, replace the version number from above -with the one you want to install. +This is just an example. +If you want to install a different version, replace the version number from above. -To install Felis from the Github main branch, use the following command: +To install Felis directly from the Github ``main`` branch, use the following command: .. code-block:: bash pip install 'lsst-felis @ git+http://github.com/lsst/felis.git@main' -To depend on Felis in your project, add the following to your ``requirements.txt`` file: +Felis may also be added as a dependency of your Python project by putting the following in ``requirements.txt``: .. code-block:: text lsst-felis -Felis requires Python 3.11 or later and is tested on Linux, macOS, and Windows WSL. +Felis requires Python 3.11 or later and is regularly tested on Linux, macOS, and Windows WSL. diff --git a/docs/user-guide/intro.rst b/docs/user-guide/intro.rst index 2d8fd0db..cd9c8e3e 100644 --- a/docs/user-guide/intro.rst +++ b/docs/user-guide/intro.rst @@ -1,11 +1,10 @@ Introduction ------------ -Data catalogs are a fundamental part of modern astronomical research. -They provide a means to describe the data that is available, and to search for data that matches certain -criteria. -Tabular data models are a common way to represent such catalogs. -A canonical format for describing these types of models is SQL Data Definition Language (DDL). +Data catalogs are a fundamental part of modern astronomical research, providing a means to describe available +data and search for data matching certain criteria. +Tabular data models are a common way to represent such catalogs, and a canonical format for describing them is +SQL Data Definition Language (DDL). However, DDL does not provide a way to describe the semantics of the data, such as the meaning of each column, the units of measurement, or the relationships between tables. Felis provides a way to describe these semantics in a user-friendly YAML format. @@ -13,13 +12,14 @@ Felis provides a way to describe these semantics in a user-friendly YAML format. Within astronomy, the `International Virtual Observatory Alliance `__ (IVOA) has developed a standard for describing tabular data models called `Table Access Protocol `__ (TAP). -Metadata for a specific TAP service is typically provided in a schema called TAP_SCHEMA that describes the -tables and columns available in the service. +Metadata for a specific TAP service is typically provided in a schema called TAP_SCHEMA describing tables and +columns available in the service. Felis provides a tool for translating its schema representation into TAP_SCHEMA, making a catalog's metadata available through a standard TAP service. -Compatible TAP services can use this data to populate their ``/tables`` output as well. +Compatible TAP services can use this data to populate their ``/tables`` output. -Felis also provides a mechanism for instantiating an empty catalog from its schema into relational database -objects such as tables and columns. -Supported databases include SQLite, MySQL/MariaDB, and PostgreSQL. +Felis also provides a mechanism for instantiating a catalog from its schema representation into an (empty) +relational database including the table, constraint and column definitions. This can be done using the command line interface or the Python API. +Supported databases include SQLite, MySQL/MariaDB, and PostgreSQL. + diff --git a/docs/user-guide/model.rst b/docs/user-guide/model.rst index 35344ab6..8cbe9900 100644 --- a/docs/user-guide/model.rst +++ b/docs/user-guide/model.rst @@ -3,9 +3,9 @@ Data Model ########## Felis's `data model <../dev/internals.html#module-felis.datamodel>`__ is defined by a set of -`Pydantic `__ classes which describe the semantics of tabular data. +`Pydantic `__ classes defining the semantics of tabular data. In addition to the standard conceptual constructs of a relational database schema, such as tables and columns, -Felis provides a way to attach extra metadata to these elements like units of measurement on columns. +Felis provides a way to attach extra metadata to these elements, like units of measurement on columns. ****** Schema @@ -18,10 +18,12 @@ A schema is defined by a name and will be instantiated as a and a `database object in MySQL `__ with ``CREATE DATABASE``. +Schemas may contain the following fields: + :``name``: The name of this schema. This is the name that will be used to create the schema in the database. :``@id``: An identifier for this schema. This may be used for relating schemas together at a higher level. Typically, the name of the schema can be used as the id. :``description``: A textual description of this schema. -:``tables``: The list of tables in the schema. A schema MUST have one or more tables. +:``tables``: The list of tables in the schema. A schema must have one or more tables. :``version``: Optional schema version description. ************** @@ -30,16 +32,16 @@ Schema Version A `schema version <../dev/internals/felis.datamodel.Schema.html#felis.datamodel.SchemaVersion>`__ can be used to track changes to a schema over time. -The version is defined by either a simple version string or a more complex version object. +The version is defined by either a simple string or a more complex version object. Felis does not enforce any particular format for version strings. -A version string may simply be defined as ``version: v1``, for instance. +A version string could simply be defined as ``v1``, for instance. The `schema version <../dev/internals/felis.datamodel.SchemaVersion.html>`_ object has the following attributes: :``current``: The current version of the schema. :``compatible``: A list of fully compatible versions. -:``read_compatible``: A list of versions that are read-compatible. A read compatible version is one where a database created with the older version can be read by the newer version. An example of a non-read compatible change is removing a column from a table. +:``read_compatible``: A list of versions that are read-compatible. A read compatible version is one where a database created with the older version can be read by the newer version. An example of a non-read compatible change would be removing a column from a table. While Felis does not enforce any particular format for version strings, it is recommended to use a format that can be compared using `semantic versioning `__. @@ -55,9 +57,9 @@ columns along with their indexes and constraints and has these attributes: :``@id``: A unique identifier for this table. :``description``: A textual description of this table. :``columns``: The list of columns in the table. -:``primaryKey``: A list of column names that make up the primary key for this table. -:``constraints``: The list of constraints for the table. The most typical type is a foreign key constraint. -:``indexes``: The list of indexes for the table. +:``primaryKey``: The ID of the table's primary key column or a list of IDs that make up a composite primary key. +:``constraints``: The list of constraints for the table. Refer to the :ref:`Constraint` section for more information. +:``indexes``: The list of indexes for the table. Refer to the :ref:`Index` section for more information. A table may also have the following optional attributes: @@ -73,6 +75,8 @@ A `column <../dev/internals/felis.datamodel.Schema.html#felis.datamodel.Column>` in a table. A column has a name and a data type and may have additional metadata like units of measurement. +Columns have the following primary attributes: + :``name``: The name of this column. This is the name that will be used to create the column in the database. :``@id``: A unique identifier for this column. :``description``: A textual description of this column. @@ -94,13 +98,14 @@ A column may also have the following optional properties: :``votable:arraysize``: The VOTable ``arraysize`` for this column [3]_. :``votable:datatype``: The VOTable ``datatype`` for this column [3]_. :``votable:xtype``: The VOTable ``xtype``, if any, for this column [3]_. -:``mysql:datatype``: The `MySQL data type `__ for this column. -:``postgresql:datatype``: The `PostgreSQL data type `__ for this column. +:``mysql:datatype``: A `MySQL data type `__ override for this column. +:``postgresql:datatype``: A `PostgreSQL data type `__ override for this column. .. [1] The ``ivoa:unit`` and ``fits:tunit`` fields are mutually exclusive. Only one of these fields should be set on a column object. .. [2] `TAP Access Protocol (TAP) specification `__ .. [3] `VOTable specification `__ +.. _Constraint: ********** Constraint @@ -153,6 +158,8 @@ additional attributes: :``columns``: One or more column names in the current table that are part of the unique constraint. This should be one or more ``@id`` values pointing to columns in the current table. +.. _Index: + ***** Index ***** diff --git a/docs/user-guide/tap.rst b/docs/user-guide/tap.rst index 150df871..ae911f64 100644 --- a/docs/user-guide/tap.rst +++ b/docs/user-guide/tap.rst @@ -10,24 +10,24 @@ Table Access Protocol (TAP) Felis can be used to generate records for `TAP_SCHEMA `_ which describes metadata for a `TAP service `_. -The TAP_SCHEMA database can either be updated directly or SQL statements can be generated and then executed +The TAP_SCHEMA database can either be updated directly, or SQL statements can be generated and then executed manually. -The SQL can be generated, and optionally executed, by using the ``load-tap`` command. -To generate SQL statements and save them to a file, a command like the following can be used: +The SQL can be generated or executed using the ``load-tap`` command. +A command like the following can be used to generate SQL statements and save them to a file: .. code-block:: bash felis load-tap --dry-run --engine-url=mysql:// $file > tap_schema.sql -This SQL file can then be used for initialization of the TAP_SCHEMA database, e.g. within a Docker container. +This SQL file may then be used for initialization of the TAP_SCHEMA database, e.g. within a Docker container. (This procedure is not covered here.) Felis can also update an existing TAP_SCHEMA database directly if a valid URL is provided: .. code-block:: bash - felis load-tap --engine--url=mysql+mysqlconnector://user:password@host:port/TAP_SCHEMA + felis load-tap --engine--url=mysql+mysqlconnector://user:password@host/TAP_SCHEMA Felis can create an empty TAP_SCHEMA database using the ``init-tap`` command. @@ -36,13 +36,14 @@ Felis can create an empty TAP_SCHEMA database using the ``init-tap`` command. felis init-tap --engine-url=mysql+mysqlconnector://user:password@host:port By default, this will create a database called ``TAP_SCHEMA``. -If you want to use a different name, you can specify it with the ``--tap-schema-name`` option: +If you want to use a different name for the TAP_SCHEMA schema itself in the database, you can specify this +with the ``--tap-schema-name`` option: .. code-block:: bash felis init-tap --engine-url=mysql+mysqlconnector://user:password@host:port --tap-schema-name=MY_TAP_SCHEMA +Standards-conformant TAP services will generally make the TAP_SCHEMA data available at a ``/tables`` endpoint. Felis does not provide the full functionality and configuration for standing up a TAP service. For that, you will need to use a TAP service implementation like `CADC TAP `_. -Standards-conformant TAP services will generally make the TAP_SCHEMA data available at a ``/tables`` endpoint. diff --git a/docs/user-guide/validation.rst b/docs/user-guide/validation.rst index b96affb7..052a6832 100644 --- a/docs/user-guide/validation.rst +++ b/docs/user-guide/validation.rst @@ -2,28 +2,32 @@ Schema Validation ################# -Felis uses `Pydantic `__ to validate the schema and all of the objects that -it defines, including tables and columns. +Felis uses `Pydantic `__ to validate the schema and all of its defined +objects, including tables and columns. Pydantic is a data validation library that uses Python type annotations to define the structure of the data -that is being validated. +being validated. Pydantic validators are defined as occuring either "before" conversion to the model objects or "after." -When a validation error is raised in the "before" stage, the "after" validators will not run. +When a validation error is raised in the "before" stage while processing the raw YAML input data, the "after" +validators will not run. This can be important to keep in mind, as fixing these "before" error may reveal others that occur in the "after" stage. After installing Felis, you can use the ``felis validate`` command to validate one or more schema files. These should be in YAML format and conform to the `Felis schema <../dev/internals/felis.datamodel.Schema.html#felis.datamodel.Schema>`__. -The `command line documentation `_ provides more information on how to use this command. +The `command line documentation `_ provides more information on how to use the +`validate `_ command. + +.. _ValidationErrors: Validation Errors ================= If there are any errors in the schema, the validation process will raise an exception called a `validation error `_ containing a list of all the -errors that were found. These errors will be printed to the console, and the command line process will -return a non-zero exit code. +errors that were found. +These errors will be printed to the console, and the command line process will return a non-zero exit code. An error message will be printed to the log indicating how many errors were found. For example: @@ -31,8 +35,8 @@ For example: ERROR:felis:1 validation error for Schema -In the case where a field was included which is not present in the data model, such as a misspelled field -name, the error message will indicate the field name that was not recognized. +When a field was included in the data which is not valid for the data model, as in the case of a misspelled +field name, the error message will indicate the field name that was not recognized. For example: :: @@ -41,12 +45,16 @@ For example: Extra inputs are not permitted [type=extra_forbidden, input_value=12345, input_type=int] For further information visit https://errors.pydantic.dev/2.8/v/extra_forbidden -This error message indicates that the field ``not_a_field`` was found in the schema but that this is not a -valid field according to the data model. -The error message also includes the value of the field and the type of the value. -The error occurred in the first table and the first column of the schema, indexed from zero. +The first line points to the location of the error in the data, which here is the first table and the first +column of the schema, indexed from zero. +The second line states that the field ``not_a_field`` was found in the schema but that this is not a +valid field according to the data model, indicated using the ``extra_forbidden`` error type and "Extra inputs +are not permitted" message. +The error message also includes the value of the field and the type of the value, as well as a link to +relevant information in the Pydantic documentation. + -Errors may also occur when a field has a value that is not valid. +Errors may also occur when a field is assigned a value which is not considered valid according to the model. For example: :: @@ -56,10 +64,11 @@ For example: For further information visit https://errors.pydantic.dev/2.8/v/string_too_short Here the ``description`` field is too short; it must be at least 3 characters, which is indicated by the -``string_too_short`` error type. +``string_too_short`` error type and "String should have at least 3 characters" message. + +An error will also occur if an object is missing a required field or has other issues that prevent it from +being valid. -An error will occur when an object is missing a required field or has other issues that prevent it from being -valid. For example: :: @@ -68,9 +77,9 @@ For example: Field required [type=missing, input_value={'name': 'customer_id', '...ue customer identifier'}, input_type=dict] For further information visit https://errors.pydantic.dev/2.8/v/missing -In this case, the column is missing an ``@id`` field, which is always required for every object in the schema. +In this case, the column is missing an ``@id`` field, which is required on every object in the schema data. -Errors may also occur which reference an object like a column, as in: +Errors may also occur which generate a reference to an object like a column, as in: :: @@ -78,8 +87,7 @@ Errors may also occur which reference an object like a column, as in: Value error, Length must be provided for type 'string' in column '#customers.name' [type=value_error, input_value={'name': 'name', '@id': '...ame', 'nullable': False}, input_type=dict] For further information visit https://errors.pydantic.dev/2.8/v/value_error -This error indicates that the column is missing a length field, which is required for string columns. -The error occured in the second column of the first table. +This error indicates that the column is missing a length field, which is required on string columns. Optional Checks =============== @@ -93,7 +101,7 @@ These include the following: :``--check-tap-table-indexes``: Check that the ``tap:table_index`` field is unique for each table in the schema. :``--check-tap-principal``: Check that the ``tap:principal`` field is set for at least one column in each table. -Simply include one or more of these flags in the validation command to enable them: +Simply include one or more of these flags on the command line to enable them: .. code-block:: bash @@ -104,18 +112,26 @@ Simply include one or more of these flags in the validation command to enable th Validating with the Python API ============================== -The Python API can also be used to validate a schema. -This is done by creating a ``Schema`` object and calling the ``model_validate`` method: +The Python API can also be used to validate a schema by creating a ``Schema`` object and then calling the +``model_validate`` method: .. code-block:: python import yaml + from pydantic import ValidationError + from felis.datamodel import Schema data = yaml.safe_load(open("schema.yaml", "r")) - schema = Schema.model_validate(data) -This will raise an exception if the schema contains validation errors, printing them to the console like the -command line interface. -If there are no errors, the method will return the validated schema object, which can be used to create a -database or perform other operations. + try: + schema = Schema.model_validate(data) + except ValidationError as e: + print(e) + +If the schema contains validation errors, a message will be printed to the console/stdout printing all of +them individually. +Please see the :ref:`detailed error descriptions ` for more information on the format of +these error messages. +If there are no errors, the schema object will be returned and can be used to create a database or perform +other operations.