add settings validation docs #1648

ThomasHepworth · 2023-10-12T14:57:03Z

Type of PR

BUG
FEAT
MAINT
DOC

Is your Pull Request linked to an existing Issue or Pull Request?

#1252

Give a brief description for the solution you have provided

Adds documentation to help in the process of extending our existing settings validation code.

These docs cover:

The settings schema and some basic information on how to update it
Our additional validation checks, giving an overview of the current code and information on how to extend the existing checks.

PR Checklist

Added documentation for changes
Added feature to example notebooks or tutorial (if appropriate)
Added tests (if appropriate)
Made changes based off the latest version of Splink
Run the linter

RossKen · 2023-10-17T09:58:05Z

Can we add a link and one liner about this to the index.md in dev guides?

docs/dev_guides/settings_validation.md

RossKen · 2023-10-17T16:13:40Z

docs/dev_guides/settings_validation.md

+To extend the column checks, you simply need to add an additional validation method to the `InvalidColValidator` class. The implementation of this will vary depending on whether you merely need to assess the validity of a single column or columns within a SQL statement.
+
+#### Single Column Checks
+To assess single columns, the `validate_settings_column` should typically be used. This takes in a `setting_id` (analogous to the title you want to give your log string) and a list of columns to be checked.


Should valid_settings_column have a link? Also, it is not clear what it is. A function? A class?

Also, here I think it may be helpful to state the context at the start i.e. Checks are applied on all columns by default, but there is functionality available to apply more specific checks

Is it helpful here to refer to as "specific column checks" or "named column checks" as this mentions being able to refer to multiple columns, not just one? So I'm not sure if "single column checks" is the clearest naming. However I may be misunderstanding!

RossKen

Really helpful content - just a couple of stylistic/wording changes

RossKen

Really helpful content - just a couple of stylistic/wording changes

…dates improve the settings validation documentation

RossKen · 2023-11-02T11:14:11Z

docs/dev_guides/index.md

@@ -29,4 +29,4 @@ Splink is quite a large, complex codebase. The guides in this section lay out so
 * [Comparison and Comparison Level Libraries](./comparisons/new_library_comparisons_and_levels.md) - demonstrates how `Comparison` Library and `ComparisonLevel` Library functions are structured within Splink, including how to add new functions and edit existing functions.
 * [Charts](./charts.ipynb) - demonstrates how charts are built in Splink, including how to add new charts and edit existing charts.
 * [User-Defined Functions](./udfs.md) - demonstrates how User Defined Functions (UDFs) are used to provide functionality within Splink that is not native to a given SQL backend.
-
+* [Settings Validation](./settings_validation.md) - summarises how to use and expand the existing settings schema and validation functions.


This link is broken

docs/dev_guides/settings_validation/settings_validation_overview.md

docs/dev_guides/settings_validation/extending_settings_validator.md

RossKen

Thanks for making those changes - I think it is much clearer now. 🙌

Approving, but have left some minor comments to tweak before merging

…dates Settings val updates

add settings validation docs

97334a9

ThomasHepworth requested a review from RossKen October 12, 2023 14:57