Skip to content

Commit

Permalink
Release notes soda-library 1.5.21 (#843)
Browse files Browse the repository at this point in the history
* add troubleshooting tip for SodaCL

* anomaly dashboard known issues
  • Loading branch information
janet-can authored Jul 31, 2024
1 parent 879f739 commit fbddc23
Show file tree
Hide file tree
Showing 4 changed files with 39 additions and 0 deletions.
16 changes: 16 additions & 0 deletions _release-notes/soda-library-1.5.21.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
name: "1.5.21"
date: 2024-07-31
products:
- soda-library
---

## 1.5.21 Fixes

* Add nchar, nvarchar and binary to text types for profiling. by @jzalucki in #281
* CLOUD 8061: alias table names in sql queries by @jzalucki in #280
* Oracle data source properties prefix should be None instead of "None" when no service name is provided. by @jzalucki in #283
* Sqlserver: use appropriate aggregate methods to build queries by @jzalucki in #284
* Cross row count check should support custom identity. by @jzalucki in #285
* Copyedit on frequency detection error message by @janet-can in #287
* Chore: update auto-assignments by @milanaleksic in #289
13 changes: 13 additions & 0 deletions soda-cl/troubleshoot.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ parent: SodaCL reference
[Errors when using in-check filters](#errors-when-using-in-check-filters)<br />
[Using reference checks with Spark DataFrames](#using-reference-checks-with-spark-dataframes)<br />
[Single quotes in valid values list result in error](#single-quotes-in-valid-values-list-result-in-error)<br />
[Databricks issue with column names that being with a number](#databricks-issue-with-column-names-that-being-with-a-number)<br />
<br />

<hr/>
Expand Down Expand Up @@ -242,6 +243,18 @@ checks for my_dataset:
{% include single-quotes.md %}


## Databricks issue with column names that being with a number

**Problem:** When running scans on Databricks, Soda encounters an error on columns that begin with a number.

**Solution:** In Databricks, when dealing with column names that start with numbers or contain special characters such as spaces, you typically need to use backticks to enclose the column identifier. This is because Databricks uses a SQL dialect that is similar to Hive SQL, which supports backticks for escaping identifiers. For example:
```yaml
checks for soda_test:
- missing_count(`1_bigint`):
name: test
fail: when > 0
```
## Go further
* Need help? Join the <a href="https://community.soda.io/slack" target="_blank"> Soda community on Slack</a>.
Expand Down
5 changes: 5 additions & 0 deletions soda-cloud/anomaly-dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ After establishing these patterns, Soda automatically detects anomalies relative
&nbsp;&nbsp;&nbsp;&nbsp;[Activate an anomaly dashboard to an existing dataset](#activate-an-anomaly-dashboard-to-an-existing-dataset)<br />
[About the anomaly dashboard](#about-the-anomaly-dashboard)<br />
&nbsp;&nbsp;&nbsp;&nbsp;[Empty metrics tiles](#empty-metrics-tiles)<br />
&nbsp;&nbsp;&nbsp;&nbsp;[Known issues and limitations](#known-issues-and-limitations)<br />
[Add anomaly notifications](#add-anomaly-notifications)<br />
[About profiling and partitioning](#about-profiling-and-partitioning)<br />
&nbsp;&nbsp;&nbsp;&nbsp;[Change the time partitioning column](#change-the-time-partitioning-column)<br />
Expand Down Expand Up @@ -105,6 +106,10 @@ If, after the anomaly detection algorithm has completed its pattern training, th
* There is no column that contains NUMBER type data (INT, FLOAT, etc.) which an average metric check requires. Where it cannot detect a column with the necessary data type, Soda leaves the **Average** tile blank.
### Known issues and limitations
* Soda anomaly dashboard does not profile columns that contain timestamps or dates. As such, Soda only executes a freshness check for such columns for the anomaly dashboard to validate data freshness, but not anomalies in the columns that contain dates or timestamps.
## Add anomaly notifications
{% include anomaly-notifs.md %}
Expand Down
5 changes: 5 additions & 0 deletions soda/new-documentation.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,11 @@ parent: Learning resources

<br />

#### July 31, 2024
* Added [release notes]({% link release-notes/all.md %}) documentation for Soda Library 1.5.21.
* Added [troubleshooting tip]({% link soda-cl/troubleshoot.md %}#databricks-issue-with-column-names-that-being-with-a-number) for running Soda scans on Databricks where column names beging with numbers.
* Added [Known issues and limitations]({% link soda-cloud/anomaly-dashboard.md %}#known-issues-and-limitations) section to anomaly dashboard content.

#### July 29, 2024
* Added [release notes]({% link release-notes/all.md %}) documentation for Soda Core 3.3.13.

Expand Down

0 comments on commit fbddc23

Please sign in to comment.