Skip to content

Commit

Permalink
Deployed d04c754 with MkDocs version: 1.5.3
Browse files Browse the repository at this point in the history
  • Loading branch information
Unknown committed Mar 14, 2024
1 parent e4a4379 commit 797bb58
Show file tree
Hide file tree
Showing 7 changed files with 191 additions and 133 deletions.
103 changes: 66 additions & 37 deletions changelog/CHANGELOG/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@
<div data-md-component="skip">


<a href="#140-2024-03-13" class="md-skip">
<a href="#141-2024-03-14" class="md-skip">
Skip to content
</a>

Expand Down Expand Up @@ -1099,6 +1099,30 @@
</label>
<ul class="md-nav__list" data-md-component="toc" data-md-scrollfix>

<li class="md-nav__item">
<a href="#141-2024-03-14" class="md-nav__link">
<span class="md-ellipsis">
1.4.1 (2024-03-14)
</span>
</a>

<nav class="md-nav" aria-label="1.4.1 (2024-03-14)">
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#bug-fixes" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
</a>

</li>

</ul>
</nav>

</li>

<li class="md-nav__item">
<a href="#140-2024-03-13" class="md-nav__link">
<span class="md-ellipsis">
Expand Down Expand Up @@ -1134,7 +1158,7 @@
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#bug-fixes" class="md-nav__link">
<a href="#bug-fixes_1" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand All @@ -1158,7 +1182,7 @@
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#bug-fixes_1" class="md-nav__link">
<a href="#bug-fixes_2" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1206,7 +1230,7 @@
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#bug-fixes_2" class="md-nav__link">
<a href="#bug-fixes_3" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand All @@ -1230,7 +1254,7 @@
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#bug-fixes_3" class="md-nav__link">
<a href="#bug-fixes_4" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1263,7 +1287,7 @@
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#bug-fixes_4" class="md-nav__link">
<a href="#bug-fixes_5" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1311,7 +1335,7 @@
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#bug-fixes_5" class="md-nav__link">
<a href="#bug-fixes_6" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand All @@ -1335,7 +1359,7 @@
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#bug-fixes_6" class="md-nav__link">
<a href="#bug-fixes_7" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1368,7 +1392,7 @@
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#bug-fixes_7" class="md-nav__link">
<a href="#bug-fixes_8" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1401,7 +1425,7 @@
</li>

<li class="md-nav__item">
<a href="#bug-fixes_8" class="md-nav__link">
<a href="#bug-fixes_9" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1458,7 +1482,7 @@
</li>

<li class="md-nav__item">
<a href="#bug-fixes_9" class="md-nav__link">
<a href="#bug-fixes_10" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand All @@ -1482,7 +1506,7 @@
<ul class="md-nav__list">

<li class="md-nav__item">
<a href="#bug-fixes_10" class="md-nav__link">
<a href="#bug-fixes_11" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1515,7 +1539,7 @@
</li>

<li class="md-nav__item">
<a href="#bug-fixes_11" class="md-nav__link">
<a href="#bug-fixes_12" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1548,7 +1572,7 @@
</li>

<li class="md-nav__item">
<a href="#bug-fixes_12" class="md-nav__link">
<a href="#bug-fixes_13" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1581,7 +1605,7 @@
</li>

<li class="md-nav__item">
<a href="#bug-fixes_13" class="md-nav__link">
<a href="#bug-fixes_14" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1614,7 +1638,7 @@
</li>

<li class="md-nav__item">
<a href="#bug-fixes_14" class="md-nav__link">
<a href="#bug-fixes_15" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1647,7 +1671,7 @@
</li>

<li class="md-nav__item">
<a href="#bug-fixes_15" class="md-nav__link">
<a href="#bug-fixes_16" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1680,7 +1704,7 @@
</li>

<li class="md-nav__item">
<a href="#bug-fixes_16" class="md-nav__link">
<a href="#bug-fixes_17" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1713,7 +1737,7 @@
</li>

<li class="md-nav__item">
<a href="#bug-fixes_17" class="md-nav__link">
<a href="#bug-fixes_18" class="md-nav__link">
<span class="md-ellipsis">
Bug Fixes
</span>
Expand Down Expand Up @@ -1929,18 +1953,23 @@

<h1>Changelog</h1>

<h2 id="141-2024-03-14"><a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/compare/v1.4.0...v1.4.1">1.4.1</a> (2024-03-14)</h2>
<h3 id="bug-fixes">Bug Fixes</h3>
<ul>
<li>fixed bug in results saving logic (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/32">#32</a>) (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/613b596c2867ecdf71d565d759d39a17abf866dd">613b596</a>)</li>
</ul>
<h2 id="140-2024-03-13"><a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/compare/v1.3.2...v1.4.0">1.4.0</a> (2024-03-13)</h2>
<h3 id="features">Features</h3>
<ul>
<li>new db table to store metric errors (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/31">#31</a>) (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/37db1edc940d870815dda2ba86d3d2b277ba0650">37db1ed</a>)</li>
</ul>
<h2 id="132-2024-03-11"><a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/compare/v1.3.1...v1.3.2">1.3.2</a> (2024-03-11)</h2>
<h3 id="bug-fixes">Bug Fixes</h3>
<h3 id="bug-fixes_1">Bug Fixes</h3>
<ul>
<li>resolve config values substitution logic during encryption/decryption. (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/30">#30</a>) (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/b252f9d2ca4b9eec3c0ad92692b5fa323b4ed953">b252f9d</a>)</li>
</ul>
<h2 id="131-2024-3-4"><a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/compare/v1.3.0...v1.3.1">1.3.1</a> (2024-3-4)</h2>
<h3 id="bug-fixes_1">Bug Fixes</h3>
<h3 id="bug-fixes_2">Bug Fixes</h3>
<ul>
<li>update prepare-release scripts in order to fix auto-versioning and changelog formatting. (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/29">#29</a>) (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/9a30d59ec0b6e84225e0bbbf458546b50887c083">9a30d59</a>)</li>
</ul>
Expand All @@ -1950,12 +1979,12 @@ <h3 id="features_1">Features</h3>
<li>job configuration enhancements: added description and metadata fields and refactored hive source partition filtering (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/28">#28</a>) (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/7cf24db0d4999e064612f0658ddf0f836488ec86">7cf24db</a>)</li>
</ul>
<h2 id="121-2024-2-26"><a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/compare/v1.2.0...v1.2.1">1.2.1</a> (2024-2-26)</h2>
<h3 id="bug-fixes_2">Bug Fixes</h3>
<h3 id="bug-fixes_3">Bug Fixes</h3>
<ul>
<li>enhance casting methods for proper conversion of primitive values to target types (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/27">#27</a>) (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/744384b77cee7bac7c23d3af700ae7fb78c0a846">744384b</a>)</li>
</ul>
<h2 id="120-2024-2-16"><a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/compare/v1.1.1...v1.2.0">1.2.0</a> (2024-2-16)</h2>
<h3 id="bug-fixes_3">Bug Fixes</h3>
<h3 id="bug-fixes_4">Bug Fixes</h3>
<ul>
<li>docs update (describe new type of conn) and minor fixes (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/23">#23</a>) (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/c6abb811159df60499516594dde4900fd8a4ca1a">c6abb81</a>)</li>
<li>minor code fixes (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/25">#25</a>) (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/688929d3e8e41ff14d51d619d1acb3ba1990faa3">688929d</a>)</li>
Expand All @@ -1967,7 +1996,7 @@ <h3 id="features_2">Features</h3>
<li>new entity in storage db - job config state (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/24">#24</a>) (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/fbe650c6cf79dc0e8991c8e88111ec482916b8ee">fbe650c</a>)</li>
</ul>
<h2 id="111-2023-12-20"><a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/compare/v1.1.0...v1.1.1">1.1.1</a> (2023-12-20)</h2>
<h3 id="bug-fixes_4">Bug Fixes</h3>
<h3 id="bug-fixes_5">Bug Fixes</h3>
<ul>
<li>enhance kakfa windowing for streams and enable avro kafka message format (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/20">#20</a>) (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/a5e9827213da36ebc8932195018943204c3db309">a5e9827</a>)</li>
</ul>
Expand All @@ -1979,12 +2008,12 @@ <h3 id="features_3">Features</h3>
<li>adding streaming sources and stream readers (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/16">#16</a>) (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/b5398fa2c8b22fac191e5853aed6f29d5d4f3821">b5398fa</a>)</li>
</ul>
<h2 id="101-2023-11-13"><a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/compare/v1.0.0...v1.0.1">1.0.1</a> (2023-11-13)</h2>
<h3 id="bug-fixes_5">Bug Fixes</h3>
<h3 id="bug-fixes_6">Bug Fixes</h3>
<ul>
<li>sbt assembly fixes and custom source restoring (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/12">#12</a>) (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/b6855ea89adcfb5af0a0b742ce4a0d8a42f343ac">b6855ea</a>)</li>
</ul>
<h2 id="100-2023-11-01">1.0.0 (2023-11-01)</h2>
<h3 id="bug-fixes_6">Bug Fixes</h3>
<h3 id="bug-fixes_7">Bug Fixes</h3>
<ul>
<li>fixes related to project build for different version of Spark (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/10">#10</a>) (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/c7218bd96b377a5e3cdfbcde4ee899bff5488a03">c7218bd</a>), closes <a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/7">#7</a></li>
</ul>
Expand All @@ -1996,7 +2025,7 @@ <h3 id="features_4">Features</h3>
<li>Publishing major update of Checkita DQ (<a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/commit/09af3a2f065c0649849a026db41c5d3703d6233d">09af3a2</a>), closes <a href="https://github.com/Raiffeisen-DGTL/checkita-data-quality/issues/3">#3</a></li>
</ul>
<h2 id="035-2023-09-19">0.3.5 (2023-09-19)</h2>
<h3 id="bug-fixes_7">Bug Fixes</h3>
<h3 id="bug-fixes_8">Bug Fixes</h3>
<ul>
<li>Ensure that JDBC connection is alive prior saving results.
There could be situation when Application runs quite long and DB server can close idle connection due to
Expand All @@ -2007,7 +2036,7 @@ <h3 id="features_5">Features</h3>
<ul>
<li>Added support of customisable email subject templates.</li>
</ul>
<h3 id="bug-fixes_8">Bug Fixes</h3>
<h3 id="bug-fixes_9">Bug Fixes</h3>
<ul>
<li>Fixed duplicateValues metric: some duplicate values can only be determined during metric calculator merge.</li>
<li>Fix sender name for check Alert. It was hardcoded but need to refer the name configured in application configuration file.</li>
Expand All @@ -2030,13 +2059,13 @@ <h3 id="features_7">Features</h3>
<li>Added custom source</li>
<li>Docs update</li>
</ul>
<h3 id="bug-fixes_9">Bug Fixes</h3>
<h3 id="bug-fixes_10">Bug Fixes</h3>
<ul>
<li>Pass both source and virtual sources to target processors</li>
<li>Fix json serialization bugs.</li>
</ul>
<h2 id="031-2023-08-17">0.3.1 (2023-08-17)</h2>
<h3 id="bug-fixes_10">Bug Fixes</h3>
<h3 id="bug-fixes_11">Bug Fixes</h3>
<ul>
<li>Fix params json when sending results to kafka</li>
<li>Add sourceId and sourceKeyFields to errorCollection reports (or kafka messages)</li>
Expand All @@ -2057,7 +2086,7 @@ <h3 id="features_8">Features</h3>
<li>Added option to run DQ in Shared Spark Context</li>
<li>Added new types of history DB: Hive and File (both managed by spark without extra services)</li>
</ul>
<h3 id="bug-fixes_11">Bug Fixes</h3>
<h3 id="bug-fixes_12">Bug Fixes</h3>
<ul>
<li>Fixed SQL checks</li>
<li>Made DQ case-insensitive in terms of column names</li>
Expand All @@ -2070,7 +2099,7 @@ <h3 id="features_9">Features</h3>
<li>Project is rebuild for Scala 2.12.18</li>
<li>Added test for HdfsReader</li>
</ul>
<h3 id="bug-fixes_12">Bug Fixes</h3>
<h3 id="bug-fixes_13">Bug Fixes</h3>
<ul>
<li>Fixed HdfsReader in terms of loading fixed-width files: added casting to requested column types</li>
<li>HBase source is temporarily turned off (due to connector is incompatible with newer versions of Scala and Spark)</li>
Expand All @@ -2089,7 +2118,7 @@ <h3 id="features_10">Features</h3>
<li>Added new DQ application argument -v to pass extra variables to be prepended to application configuration file. Can be used to pass secrets for email, mattermost and storage DB on startup.</li>
<li>Documentation is updated according to new features.</li>
</ul>
<h3 id="bug-fixes_13">Bug Fixes</h3>
<h3 id="bug-fixes_14">Bug Fixes</h3>
<ul>
<li>Fixed email configuration for cases when smpt support anonymous connection (user and password are undefined)</li>
</ul>
Expand All @@ -2105,7 +2134,7 @@ <h3 id="features_11">Features</h3>
<li>Update application config files with default production settings for mailing</li>
<li>Update documentation</li>
</ul>
<h3 id="bug-fixes_14">Bug Fixes</h3>
<h3 id="bug-fixes_15">Bug Fixes</h3>
<ul>
<li>Update spark accumulator for metric error collection</li>
<li>Add TLS Support to mailer</li>
Expand All @@ -2123,7 +2152,7 @@ <h3 id="features_12">Features</h3>
DQ configuration file in runtime and may be references within it.</li>
<li>Updated documentation to reflect aforementioned changes.</li>
</ul>
<h3 id="bug-fixes_15">Bug Fixes</h3>
<h3 id="bug-fixes_16">Bug Fixes</h3>
<ul>
<li>Updated metric tests to cover Statusable calculation.</li>
<li>Refactored configuration file parsing: now the variables are prepended to configuration file by means of streams
Expand All @@ -2136,7 +2165,7 @@ <h3 id="features_13">Features</h3>
<li>Added new results view to database init sql</li>
<li>Added jobId to metric results model</li>
</ul>
<h3 id="bug-fixes_16">Bug Fixes</h3>
<h3 id="bug-fixes_17">Bug Fixes</h3>
<ul>
<li>Fixed composed metric calculator: increased power operation priority</li>
<li>Fixed database config reading</li>
Expand All @@ -2149,7 +2178,7 @@ <h3 id="features_14">Features</h3>
<li>Added unit tests for column and file metric calculators;</li>
<li>Refactored date-related metrics to make them work with timestamp column type correctly;</li>
</ul>
<h3 id="bug-fixes_17">Bug Fixes</h3>
<h3 id="bug-fixes_18">Bug Fixes</h3>
<ul>
<li>Fix build.sbt for production environment</li>
<li>Add hive meta schema initialisation (script for schema initialisation in DQ database, not part of DQ application);</li>
Expand Down
2 changes: 1 addition & 1 deletion index.html
Original file line number Diff line number Diff line change
Expand Up @@ -1225,7 +1225,7 @@


<h1 id="home">Home</h1>
<p><strong>Latest Version: 1.4.0</strong></p>
<p><strong>Latest Version: 1.4.1</strong></p>
<p>To ensure quality of big data, it is necessary to perform calculations of a large number of metrics and checks
on huge datasets, which in turn is a difficult task.</p>
<p>Checkita is a Data Quality Framework that solves this problem by formalizing and simplifying the process connecting
Expand Down
Loading

0 comments on commit 797bb58

Please sign in to comment.