Releases: apache/incubator-devlake
v0.12.0
Breaking Changes
- MultiConnection was added to all our data-source plugins, all collected data will be wiped during the upgrade in order to ensure data integrity. #636 #1978 #2262
pipeline.tasks
was renamed topipeline.plan
#1951pipeline.plan[][].options.tasks
was moved topipeline.plan[][].subtasks
#1412- Separated Data Transformation Rules from Connection and bound to Scope #1548
- Introduced
Dal
interface for Database decoupling (deprecating*gorm.DB
dependency) #2183 - All
connection.rate_limit
were renamed toconnection.rate_limit_per_hour
#2360 - All
*users
tables were renamed to*accounts
, and theusers
table in Domain Layer is now representing Unique Identity #2431
Features
- New User Flow based on Blueprint #1951 #1979 #1980 #1981 #1982 #1986 #1987 #2468 #2472
- Team feature support #2361 #2426 #2427 #2428 #2429 #2430 #2431 #2477
- Support collecting pr/issue comments from Github / Gitlab #1746, #1758, #1760
- Automatic test including unit-test/e2e-test #1754 #1765 #2491 #1824 #1851 #1853
- New Dashboard "Community Overview" and other dashboard updates #1995 #2323 #2328
- Support collecting data from Jenkins Folders #2145
- Skip HTTPS certificate verification #2370
- Separated Github Tokens #2384 #2423
- Collect organization information for Github users #2408
- Skip collecting changelog for epic issues to reduce total time #2434
- Acquire user's consent before db migration #2440
- Collect Github comments/reviews #2452
- Throttling total number of pipelines running in parallel #2473
Others
- Remove outdated field for Jira plugin #1711
- Release
scheduler
#1772 - Total number of fixed bugs: 43
- Total number of commits: 359
- Files changed: 1288
New Contributors
- @SnowMoon-Dev made their first contribution in #2074
- @moonlzl made their first contribution in #2221
- @long2ice made their first contribution in #2200
- @marcemv90 made their first contribution in #2238
- @krshkodes made their first contribution in #2388
- @yumengwang03 made their first contribution in #2479
- @phzbbbbbbbb made their first contribution in #2492
- @Pranshu-Raj made their first contribution in #2488
Full Changelog: v0.11.0...v0.12.0
v0.11.0
Changelogs:
- Support Postgres
- Support collecting private git repos
- Improve unit test
- Redesign and implement configuration flow
v0.10.1
v0.10.1 is a minor release on top of v0.10.0 with mostly dashboard/docs updates.
Major Changes since Last Release (v0.10.0)
- Support incremental collection for Jira worklog
- Add a dashboard for weekly bug retro use case (
grafana/WeeklyBugRetro.json
) - Add a generic Jenkins dashboard (
grafana/Jenkins.json
) - Improve the Github basic metrics dashboard (
grafana/GithubBasicMetrics.json
) - Update dashboards according to the latest domain layer schema
- Adopt a new file name convention to shorten file names
- Add docs for the advanced mode of Config UI
Full Changelog: v0.10.0...v0.10.1
v0.10.0
Major Changes since Last Release (v0.9.3)
- Pipeline Blueprint allows users to schedule recurring pipelines with cron expression
- Introduce DB migrations to DevLake to provide a smoother upgrade experience
- Add temporal mode for users with heavy data collection workload
- Add dbt plugin for users who want to define their own data transformations
- Store/cache API responses from data sources in the raw data layer for instant replay of data transformations
Full Changelog: v0.9.3...v0.10.0
Bug Fixes & Improvements since Last Beta Release (v0.10.0-beta1)
- Improve documentation for user setup and GitHub data collection process
- Fix incorrect
created_date
in domain layer - Fix stuck pipelines due to retries
- Add users for plugin Jira
Full Changelog: v0.10.0-beta1...v0.10.0
New Contributors
- @liangjingyang made their first contribution in https://github.com/merico-dev/lake/pull/377
- @nitinvasanth made their first contribution in https://github.com/merico-dev/lake/pull/526
- @IronCore864 made their first contribution in https://github.com/merico-dev/lake/pull/747
- @ggcannard made their first contribution in https://github.com/merico-dev/lake/pull/752
- @MaximDub made their first contribution in https://github.com/merico-dev/lake/pull/885
- @Juneezee made their first contribution in https://github.com/merico-dev/lake/pull/1013
- @basicthinker made their first contribution in https://github.com/merico-dev/lake/pull/1178
- @daniel-hutao made their first contribution in https://github.com/merico-dev/lake/pull/1193
- @abeizn made their first contribution in https://github.com/merico-dev/lake/pull/1343
v0.10.0-beta1
Major Changes since Last Release (v0.9.3)
- Pipeline Blueprint allows users to schedule recurring pipelines with cron expression
- Introduce DB migrations to DevLake to provide a smoother upgrade experience
- Add temporal mode for users with heavy data collection workload
- Add dbt plugin for users who want to define their own data transformations
- Store/cache API responses from data sources in the raw data layer for instant replay of data transformations
Full Changelog: v0.9.3...v0.10.0-beta1
New Contributors
- @liangjingyang made their first contribution in https://github.com/merico-dev/lake/pull/377
- @nitinvasanth made their first contribution in https://github.com/merico-dev/lake/pull/526
- @IronCore864 made their first contribution in https://github.com/merico-dev/lake/pull/747
- @ggcannard made their first contribution in https://github.com/merico-dev/lake/pull/752
- @MaximDub made their first contribution in https://github.com/merico-dev/lake/pull/885
- @Juneezee made their first contribution in https://github.com/merico-dev/lake/pull/1013
- @basicthinker made their first contribution in https://github.com/merico-dev/lake/pull/1178
- @daniel-hutao made their first contribution in https://github.com/merico-dev/lake/pull/1193
- @abeizn made their first contribution in https://github.com/merico-dev/lake/pull/1343
v0.9.3
v0.9.2
Fixed issues found in v0.9.1
- Improved
gitextractor
performance, from Forever to less than an hour on Clickhouse repo #1435 - Fixed
commit_sha
being incorrect fromrefs
table #1441 - Skipped invalid
pull_request_issues
records when enriching data from github plugin #1438 - Enhanced
config-ui
field behavior on connection settings and other tweaks #1427, #1422, #1436, #1444 - Added
author_id
autho_name
anddescription
field topull_request
table #1431 - Fixed
pull_request_commits
data corruption #1430 - Fixed connection status on connection page from
config-ui
v0.9.1
Fixed issues found in v0.9.0
- v0.9.0 wouldn't pick up github bug/requirement settings unless devlake got restarted (https://github.com/merico-dev/lake/issues/1394)
- v0.9.0 github settings - field saving issues (https://github.com/merico-dev/lake/issues/1393)
- Github configuration: the placeholder of severity is incorrect (https://github.com/merico-dev/lake/issues/1384)
Other improvements:
- Improve demo dashboards (https://github.com/merico-dev/lake/pull/1398)
v0.9.0
IMPORTANT: DevLake doesn't support Database Schema Migration yet, upgrading an existing instance is likely to break, we recommend that you deploy a new instance instead.
What's Changed
- fix: gitlab provider - disable project id mappings field by @e2corporation in https://github.com/merico-dev/lake/pull/1260
- EE integration bug fix by @mindlesscloud in https://github.com/merico-dev/lake/pull/1132
- feat: collect gitlab tags by @likyh in https://github.com/merico-dev/lake/pull/1267
fix
: create pipeline : optimize advanced menu accessibility by @e2corporation in https://github.com/merico-dev/lake/pull/1268- refactor(github): improve github collection performance by @warren830 in https://github.com/merico-dev/lake/pull/1253
- fix(plugins): fix concurrent problems by @warren830 in https://github.com/merico-dev/lake/pull/1275
- docs: refine the main readme by @Startrekzky in https://github.com/merico-dev/lake/pull/1277
- docs: update Chinese main README by @Startrekzky in https://github.com/merico-dev/lake/pull/1283
- fix: multithread read empty config by @mappjzc in https://github.com/merico-dev/lake/pull/1263
- Feat add GitHub response field by @mappjzc in https://github.com/merico-dev/lake/pull/1282
- feat(github): extract relation between pr and issue by @warren830 in https://github.com/merico-dev/lake/pull/1285
- fix(jenkins): change delete scope by @warren830 in https://github.com/merico-dev/lake/pull/1290
- fix(github): fix test connection by @warren830 in https://github.com/merico-dev/lake/pull/1289
- fix: spell component by @mappjzc in https://github.com/merico-dev/lake/pull/1291
- feat: add validate for force required field by @mappjzc in https://github.com/merico-dev/lake/pull/1297
- fix: jira settings > enable reset query on multiselect fields by @e2corporation in https://github.com/merico-dev/lake/pull/1296
- fix: create pipeline > restore basic textarea editor in advanced mode by @e2corporation in https://github.com/merico-dev/lake/pull/1295
- fix: jira proxy should be lowercase by @klesh in https://github.com/merico-dev/lake/pull/1292
- Feat creat ref bug stats by @warren830 in https://github.com/merico-dev/lake/pull/1293
- fix(plugins): add support for proxy field in api by @warren830 in https://github.com/merico-dev/lake/pull/1298
- fix(gitlab): fix confusing tables by @warren830 in https://github.com/merico-dev/lake/pull/1300
- feat: reconfigure proxy and extend github settings by @e2corporation in https://github.com/merico-dev/lake/pull/1276
- jira server support by @mindlesscloud in https://github.com/merico-dev/lake/pull/1301
- fix: handle status code 404 by @mindlesscloud in https://github.com/merico-dev/lake/pull/1305
- fix(frontend): modify jira url for issuetyp and field by @warren830 in https://github.com/merico-dev/lake/pull/1309
- fix(jenkins): change DurationSec unit to second by @warren830 in https://github.com/merico-dev/lake/pull/1310
- chore(docs): release v0.9.0 by @warren830 in https://github.com/merico-dev/lake/pull/1313
Full Changelog: merico-dev/lake@v0.8.0...v0.9.0
v0.8.0
IMPORTANT: DevLake doesn't support Database Schema Migration yet, upgrading an existing instance is likely to break, we recommend that you deploy a new instance instead.
What's Changed
- docs: update issue templates by @Startrekzky in https://github.com/merico-dev/lake/pull/1173
- Automated tests by @joncodo in https://github.com/merico-dev/lake/pull/1174
- Update README by @basicthinker in https://github.com/merico-dev/lake/pull/1178
- Fix store pr labels individually by @warren830 in https://github.com/merico-dev/lake/pull/1180
- fix(plugins): pass pointer instead value by @warren830 in https://github.com/merico-dev/lake/pull/1183
docs
: update main README.md by @e2corporation in https://github.com/merico-dev/lake/pull/1152- docs: fixed instructions for updated installation by @kevin-kline in https://github.com/merico-dev/lake/pull/1154
- refactor(plugins): use cursor to iterate dbdata by @warren830 in https://github.com/merico-dev/lake/pull/1185
feat
: setup multi-selectors for jira issue types w/ api proxy |1064
by @e2corporation in https://github.com/merico-dev/lake/pull/1177- Task cancelling for GitHub plugin by @kevin-kline in https://github.com/merico-dev/lake/pull/1179
- WIP by @joncodo in https://github.com/merico-dev/lake/pull/1188
- Feat: Adding Kubernetes deploy support by @daniel-hutao in https://github.com/merico-dev/lake/pull/1193
- refactor(domain layer,plugins): indicate which sub task failed by @warren830 in https://github.com/merico-dev/lake/pull/1181
- docs: update main README for v0.7.0 by @klesh in https://github.com/merico-dev/lake/pull/1197
- docs: update English and Chinese Readmes to v0.7.0 by @Startrekzky in https://github.com/merico-dev/lake/pull/1198
hotfix
: config-ui - fix create pipeline page |1176
by @e2corporation in https://github.com/merico-dev/lake/pull/1191- feat: git extractor support private repo by @mindlesscloud in https://github.com/merico-dev/lake/pull/1202
- E2E tests that can be run manually once per iteration by @joncodo in https://github.com/merico-dev/lake/pull/1189
- fix: refdiff rook_pkg_path typo by @klesh in https://github.com/merico-dev/lake/pull/1206
- feat:Encrypt for JiraAuth by @mappjzc in https://github.com/merico-dev/lake/pull/1205
- docs: fix typo for
gitextractor
plugin by @klesh in https://github.com/merico-dev/lake/pull/1208 - docs: update readme example code by @klesh in https://github.com/merico-dev/lake/pull/1209
- fix(refdiff): fix progress by @warren830 in https://github.com/merico-dev/lake/pull/1210
- Fix jenkins save builds by @warren830 in https://github.com/merico-dev/lake/pull/1204
- feat: create pipeline - advanced pipeline mode |
1127
by @e2corporation in https://github.com/merico-dev/lake/pull/1203 - fix(plugins): change assignee id by @warren830 in https://github.com/merico-dev/lake/pull/1211
- fix: use hooks to handle encryption and decryption by @mappjzc in https://github.com/merico-dev/lake/pull/1212
- Fix empty pipeline stage causing backend to crash by @klesh in https://github.com/merico-dev/lake/pull/1214
- docs: update developer setup procedure by @klesh in https://github.com/merico-dev/lake/pull/1215
- fix(jenkins): fix table definition by @warren830 in https://github.com/merico-dev/lake/pull/1219
- fix: github reporting 'invalid token' for invalid endpoint url by @klesh in https://github.com/merico-dev/lake/pull/1218
- fix(github): make sure of data integration by @warren830 in https://github.com/merico-dev/lake/pull/1224
- fix: remove empty workflow by @klesh in https://github.com/merico-dev/lake/pull/1225
- fix(github): github not collecting pull requests by @klesh in https://github.com/merico-dev/lake/pull/1227
- fix: retry with new nonce by @mindlesscloud in https://github.com/merico-dev/lake/pull/1228
- fix gitextractor arguments validation by @klesh in https://github.com/merico-dev/lake/pull/1236
- fix(refdiff): correct progress by @warren830 in https://github.com/merico-dev/lake/pull/1235
- fix: ae plugin tasks argument not working by @klesh in https://github.com/merico-dev/lake/pull/1237
- fix: collect pipeline find with project_id by @mappjzc in https://github.com/merico-dev/lake/pull/1241
- fix(feishu): decrease ratelimit by @warren830 in https://github.com/merico-dev/lake/pull/1242
- fix(frontend): to run in advanced mode by @warren830 in https://github.com/merico-dev/lake/pull/1240
- Fix jira collect board panic by @mappjzc in https://github.com/merico-dev/lake/pull/1244
- User setup fixes by @klesh in https://github.com/merico-dev/lake/pull/1254
- chore: release v0.8.0 by @klesh in https://github.com/merico-dev/lake/pull/1255
- Hotfix for github PR by @klesh in https://github.com/merico-dev/lake/pull/1259
New Contributors
- @basicthinker made their first contribution in https://github.com/merico-dev/lake/pull/1178
- @daniel-hutao made their first contribution in https://github.com/merico-dev/lake/pull/1193
- @mappjzc made their first contribution in https://github.com/merico-dev/lake/pull/1205
Full Changelog: merico-dev/lake@v0.7.0...v0.8.0