Skip to content

Latest commit

 

History

History
961 lines (483 loc) · 58.8 KB

CHANGELOG.md

File metadata and controls

961 lines (483 loc) · 58.8 KB

0.9.0 (2024-12-02)

What's Changed

  • Relationships by @matteo-s in #133
  • feat: Create ModelServe functions from model reference by @kazhamiakin in #135
  • Schema exporting by @matteo-s in #136
  • Argoworkflows v2 by @matteo-s in #137
  • Argoworkflows by @kazhamiakin in #134
  • workflow runs refactor by @matteo-s in #140
  • fix(huggingfaceserve): Correct translation of LLM task types by @kazhamiakin in #139
  • refactor: fsm for states + transitions and new lifecycle manager by @matteo-s in #142
  • fix: runAsUser added to the argo workflow spec by @kazhamiakin in #143
  • Argo disable serviceaccount by @matteo-s in #144
  • API for templates by @micnori in #145
  • Update LICENSE by @Calcagiara in #146

Features

  • container runtime expose runAs into task specs (ad0a6d6)
  • function templates (7ab8653)
  • add generated schemas to repo (fcc5ecf)
  • add metadata.ref to exported resources (3d0b90a)
  • add run_of relationship to all runs (6e2fb40)
  • Create ModelServe functions from model reference (93a4587)
  • drop nefertem runtime from builds (56fcf71)
  • implement keyGenerator for id generation (aacce6f)
  • key utils + accessor (62d343e)
  • kfp workflow spec update to use source format for build (0dc9759)
  • openapi generator goal + export (6c2b6f7)
  • project expose relationships for all embedded entities (22eaaec)
  • remove config_map volume type (9f0c95b)
  • remove deprecated handler field from workflow spec (dfdac79)
  • schema exporter runner (wip) (e01790d)
  • spec exported to file for all specs with custom profile (b393c16)
  • Support for Build task and native Argo Workflow framework. (ed6325a)
  • support k8s jobs as suspended by default (f0951e2)
  • update spring + spring boot and bump version (8ddab95)

Bug Fixes

  • use varbinary instead of lob to avoid orhpaned oids in postgres (ced0154)
  • clear leftover extra fields (63018c5)
  • snake_case for k8s fields (ae97821)
  • add templates to dockerfile to include in images (3ad4183)
  • replace getFile with getInputStream to allow template service to read from classpath (61f4ef2)
  • lock jetty deps for solr (6bf0bcd)
  • remove outputs from python run spec (ceb5173)
  • avoid external state modification for non-local runs (ffd5faa)
  • fix nullpointer (+potential npes) in argo framework (7b44208)
  • remove NONE entityType (d1a9485)
  • disable service account and token mont config (e2149f2)
  • cascade delete models on project delete (b624260)
  • enforce run spec readonly after created stage (ff59564)
  • fix compile error for kfp build task (fdc1f36)
  • fix imports forargo workflow in kfp runtime (f64ecde)
  • fix kfp pipeline runner reading from base64 encoded spec (3d15c2a)
  • fix relationships repo definition (89524e3)
  • huggingfaceserve: Correct translation of LLM task types (7f75530)
  • keep snake_case for values in metadata (6cc1832)
  • leftover check for functionId in task service + run rels (238d233)
  • reprase relationship find query to be more precise (46457b1)
  • runAccessor task field (f5fbb1c)
  • runAsUser added to the argo workflow spec (e9414d0)
  • runmanager loads workflows (d96ee4f)

Full Changelog: https://github.com/scc-digitalhub/digitalhub-core/compare/0.8.0...0.9.0

0.8.0 (2024-10-24)

What's Changed

  • Websocket for run states by @etomaselli in #120
  • Project sharing by @matteo-s in #123
  • add solr basic auth by @micnori in #122
  • refactor: solr update + implement reindex + fixes by @matteo-s in #124
  • feat: drop mlrun runtime by @matteo-s in #125
  • refactor: secrets handled via providers + clear path logic to avoid s… by @matteo-s in #129

Features

  • update embedding to keep ids by default and reduce meta to base metadata (360ab4c)
  • add fsGroup to secContext + make user configurable for container (ee75082)
  • drop mlrun from application (d54ec17)
  • drop mlrun runtime (71e3a4b)
  • kfp workflow encoded as base64 (8871bd8)
  • projects search by user (71f746b)
  • refresh project authorities for internal tokens (e6343df)
  • use preferred_username by default for jwt auth, with fallback to sub (5c57559)
  • container tasks export context for source (1356ee2)
  • k8s service status info detailed collection (88d4149)
  • notify updatedBy user if different from owner (769b7c0)
  • k8s api response error reporting (c33456e)
  • cache on project share lookup (80f989d)
  • k8s serve/deploy resume + framework messages (f0c3d42)
  • project sharing (03dc7a4)
  • project sharing + user permissions (017e6bc)
  • user notifications model (41da04e)
  • websocket authentication + config (fdcb968)

Bug Fixes

  • secrets should be exposed as ENVS as provided (3faa787)
  • builder-tool as non root (7587cd1)
  • builder-tool does not exit for chmod issues (07b4667)
  • cronJob framework avoid double build of volumes (d17e52e)
  • project spec includes models (a63c7e6)
  • align secrets provider + add path checks (047ae8f)
  • fix huggingface path regex (d957384)
  • sanitize project secrets name for k8s (5444f16)
  • fix crontab regex (6c783b2)
  • python init_function field is a plain string (81e7827)
  • refresh_token native queries require token as text, postgres uses oid by default (d62c80e)
  • rewrite native query for refresh_tokens to fix text issues (e78b808)
  • token exchange should propagate claims as extracted by securityConfig (e904839)
  • wrong kfp wrapper image (9677b2e)
  • container build workdir set to /shared (1a18ba1)
  • fix container build mismatch between workdir and copy dest (bbb1655)
  • fix serve framework nullpointer ex (d3ee34b)
  • service processor should avoid looking for status if type != loadBalancer (569b7b9)
  • set LOGIN_URL for console for oidc (1fc3a2f)
  • fix nullpointer on run callback when not existing (8607fd5)
  • k8s pvc claimName in place of claim_name (0dab756)
  • batch delete should send events (72d17be)
  • fix delete of run with no runnable (e6dc6d7)
  • specRegistry schema generator additional modules should be optional + cleanups (013eb30)

0.7.0 (2024-09-20)

Bug Fixes

  • python build should set WORKDIR for user instructions (b270412)

0.7.0-beta10 (2024-09-19)

Features

  • add locking to run manager to fix fsm concurrency + add error messages to status (216d5b2)

0.7.0-beta9 (2024-09-18)

Bug Fixes

  • fix python install of requirements.txt (d8252ae)
  • fix seriazable definition to avoid loop + fix array def (6b03c9a)

Features

  • mlserve runtimes add validation to image and path (523b204)
  • python job schedule support + schedule validation via regex as crontab (41b531a)

0.7.0-beta8 (2024-09-16)

Bug Fixes

  • build sub path on download only if it differs from path (81566f8)
  • clean fileInfo and fix bug in httpStore (e17053e)
  • fix secret removal not working due to patch issue (63f76a9)
  • fix store key building (50ad12d)
  • handle log collection errors in loop to avoid stopping on first error (a9e9c25)
  • keep k8s status when there is no updates (97e2229)
  • remove source path modification for ml serving (66ed314)
  • use latest tag for python runtimes (8995114)

Features

  • add deps install for mlflow (c79db7c)
  • add transitions log to run status (d673785)
  • hide secret and configMaps details from run status (50a628b)
  • monitor restartCounts for deployments to check for errors (d56de95)

0.7.0-beta7r2 (2024-09-12)

Bug Fixes

  • fix kaniko init volume double definition (e4fadea)
  • remove double slash from upload url built path (5eddbc8)

Features

0.7.0-beta6 (2024-09-06)

Bug Fixes

  • fix service url building (b370ea9)
  • schema generation for serializable as ref to fix gui (ae6be88)

Features

  • expose fake node for profiles descriptions (f5bec00)
  • json schema custom generator for serializable fields + proxy building for specs before generation (c5e1664)
  • support templates for k8s objects via composite profiles (663d643)
  • transform schema utils into factory + add k8s schema module + enable proxy + build custom spec for templates/profiles (a8dd58a)
  • update run specs to exclude embedded function from schema generation (69982c3)
  • use resources definition from template when provided over user provided (49c02eb)

0.7.0-beta5 (2024-08-30)

0.7.0-beta4 (2024-08-28)

0.7.0-beta3 (2024-08-23)

Bug Fixes

  • add scanning for subpackages to include authorization (3fb3dbe)
  • enforce name as read-only for entities (c047743)
  • fix wrong auth used for refresh token consumption (71a60df)

Features

  • authorization aware service for projects + helper auth to inject roles for project owners (030fe1b)

0.7.0-beta2 (2024-08-22)

Bug Fixes

  • fix anonymous auth for token endpoint (becb7bd)
  • revert changed application properties and apply new config (bc13096)
  • revert changed jwt configuration to correct default (3c97c06)

Features

  • add exchange token flow between client and (internal/external) jwt auth (51a24be)
  • implement auth server + add refresh + add endpoints + refactoring (db5782b)

0.6.0 (2024-07-31)

Bug Fixes

  • fix build issues (4ecec34)
  • fix k8s gpu and volumes usage (ab1981b)
  • fix kaniko framework usage of secret volume mounts (62c23b6)

0.6.0-beta13 (2024-07-22)

Bug Fixes

  • fix k8s names sanitization, shorten generated names + disable fix for throw error (fae02ad)
  • fix templates reading for base k8s framework (55d4774)
  • runtimes should start without k8s (3cd8046)

Features

  • add label support to k8s templats (6cfc7fd)
  • add security config and default resources to k8s (6670cc7)
  • k8s improve observability (8d8e3e7)
  • k8s make image pull policy configurable (c526fdd)
  • k8s profiles as templates (8295d68)
  • update k8s configurability and logging (2f28f9e)

0.6.0-beta12 (2024-07-19)

Bug Fixes

  • fix image pull policy wrongly specified for k8s (91c3c8f)
  • fix wrong type in runnables written by runtimes (0d3d950)

Features

  • add custom entrypoint for python-runtime (d7f40d0)
  • add function name label to python and container runnables (6372ac8)
  • clean up k8s details in run after deletion (8929233)

0.6.0-beta11 (2024-07-17)

Bug Fixes

  • fix issue with deleting secret values from existing secrets (a9e44b9)
  • fix issue with s3 presigner not using path style with custom endpoints (166e921)
  • fix removal of orphaned pods for k8s jobs via propagaition policy (4f4169b)

Features

  • add k8s function label to container runtime runnables (895895b)
  • add namespaced labels to k8s resources (af26f64)
  • add prev entity to update events (debe27f)

0.6.0-beta10 (2024-07-12)

Features

  • add service to k8s run status (2b054b9)
  • build container supports source (22f40db)

0.6.0-beta9 (2024-07-12)

Bug Fixes

  • fix incorrect usage of k8deployment framework for serving (5adc24d)
  • schema controller should be accessible to all users (5012805)

0.6.0-beta8 (2024-07-11)

Bug Fixes

  • add secrets to serving (43dc2c5)
  • enforce sanitization on names generated for k8s and images (159e5c2)

0.6.0-beta7 (2024-07-11)

Bug Fixes

  • fix k8s init config map and cleanup (1414aaf)
  • remove notNull annotation from python serve: task should have all fields optional (0ec534e)

0.6.0-beta6 (2024-07-10)

Bug Fixes

  • fix issue with event publisher not distinguishing between runnable listeners (7d5492d)

Features

  • update python spec to default serviceType to nodePort (f56d8d4)

0.6.0-beta4 (2024-07-08)

0.6.0-beta3 (2024-07-04)

Bug Fixes

  • fix incorrect usage of urlEncoder for base64 (0957532)
  • fix issue with patching k8s secrets (61651c4)
  • fix unauthorized response from project controller with auth disabled (39f8ef0)
  • fix wrong state management on delete (b7c924a)

Features

  • add init_function to python runtime (30bf8cf)
  • cleanup k8s listeners and fix errors (0324ea5)
  • container runtime from base runtime (34542bd)
  • dbt runtime from base runtime (7ea4f8e)
  • kfp runtime from base runtime (c640230)
  • mlrun runtime from base runtime (1a0ce5a)
  • nefertem runtime from base runtime (befe3a1)
  • python runtime from base runtime (e0c765c)
  • refactor runtimes to leverage base impl (4dac16f)
  • refactor specs usage (84ab23f)
  • set default value in schema for single value enums (ff9474a)
  • use yaml for kdp workflow field (83d54d7)

0.6.0-beta2 (2024-06-24)

Bug Fixes

  • fix python build issues with requirements (d39e06a)

0.6.0-beta1 (2024-06-20)

Bug Fixes

  • align python builders logic (073ce75)
  • clear edited files (3a87962)
  • clear modified runtimes (e3782ec)
  • disable cors support by default (0b7ed7e)
  • fix build-tool image pinning to latest (b0c5e05)
  • fix issues with building python images + update config (52156d9)
  • fix rsync error including hidden files in builder-tool (caa391f)
  • fix wrongly modified container spec (a8c064a)
  • name audit metadata with snake_case (f6fb81d)
  • properly set condition on s3 store (e3abd76)
  • resolve symlink copy for build-tool (df23b58)

Features

  • add headers for api version (3f5b8eb)
  • cleanup redundant python builders (ae727f7)
  • files proxy initial support for s3/http (0803999)

0.5.0-beta8 (2024-06-10)

Bug Fixes

  • update run specs for input/outputs defs (5b21e99)

0.5.0-beta7 (2024-06-06)

Features

  • add k8s runtime/priority classes (b643c9f)
  • run context filter for id like (9511136)

0.5.0-beta6 (2024-06-05)

Features

  • python serve task (a5a9ee9)
  • remove runs after finals state (completed+error) via delete + cleanups (cb4cf32)

0.5.0-beta4 (2024-06-04)

Bug Fixes

  • handle local path in source for python runtime (0c166b9)

Features

  • add input/outputs to python run (2caa488)

0.5.0-beta3 (2024-05-31)

Bug Fixes

  • fix kaniko building contextSource map as array instead of strings (4e6fe41)

Features

  • add openmetadata integration as custom meta obj (263801c)
  • python runtime job execution (0593154)
  • update specs and add labels (0390fad)

0.5.0-beta2 (2024-05-29)

Bug Fixes

  • fix schema generation for records (65f2826)

Features

  • add title+desc to schemas derived for specs via annotation scan (d3b27de)
  • auto-generate title and desc in schema for fields + unwrap run specs (0978f0e)
  • unwrap container run specs + tweak json schema generation (8d954c8)
  • update k8s internal logging (77b4c10)

0.5.0-beta1 (2024-05-20)

Features

  • k8s metrics collection (462b11f)
  • log collection and proper refactoring of monitors (31b4b35)
  • model service align (d572dcd)
  • remove timestamp from logs (3a50713)
  • support models for ml (aa1ba15)
  • update log handling, collect all containers (759d536)

0.4.0 (2024-05-14)

Bug Fixes

  • always use (dhcore) for managed-by label (55ecad0)

0.4.0-beta13 (2024-05-14)

Bug Fixes

  • fix issues with run lifecycle in k8s + bump kubernetes client dep (0ce18b9)
  • fix name in pom + fix nullpointer in k8serve (f42b439)

0.4.0-beta12 (2024-05-10)

Bug Fixes

  • fix k8s labels (b774509)
  • project name and must must be equals (5c4b20f)

Features

  • use release name for core version (7fe8e58)
  • use revision for release version (3a849c1)
  • validation + search latest api + fixes (6117fbb)

0.4.0-beta11 (2024-05-09)

Bug Fixes

  • add converter to state in entities (3beff36)
  • fix issues with run fsm and concurrency + various fixes (34e42f1)

0.4.0-beta10 (2024-05-09)

Features

  • add registry config shared between k8s and kaniko (0d9a6ed)

0.4.0-beta9 (2024-05-07)

Bug Fixes

  • fix labels on k8s objects + remove required imageName from container (1e5c907)

0.4.0-beta8 (2024-05-07)

Bug Fixes

  • store secret for kaniko runs (5ffba49)

0.4.0-beta7 (2024-05-03)

0.4.0-beta3 (2024-04-29)

Features

  • secured runnable can transport auth details to frameworks (d72c2d5)

0.4.0-beta6 (2024-04-30)

Bug Fixes

  • builder tool should skip refs if not set (e9bd4be)
  • fix dockerfile building for container runtime (78ac814)

Features

  • update container function image after build via kaniko + clear k8s labels usage (fedd1b0)

0.4.0-beta5 (2024-04-30)

Bug Fixes

  • reattach console to upstream (0ed5389)

0.4.0-beta4 (2024-04-29)

Bug Fixes

  • update K8sKanikoFramework.java to use image registry (b0bf5cc)

0.4.0-beta3 (2024-04-29)

Features

  • secured runnable can transport auth details to frameworks (d72c2d5)

0.4.0-beta2 (2024-04-24)

Bug Fixes

  • fix entity type in workflow indexer (5b9ee5b)
  • fix oidc config params (1c08ade)

Features

  • action for building build-tool image (62b765f)
  • kaniko build infrastructure (5bd4fed)

0.4.0-beta1 (2024-04-16)

Bug Fixes

  • align kfp kinds and id handling (241aff4)
  • disable request logging filter to avoid breaking request handling (97c2e28)
  • fix auditor issue with null authentication on async threads (77954e9)
  • make oauth2 scopes configurable via env (eec40cd)

Features

  • add cron job execution infrastructure + cleanups (df983c1)
  • add handler to sourceCode in specs (3b566a2)
  • add user filters to search and services (f3e37f6)
  • add user to DTOs and audit user in entities + cleanups (92eaac9)
  • add validation instructions to k8s spec fields (b1d2ece)
  • core volume as class with enum for volumeTypes (1cde68f)
  • enable/disable solr for console via props (e0a96f3)
  • indexers and indexable services + update controllers (a1f5509)
  • update k8s core resource as (cpu/mem/gpu) (a1c8623)
  • update project controller to let owners modify/delete owned projects (1b507e0)

0.3.0-beta8 (2024-04-04)

Bug Fixes

  • fix blob handling in psql (5dabf59)

0.3.0-beta7 (2024-04-04)

Bug Fixes

  • fix errors in FSM and disable path traversal for more than 1-step state changes (4a30939)
  • include only non null elements in spec toMap to avoid inserting invalid values in kube + handle find null in store (b1db674)
  • typo in sql (9748058)

0.3.0-beta6 (2024-04-02)

Bug Fixes

  • cleanup security config + add auth endpoint for console (f36d53d)

0.3.0-beta5 (2024-04-02)

Bug Fixes

  • fix issues with jdbc initialization and refactor runnable store (fb51646)
  • fix postgresql schema sql naming (1adffb0)

Features

  • move actuator management endpoint to dedicated port + fix configuration issues (146c50b)
  • update specs and define tableSchema (b0c8bfa)

0.3.0-beta4 (2024-03-27)

0.3.0-beta3 (2024-03-26)

Bug Fixes

  • add deleting transaction to all run state (0b57c91)
  • align specs and correct implementations (c9463cc)
  • check for project existence before creating entities (c1a42dd)
  • check for secret existence before deleting (39e0964)
  • correct mlrun task job spec (18d7aa1)
  • correct wrongly configured pagination handler to start page count from 0 (ee542d1)
  • enforce unique secret for name+project (dd09cbe)
  • fix cloud broadcast to rabbit + make async (c28e46b)
  • fix image names in Dockerfile (9fa9319)
  • fix mapping for delete all dataItem in context controller (d9750c4)
  • fix pattern usage in runUtils (cae0de7)
  • fix runmanager wrong states and transactions (056a7d7)
  • include frontend in docker builds (83cd53f)
  • include frontend in docker cache (6bac7e9)
  • refactor converters implementations and store state enum as string in db (1b95480)
  • runnable in events are optional (5e47826)
  • update base controllers to properly extract search filters + update schema generation (f519e63)
  • update dtos and specs (9381802)
  • use localDateTime with iso format for metadata + cleanups (edcbd44)
  • use offsetDateTime with iso format for metadata (bb9495f)

Features

  • add LOG_LEVEL env variable to set logging (5c6c395)
  • align runtimes to run manager + fix and align k8s frameworks + add run actions RPC to context controller (5a47880)
  • async delete listener for runs (afcae53)
  • delete runs via manager and properly handle cleanups in runtimes (3d75f4b)
  • handle common exceptions in controllerAdvice (e904621)
  • integrate user console + fix request handling for application (1ccbe0f)
  • KFP remote implementation (#71) (e6eb6a2)
  • sourceCode in spec supports a list of languages (8c1db42)
  • update specs (BREAKING) (0429407)

0.0.31 (2024-03-08)

Bug Fixes

  • update security config to enable ROLE_ADMIN access from oidc (5269937)

Features

  • add store key to all dtos (1bd0b38)

0.0.30 (2024-03-04)

Bug Fixes

  • add pathvariable to schemaController.get (ea18b12)
  • align run usage with models + handle store exceptions (d6a36d4)
  • update run.sh script (9bf299e)

Features

  • banner + correct version number in app (3b335d8)
  • cronJob k8s support + cleanups (caa4278)

0.0.29 (2024-02-21)

Bug Fixes

  • add converter for entityName enum with value, supporting lowercase+ pluralization (e867eb2)
  • cache dockerfile (8c32692)
  • use conditional kube annotation for k8s stores (a571ebb)

Features

  • auth annotations with roles on controllers (2de6ca2)

0.0.28 (2024-02-21)

0.0.27 (2024-02-20)

Bug Fixes

  • add constructor to dataitem base spec (57d578d)
  • change maven image tag (a70a19c)
  • correct default db location (dcdb033)

Features

  • add runtime to specs + add runtime filtering to schemas via registry (a78173b)
  • add schema handling with registry + controllers + utils (c742079)
  • allow runs to override k8s specs defined in tasks (a7177de)
  • auto build images on tags (153f951)

0.0.26 (2024-02-12)

0.0.25 (2024-02-06)

0.0.24 (2024-02-06)

0.0.23 (2024-02-05)

0.0.22 (2024-01-25)

0.0.21 (2024-01-25)

0.0.20 (2024-01-25)

0.0.19 (2024-01-25)

0.0.18 (2024-01-24)

0.0.17 (2024-01-22)

0.0.16 (2024-01-22)

0.0.15 (2024-01-22)

0.0.14 (2024-01-22)

0.0.13 (2024-01-19)

0.0.12 (2024-01-18)

0.0.11 (2023-12-13)

0.0.10 (2023-12-12)

0.0.9 (2023-12-12)

0.0.8 (2023-12-11)

0.0.4 (2023-11-14)

0.0.3-snapshot (2023-09-19)

0.0.2-snapshot (2023-07-11)