Skip to content

Apache Airflow 2.1.0rc1

Pre-release
Pre-release
Compare
Choose a tag to compare
@ashb ashb released this 17 May 23:24
2.1.0rc1
This tag was signed with the committer’s verified signature.
ashb Ash Berlin-Taylor
GPG key ID: 807C731A8C82A095
Learn about vigilant mode.
b7c8b59

New Features
""""""""""""

  • Add PythonVirtualenvDecorator to Taskflow API (#14761)
  • Add Taskgroup decorator (#15034)
  • Create a DAG Calendar View (#15423)
  • Create cross-DAG dependencies view (#13199)
  • Add rest API to query for providers (#13394)
  • Mask passwords and sensitive info in task logs and UI (#15599)
  • Add SubprocessHook for running commands from operators (#13423)
  • Add DAG Timeout in UI page "DAG Details" (#14165)
  • Add WeekDayBranchOperator (#13997)
  • Add JSON linter to DAG Trigger UI (#13551)
  • Add DAG Description Doc to Trigger UI Page (#13365)
  • Add airflow webserver URL into SLA miss email. (#13249)
  • Add read only REST API endpoints for users (#14735)
  • Add files to generate Airflow's Python SDK (#14739)
  • Add dynamic fields to snowflake connection (#14724)
  • Add read only REST API endpoint for roles and permissions (#14664)
  • Add new datetime branch operator (#11964)
  • Add Google leveldb hook and operator (#13109) (#14105)
  • Add plugins endpoint to the REST API (#14280)
  • Add worker_pod_pending_timeout support (#15263)
  • Add support for labeling DAG edges (#15142)
  • Add CUD REST API endpoints for Roles (#14840)
  • Import connections from a file (#15177)
  • A bunch of template_fields_renderers additions (#15130)
  • Add REST API query sort and order to some endpoints (#14895)
  • Add timezone context in new ui (#15096)
  • Add query mutations to new UI (#15068)
  • Add different modes to sort dag files for parsing (#15046)
  • Auto refresh on Tree View (#15474)
  • BashOperator to raise AirflowSkipException on exit code 99 (by default, configurable) (#13421) (#14963)
  • Clear tasks by task ids in REST API (#14500)
  • Support jinja2 native Python types (#14603)
  • Allow celery workers without gossip or mingle modes (#13880)
  • Add airflow jobs check CLI command to check health of jobs (Scheduler etc) (#14519)
  • Rename DateTimeBranchOperator to BranchDateTimeOperator (#14720)

Improvements
""""""""""""

  • Add optional result handler callback to DbApiHook (#15581)
  • Update Flask App Builder limit to recently released 3.3 (#15792)
  • Prevent creating flask sessions on REST API requests (#15295)
  • Sync DAG specific permissions when parsing (#15311)
  • Increase maximum length of pool name on Tasks to 256 characters (#15203)
  • Enforce READ COMMITTED isolation when using mysql (#15714)
  • Auto-apply apply_default to subclasses of BaseOperator (#15667)
  • Emit error on duplicated DAG ID (#15302)
  • Update KubernetesExecutor pod templates to allow access to IAM permissions (#15669)
  • More verbose logs when running airflow db check-migrations (#15662)
  • When one_success mark task as failed if no success (#15467)
  • Add an option to trigger a dag w/o changing conf (#15591)
  • Add Airflow UI instance_name configuration option (#10162)
  • Add a decorator to retry functions with DB transactions (#14109)
  • Add return to PythonVirtualenvOperator's execute method (#14061)
  • Add verify_ssl config for kubernetes (#13516)
  • Add description about secret_key when Webserver > 1 (#15546)
  • Add Traceback in LogRecord in JSONFormatter (#15414)
  • Add support for arbitrary json in conn uri format (#15100)
  • Adds description field in variable (#12413) (#15194)
  • Add logs to show last modified in SFTP, FTP and Filesystem sensor (#15134)
  • Execute on_failure_callback when SIGTERM is received (#15172)
  • Allow hiding of all edges when highlighting states (#15281)
  • Display explicit error in case UID has no actual username (#15212)
  • Serve logs with Scheduler when using Local or Sequential Executor (#15557)
  • Deactivate trigger, refresh, and delete controls on dag detail view. (#14144)
  • Turn off autocomplete for connection forms (#15073)
  • Increase default worker_refresh_interval to 6000 seconds (#14970)
  • Only show User's local timezone if it's not UTC (#13904)
  • Suppress LOG/WARNING for a few tasks CLI for better CLI experience (#14567)
  • Configurable API response (CORS) headers (#13620)
  • Allow viewers to see all docs links (#14197)
  • Update Tree View date ticks (#14141)
  • Make the tooltip to Pause / Unpause a DAG clearer (#13642)
  • Warn about precedence of env var when getting variables (#13501)
  • Move [celery] default_queue config to [operators] default_queue to re-use between executors (#14699)

Bug Fixes
"""""""""

  • Fix 500 error from updateTaskInstancesState API endpoint when dry_run not passed (#15889)
  • Ensure that task preceding a PythonVirtualenvOperator doesn't fail (#15822)
  • Prevent mixed case env vars from crashing processes like worker (#14380)
  • Fixed type annotations in DAG decorator (#15778)
  • Fix on_failure_callback when task receive SIGKILL (#15537)
  • Fix dags table overflow (#15660)
  • Fix changing the parent dag state on subdag clear (#15562)
  • Fix reading from zip package to default to text (#13962)
  • Fix wrong parameter for drawDagStatsForDag in dags.html (#13884)
  • Fix QueuedLocalWorker crashing with EOFError (#13215)
  • Fix typo in NotPreviouslySkippedDep (#13933)
  • Fix parallelism after KubeExecutor pod adoption (#15555)
  • Fix kube client on mac with keepalive enabled (#15551)
  • Fixes wrong limit for dask for python>3.7 (should be <3.7) (#15545)
  • Fix Task Adoption in KubernetesExecutor (#14795)
  • Fix timeout when using XCom with KubernetesPodOperator (#15388)
  • Fix deprecated provider aliases in "extras" not working (#15465)
  • Fixed default XCom deserialization. (#14827)
  • Fix used_group_ids in dag.partial_subset (#13700) (#15308)
  • Further fix trimmed pod_id for KubernetesPodOperator (#15445)
  • Bugfix: Invalid name when trimmed pod_id ends with hyphen in KubernetesPodOperator (#15443)
  • Fix incorrect slots stats when TI pool_slots > 1 (#15426)
  • Fix DAG last run link (#15327)
  • Fix sync-perm to work correctly when update_fab_perms = False (#14847)
  • Fixes limits on Arrow for plexus test (#14781)
  • Fix UI bugs in tree view (#14566)
  • Fix AzureDataFactoryHook failing to instantiate its connection (#14565)
  • Fix permission error on non-POSIX filesystem (#13121)
  • Fix spelling in "ignorable" (#14348)
  • Fix get_context_data doctest import (#14288)
  • Correct typo in GCSObjectsWtihPrefixExistenceSensor (#14179)
  • Fix order of failed deps (#14036)
  • Fix critical CeleryKubernetesExecutor bug (#13247)
  • Fix four bugs in StackdriverTaskHandler (#13784)
  • func.sum may return Decimal that break rest APIs (#15585)
  • Persist tags params in pagination (#15411)
  • API: Raise AlreadyExists exception when the execution_date is same (#15174)
  • Remove duplicate call to sync_metadata inside DagFileProcessorManager (#15121)
  • Extra docker-py update to resolve docker op issues (#15731)
  • Ensure executors end method is called (#14085)
  • Remove user_id from API schema (#15117)
  • Prevent clickable bad links on disabled pagination (#15074)
  • Acquire lock on db for the time of migration (#10151)
  • Skip SLA check only if SLA is None (#14064)
  • Print right version in airflow info command (#14560)
  • Make airflow info work with pipes (#14528)
  • Rework client-side script for connection form. (#14052)
  • API: Add CollectionInfo in all Collections that have total_entries (#14366)
  • Fix task_instance_mutation_hook when importing airflow.models.dagrun (#15851)

Doc only changes
""""""""""""""""

  • Fix docstring of SqlSensor (#15466)
  • Small changes on "DAGs and Tasks documentation" (#14853)
  • Add note on changes to configuration options (#15696)
  • Add docs to the markdownlint and yamllint config files (#15682)
  • Rename old "Experimental" API to deprecated in the docs. (#15653)
  • Fix documentation error in git_sync_template.yaml (#13197)
  • Fix doc link permission name (#14972)
  • Fix link to Helm chart docs (#14652)
  • Fix docstrings for Kubernetes code (#14605)
  • docs: Capitalize & minor fixes (#14283) (#14534)
  • Fixed reading from zip package to default to text. (#13984)
  • An initial rework of the "Concepts" docs (#15444)
  • Improve docstrings for various modules (#15047)
  • Add documentation on database connection URI (#14124)
  • Add Helm Chart logo to docs index (#14762)
  • Create a new documentation package for Helm Chart (#14643)
  • Add docs about supported logging levels (#14507)
  • Update docs about tableau and salesforce provider (#14495)
  • Replace deprecated doc links to the correct one (#14429)
  • Refactor redundant doc url logic to use utility (#14080)
  • docs: NOTICE: Updated 2016-2019 to 2016-now (#14248)
  • Skip DAG perm sync during parsing if possible (#15464)
  • Add picture and examples for Edge Labels (#15310)
  • Add example DAG & how-to guide for sqlite (#13196)
  • Add links to new modules for deprecated modules (#15316)
  • Add note in Updating.md about FAB data model change (#14478)

Misc/Internal
"""""""""""""

  • Fix logging.exception redundancy (#14823)
  • Bump stylelint to remove vulnerable sub-dependency (#15784)
  • Add resolution to force dependencies to use patched version of lodash (#15777)
  • Update croniter to 1.0.x series (#15769)
  • Get rid of Airflow 1.10 in Breeze (#15712)
  • Run helm chart tests in parallel (#15706)
  • Bump ssri from 6.0.1 to 6.0.2 in /airflow/www (#15437)
  • Remove the limit on Gunicorn dependency (#15611)
  • Better "dependency already registered" warning message for tasks #14613 (#14860)
  • Pin pandas-gbq to <0.15.0 (#15114)
  • Use Pip 21.* to install airflow officially (#15513)
  • Bump mysqlclient to support the 1.4.x and 2.x series (#14978)
  • Finish refactor of DAG resource name helper (#15511)
  • Refactor/Cleanup Presentation of Graph Task and Path Highlighting (#15257)
  • Standardize default fab perms (#14946)
  • Remove datepicker for task instance detail view (#15284)
  • Turn provider's import warnings into debug logs (#14903)
  • Remove left-over fields from required in provider_info schema. (#14119)
  • Deprecate tableau extra (#13595)
  • Use built-in cached_property on Python 3.8 where possible (#14606)
  • Clean-up JS code in UI templates (#14019)
  • Bump elliptic from 6.5.3 to 6.5.4 in /airflow/www (#14668)
  • Switch to f-strings using flynt. (#13732)
  • use jquery ready instead of vanilla js (#15258)
  • Migrate task instance log (ti_log) js (#15309)
  • Migrate graph js (#15307)
  • Migrate dags.html javascript (#14692)
  • Removes unnecessary AzureContainerInstance connection type (#15514)
  • Separate Kubernetes pod_launcher from core airflow (#15165)
  • update remaining old import paths of operators (#15127)
  • Remove broken and undocumented "demo mode" feature (#14601)
  • Simplify configuration/legibility of Webpack entries (#14551)
  • remove inline tree js (#14552)
  • Js linting and inline migration for simple scripts (#14215)
  • Remove use of repeated constant in AirflowConfigParser (#14023)
  • Deprecate email credentials from environment variables. (#13601)
  • Remove unused 'context' variable in task_instance.py (#14049)
  • Disable suppress_logs_and_warning in cli when debugging (#13180)
Assets 2
Loading