This repository has been archived by the owner on Apr 8, 2021. It is now read-only.
forked from apache/airflow
-
Notifications
You must be signed in to change notification settings - Fork 2
/
CHANGELOG.txt
1126 lines (1113 loc) · 58.7 KB
/
CHANGELOG.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
AIRFLOW 1.9.0, 2018-01-02
-------------------------
[AIRFLOW-1525] Fix minor LICENSE and NOTICE issues
[AIRFLOW-XXX] Bump version to 1.9.0
[AIRFLOW-1897][AIRFLOW-1873] Task Logs for running instance not visible in WebUI
[AIRFLOW-XXX] Make sure session is committed
[AIRFLOW-1896] FIX bleach <> html5lib incompatibility
[AIRFLOW-XXX] Fix log handler test
[AIRFLOW-1873] Set TI.try_number to right value depending TI state
[AIRFLOW-1554] Fix wrong DagFileProcessor termination method call
[AIRFLOW-1872] Set context for all handlers including parents
[AIRFLOW-XXX] Add dask lock files to excludes
[AIRFLOW-1839] Fix more bugs in S3Hook boto -> boto3 migration
[AIRFLOW-1795] Correctly call S3Hook after migration to boto3
[AIRFLOW-1813] Bug SSH Operator empty buffer
[AIRFLOW-1794] Remove uses of Exception.message for Python 3
[AIRFLOW-1799] Fix logging line which raises errors
[AIRFLOW-1102] Upgrade Gunicorn >=19.4.0
[AIRFLOW-1756] Fix S3TaskHandler to work with Boto3-based S3Hook
[AIRFLOW-1797] S3Hook.load_string didn't work on Python3
[AIRFLOW-1792] Missing intervals DruidOperator
[AIRFLOW-1789][AIRFLOW-1712] Log SSHOperator stderr to log.warning
[AIRFLOW-1669] Fix Docker and pin Moto to 1.1.19
[AIRFLOW-71] Add support for private Docker images
[AIRFLOW-1779] Add keepalive packets to ssh hook
[AIRFLOW-XXX] Give a clue what the 'ds' variable is
[AIRFLOW-XXX] Correct typos in the faq docs page
[AIRFLOW-1571] Add AWS Lambda Hook
[AIRFLOW-1675] Fix docstrings for API docs
[AIRFLOW-1712][AIRFLOW-756][AIRFLOW-751] Log SSHOperator output
[AIRFLOW-1776] Capture stdout and stderr for logging
[AIRFLOW-1765] Make experimental API securable without needing Kerberos.
[AIRFLOW-1764] The web interface should not use the experimental API
[AIRFLOW-1634] Adds task_concurrency feature
[AIRFLOW-1018] Make processor use logging framework
[AIRFLOW-1695] Add RedshiftHook using boto3
[AIRFLOW-1706] Fix query error for MSSQL backend
[AIRFLOW-1711] Use ldap3 dict for group membership
[AIRFLOW-1757] Add missing options to SparkSubmitOperator
[AIRFLOW-1734][Airflow 1734] Sqoop hook/operator enhancements
[AIRFLOW-1731] Set pythonpath for logging
[AIRFLOW-1641] Handle executor events in the scheduler
[AIRFLOW-1744] Make sure max_tries can be set
[AIRFLOW-1330] Add conn_type argument to CLI when adding connection
[AIRFLOW-926] Fix JDBC Hook
[AIRFLOW-1520] Boto3 S3Hook, S3Log
[AIRFLOW-XXX] Fix DateTime in Tree View
[AIRFLOW-1432] Charts label for Y axis not visible
[AIRFLOW-1743] Verify ldap filters correctly
[AIRFLOW-1745] Restore default signal disposition
[AIRFLOW-1741] Correctly hide second chart on task duration page
[AIRFLOW-1726] Add copy_expert psycopg2 method to PostgresHook
[AIRFLOW-1698] Remove SCHEDULER_RUNS env var in systemd
[AIRFLOW-1694] Stop using itertools.izip
[AIRFLOW-1692] Change test_views filename to support Windows
[AIRFLOW-1722] Fix typo in scheduler autorestart output filename
[AIRFLOW-1691] Add better Google cloud logging documentation
[AIRFLOW-1690] Add detail to gcs error messages
[AIRFLOW-1682] Make S3TaskHandler write to S3 on close
[AIRFLOW-1676] Make GCSTaskHandler write to GCS on close
[AIRFLOW-1635] Allow creating GCP connection without requiring a JSON file
[AIRFLOW-1323] Made Dataproc operator parameter names consistent
[AIRFLOW-1590] fix unused module and variable
[AIRFLOW-988] Fix repeating SLA miss callbacks
[AIRFLOW-1611] Customize logging
[AIRFLOW-1668] Expose keepalives_idle for Postgres connections
[AIRFLOW-1658] Kill Druid task on timeout
[AIRFLOW-1669][AIRFLOW-1368] Fix Docker import
[AIRFLOW-1560] Add AWS DynamoDB hook and operator for inserting batch items
[AIRFLOW-1654] Show tooltips for link icons in DAGs view
[AIRFLOW-1660] Change webpage width to full-width
[AIRFLOW-1664] write file as binary instead of str
[AIRFLOW-1659] Fix invalid obj attribute bug in file_task_handler.py
[AIRFLOW-1650] Fix custom celery config loading
[AIRFLOW-1647] Fix Spark-sql hook
[AIRFLOW-1587] Fix CeleryExecutor import error
[AIRFLOW-1636] Add AWS and EMR connection type
[AIRFLOW-1527] Refactor celery config
[AIRFLOW-1639] Fix Fernet error handling
[AIRFLOW-1628] Fix docstring of sqlsensor
[AIRFLOW-1331] add SparkSubmitOperator option
[AIRFLOW-1627] Only query pool in SubDAG init when necessary
[AIRFLOW-1629] Make extra a textarea in edit connections form
[AIRFLOW-1621] Add tests for server side paging
[AIRFLOW-1519] Add server side paging in DAGs list
[AIRFLOW-289] Make airflow timezone independent
[AIRFLOW-1356] Add `--celery_hostname` to `airflow worker`
[AIRFLOW-1591] Avoid attribute error when rendering logging filename
[AIRFLOW-1031] Replace hard-code to DagRun.ID_PREFIX
[AIRFLOW-1604] Rename logger to log
[AIRFLOW-1512] Add PythonVirtualenvOperator
[AIRFLOW-1617] Fix XSS vulnerability in Variable endpoint
[AIRFLOW-1497] Reset hidden fields when changing connection type
[AIRFLOW-1177] Fix Variable.setdefault w/existing JSON
[AIRFLOW-1600] Fix exception handling in get_fernet
[AIRFLOW-1614] Replace inspect.stack() with sys._getframe()
[AIRFLOW-1613] Make MySqlToGoogleCloudStorageOperator compaitible with python3
[AIRFLOW-1609] Fix gitignore to ignore all venvs
[AIRFLOW-1601] Add configurable task cleanup time
[AIRFLOW-XXX] Bumping Airflow 1.9.0alpha0 version
[AIRFLOW-1608] Handle pending job state in GCP Dataflow hook
[AIRFLOW-1606] Use non static DAG.sync_to_db
[AIRFLOW-1606][Airflow-1606][AIRFLOW-1605][AIRFLOW-160] DAG.sync_to_db is now a normal method
[AIRFLOW-1602] LoggingMixin in DAG class
[AIRFLOW-1593] expose load_string in WasbHook
[AIRFLOW-1597] Add GameWisp as Airflow user
[AIRFLOW-1594] Don't install test packages into python root.[]
[AIRFLOW-1582] Improve logging within Airflow
[AIRFLOW-1476] add INSTALL instruction for source releases
[AIRFLOW-XXX] Save username and password in airflow-pr
[AIRFLOW-1522] Increase text size for var field in variables for MySQL
[AIRFLOW-950] Missing AWS integrations on documentation::integrations
[AIRFLOW-XXX] 1.8.2 release notes
[AIRFLOW-1573] Remove `thrift < 0.10.0` requirement
[AIRFLOW-1584] Remove insecure /headers endpoint
[AIRFLOW-1586] Add mapping for date type to mysql_to_gcs operator
[AIRFLOW-1579] Adds support for jagged rows in Bigquery hook for BQ load jobs
[AIRFLOW-1577] Add token support to DatabricksHook
[AIRFLOW-1580] Error in string formating
[AIRFLOW-1567] Updated docs for Google ML Engine operators/hooks
[AIRFLOW-1574] add 'to' attribute to templated vars of email operator
[AIRFLOW-1572] add carbonite to company list
[AIRFLOW-1568] Fix typo in BigQueryHook
[AIRFLOW-1493][AIRFLOW-XXXX][WIP] fixed dumb thing
[AIRFLOW-1567][Airflow-1567] Renamed cloudml hook and operator to mlengine
[AIRFLOW-1568] Add datastore export/import operators
[AIRFLOW-1564] Use Jinja2 to render logging filename
[AIRFLOW-1562] Spark-sql logging contains deadlock
[AIRFLOW-1556][Airflow 1556] Add support for SQL parameters in BigQueryBaseCursor
[AIRFLOW-108] Add CreditCards.com to companies list
[AIRFLOW-1541] Add channel to template fields of slack_operator
[AIRFLOW-1535] Add service account/scopes in dataproc
[AIRFLOW-1384] Add to README.md CaDC/ARGO
[AIRFLOW-1546] add Zymergen 80to org list in README
[AIRFLOW-1545] Add Nextdoor to companies list
[AIRFLOW-1544] Add DataFox to companies list
[AIRFLOW-1529] Add logic supporting quoted newlines in Google BigQuery load jobs
[AIRFLOW-1521] Fix emplate rendering for BigqueryTableDeleteOperator
[AIRFLOW-1324] Generalize Druid operator and hook
[AIRFLOW-1516] Fix error handling getting fernet
[AIRFLOW-1420][AIRFLOW-1473] Fix deadlock check
[AIRFLOW-1495] Fix migration on index on job_id
[AIRFLOW-1483] Making page size consistent in list
[AIRFLOW-1495] Add TaskInstance index on job_id
[AIRFLOW-855] Replace PickleType with LargeBinary in XCom
[AIRFLOW-1505] Document when Jinja substitution occurs
[AIRFLOW-1504] Log dataproc cluster name
[AIRFLOW-1239] Fix unicode error for logs in base_task_runner
[AIRFLOW-1280] Fix Gantt chart height
[AIRFLOW-1507] Template parameters in file_to_gcs operator
[AIRFLOW-1452] workaround lock on method
[AIRFLOW-1385] Make Airflow task logging configurable
[AIRFLOW-940] Handle error on variable decrypt
[AIRFLOW-1492] Add gauge for task successes/failures
[AIRFLOW-1443] Update Airflow configuration documentation
[AIRFLOW-1486] Unexpected S3 writing log error
[AIRFLOW-1487] Added links to all companies officially using Airflow
[AIRFLOW-1489] Fix typo in BigQueryCheckOperator
[AIRFLOW-1349] Fix backfill to respect limits
[AIRFLOW-1478] Chart owner column should be sortable
[AIRFLOW-1397][AIRFLOW-1] No Last Run column data displyed in Airflow UI 1.8.1
[AIRFLOW-1474] Add dag_id regex feature for `airflow clear` command
[AIRFLOW-1445] Changing HivePartitionSensor UI color to lighter shade
[AIRFLOW-1359] Use default_args in Cloud ML eval
[AIRFLOW-1389] Support createDisposition in BigQueryOperator
[AIRFLOW-1349] Refactor BackfillJob _execute
[AIRFLOW-1459] Fixed broken integration .rst formatting
[AIRFLOW-1448] Revert "Fix cli reading logfile in memory"
[AIRFLOW-1398] Allow ExternalTaskSensor to wait on multiple runs of a task
[AIRFLOW-1399] Fix cli reading logfile in memory
[AIRFLOW-1442] Remove extra space from ignore_all_deps generated command
[AIRFLOW-1438] Change batch size per query in scheduler
[AIRFLOW-1439] Add max billing tier for the BQ Hook and Operator
[AIRFLOW-1437] Modify BigQueryTableDeleteOperator
[Airflow 1332] Split logs based on try number
[AIRFLOW-1385] Create abstraction for Airflow task logging
[AIRFLOW-756][AIRFLOW-751] Replace ssh hook, operator & sftp operator with paramiko based
[AIRFLOW-1393][[AIRFLOW-1393] Enable Py3 tests in contrib/spark_submit_hook[
[AIRFLOW-1345] Dont expire TIs on each scheduler loop
[AIRFLOW-1059] Reset orphaned tasks in batch for scheduler
[AIRFLOW-1255] Fix SparkSubmitHook output deadlock
[AIRFLOW-1359] Add Google CloudML utils for model evaluation
[AIRFLOW-1247] Fix ignore all dependencies argument ignored
[AIRFLOW-1401] Standardize cloud ml operator arguments
[AIRFLOW-1394] Add quote_character param to GCS hook and operator
[AIRFLOW-1402] Cleanup SafeConfigParser DeprecationWarning
[AIRFLOW-1326][[AIRFLOW-1326][AIRFLOW-1184] Don't split argument array -- it's already an array.[
[AIRFLOW-1384] Add ARGO/CaDC as a Airflow user
[AIRFLOW-1357] Fix scheduler zip file support
[AIRFLOW-1382] Add working dir option to DockerOperator
[AIRFLOW-1388] Add Cloud ML Engine operators to integration doc
[AIRFLOW-1387] Add unicode string prefix
[AIRFLOW-1366] Add max_tries to task instance
[AIRFLOW-1300] Enable table creation with TBLPROPERTIES
[AIRFLOW-1271] Add Google CloudML Training Operator
[AIRFLOW-300] Add Google Pubsub hook and operator
[AIRFLOW-1343] Fix dataproc label format
[AIRFLOW-1367] Pass Content-ID To reference inline images in an email, we need to be able to add <img src="cid:{}"/> to the HTML. However currently the Content-ID (cid) is not passed, so we need to add it
[AIRFLOW-1265] Fix celery executor parsing CELERY_SSL_ACTIVE
[AIRFLOW-1272] Google Cloud ML Batch Prediction Operator
[AIRFLOW-1352][AIRFLOW-1335] Revert MemoryHandler change ()[]
[AIRFLOW-1350] Add query_uri param to Hive/SparkSQL DataProc operator
[AIRFLOW-1334] Check if tasks are backfill on scheduler in a join
[AIRFLOW-1343] Add Airflow default label to the dataproc operator
[AIRFLOW-1273] Add Google Cloud ML version and model operators
[AIRFLOW-1273]AIRFLOW-1273] Add Google Cloud ML version and model operators
[AIRFLOW-1321] Fix hidden field key to ignore case
[AIRFLOW-1337] Make log_format key names lowercase
[AIRFLOW-1338][AIRFLOW-782] Add GCP dataflow hook runner change to UPDATING.md
[AIRFLOW-801] Remove outdated docstring on BaseOperator
[AIRFLOW-1344] Fix text encoding bug when reading logs for Python 3.5
[AIRFLOW-1338] Fix incompatible GCP dataflow hook
[AIRFLOW-1333] Enable copy function for Google Cloud Storage Hook
[AIRFLOW-1337] Allow log format customization via airflow.cfg
[AIRFLOW-1320] Update LetsBonus users in README
[AIRFLOW-1335] Use MemoryHandler for buffered logging
[AIRFLOW-1339] Add Drivy to the list of users
[AIRFLOW-1275] Put 'airflow pool' into API
[AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
[AIRFLOW-1317] Fix minor issues in API reference
[AIRFLOW-1308] Disable nanny usage for Dask
[AIRFLOW-1172] Support nth weekday of the month cron expression
[AIRFLOW-936] Add clear/mark success for DAG in the UI
[AIRFLOW-1294] Backfills can loose tasks to execute
[AIRFLOW-1299] Support imageVersion in Google Dataproc cluster
[AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF requirements
[AIRFLOW-1301] Add New Relic to list of companies
[AIRFLOW-1289] Removes restriction on number of scheduler threads
[AIRFLOW-1024] Ignore celery executor errors (#49)
[AIRFLOW-1265] Fix exception while loading celery configurations
[AIRFLOW-1290] set docs author to 'Apache Airflow'
[AIRFLOW-1242] Allowing project_id to have a colon in it.
[AIRFLOW-1282] Fix known event column sorting
[AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun
[AIRFLOW-1208] Speed-up cli tests
[AIRFLOW-1192] Some enhancements to qubole_operator
[AIRFLOW-1281] Sort variables by key field by default
[AIRFLOW-1277] Forbid KE creation with empty fields
[AIRFLOW-1276] Forbid event creation with end_data earlier than start_date
[AIRFLOW-1263] Dynamic height for charts
[AIRFLOW-1266] Increase width of gantt y axis
[AIRFLOW-1244] Forbid creation of a pool with empty name
[AIRFLOW-1274][HTTPSENSOR] Rename parameter params to data
[AIRFLOW-654] Add SSL Config Option for CeleryExecutor w/ RabbitMQ - Add BROKER_USE_SSL config to give option to send AMQP messages over SSL - Can be set using usual airflow options (e.g. airflow.cfg, env vars, etc.)
[AIRFLOW-1256] Add United Airlines to readme
[AIRFLOW-1251] Add eRevalue to Airflow users
[AIRFLOW-908] Print hostname at the start of cli run
[AIRFLOW-1237] Fix IN-predicate sqlalchemy warning
[AIRFLOW-1243] DAGs table has no default entries to show
[AIRFLOW-1245] Fix random failure in test_trigger_dag_for_date
[AIRFLOW-1248] Fix wrong conf name for worker timeout
[AIRFLOW-1197] : SparkSubmitHook on_kill error
[AIRFLOW-1191] : SparkSubmitHook custom cmd
[AIRFLOW-1234] Cover utils.operator_helpers with UTs
[AIRFLOW-1217] Enable Sqoop logging
[AIRFLOW-645] Support HTTPS connections in HttpHook
[AIRFLOW-1231] Use flask_wtf.CSRFProtect
[AIRFLOW-1232] Remove deprecated readfp warning
[AIRFLOW-1233] Cover utils.json with unit tests
[AIRFLOW-1227] Remove empty column on the Logs view
[AIRFLOW-1226] Remove empty column on the Jobs view
[AIRFLOW-1221] Fix templating bug with DatabricksSubmitRunOperator
[AIRFLOW-1210] Enable DbApiHook unit tests
[AIRFLOW-1199] Fix create modal
[AIRFLOW-1200] Forbid creation of a variable with an empty key
[AIRFLOW-1207] Enable utils.helpers unit tests
[AIRFLOW-1213] Add hcatalog parameters to sqoop
[AIRFLOW-1201] Update deprecated 'nose-parameterized'
[AIRFLOW-1186] Sort dag.get_task_instances by execution_date
[AIRFLOW-1203] Pin Google API client version to fix OAuth issue
[AIRFLOW-1145] Fix closest_date_partition function with before set to True If we're looking for the closest date before, we should take the latest date in the list of date before.
[AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection
[AIRFLOW-993] Update date inference logic
[AIRFLOW-1170] DbApiHook insert_rows inserts parameters separately
[AIRFLOW-1041] Do not shadow xcom_push method[]
[AIRFLOW-860][AIRFLOW-935] Fix plugin executor import cycle and executor selection
[AIRFLOW-1189] Fix get a DataFrame using BigQueryHook failing
[AIRFLOW-1184] SparkSubmitHook does not split args
[AIRFLOW-1182] SparkSubmitOperator template field
[AIRFLOW-823] Allow specifying execution date in task_info API
[AIRFLOW-1175] Add Pronto Tools to Airflow user list
[AIRFLOW-1150] Fix scripts execution in sparksql hook[]
[AIRFLOW-1141] remove crawl_for_tasks
[AIRFLOW-1193] Add Checkr to company using Airflow
[AIRFLOW-1168] Add closing() to all connections and cursors
[AIRFLOW-1188] Add max_bad_records param to GoogleCloudStorageToBigQueryOperator
[AIRFLOW-1187][AIRFLOW-1185] Fix PyPi package names in documents
[AIRFLOW-1185] Fix PyPi URL in templates
[AIRFLOW-XXX] Updating CHANGELOG, README, and UPDATING after 1.8.1 release
[AIRFLOW-1181] Add delete and list functionality to gcs_hook
[AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change
[AIRFLOW-1167] Support microseconds in FTPHook modification time
[AIRFLOW-1173] Add Robinhood to who uses Airflow
[AIRFLOW-945][AIRFLOW-941] Remove psycopg2 connection workaround
[AIRFLOW-1140] DatabricksSubmitRunOperator should template the "json" field.
[AIRFLOW-1160] Update Spark parameters for Mesos
[AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in Jinja2 templates
[AIRFLOW-1036] Randomize exponential backoff
[AIRFLOW-1155] Add Tails.com to community
[AIRFLOW-1142] Do not reset orphaned state for backfills
[AIRFLOW-492] Make sure stat updates cannot fail a task
[AIRFLOW-1119] Fix unload query so headers are on first row[]
[AIRFLOW-1089] Add Spark application arguments
[AIRFLOW-1125] Document encrypted connections
[AIRFLOW-1122] Increase stroke width in UI
[AIRFLOW-1138] Add missing licenses to files in scripts directory
(AIRFLOW-11-38) [AIRFLOW-1136] Capture invalid arguments for Sqoop
[AIRFLOW-1127] Move license notices to LICENSE
[AIRFLOW-1118] Add evo.company to Airflow users
[AIRFLOW-1121][AIRFLOW-1004] Fix `airflow webserver --pid` to write out pid file
[AIRFLOW-1124] Do not set all tasks to scheduled in backfill
[AIRFLOW-1120] Update version view to include Apache prefix
[AIRFLOW-1091] Add script that can compare jira target against merges
[AIRFLOW-1107] Add support for ftps non-default port
[AIRFLOW-1000] Rebrand distribution to Apache Airflow
[AIRFLOW-1094] Run unit tests under contrib in Travis
[AIRFLOW-1112] Log which pool when pool is full in scheduler
[AIRFLOW-1106] Add Groupalia/Letsbonus to the ReadMe
[AIRFLOW-1109] Use kill signal to kill processes and log results
[AIRFLOW-1074] Don't count queued tasks for concurrency limits
[AIRFLOW-1095] Make ldap_auth memberOf come from configuration
[AIRFLOW-1090] Add HBO
[AIRFLOW-1035] Use binary exponential backoff
[AIRFLOW-1081] Improve performance of duration chart
[AIRFLOW-1078] Fix latest_runs endpoint for old flask versions
[AIRFLOW-1085] Enhance the SparkSubmitOperator
[AIRFLOW-1050] Do not count up_for_retry as not ready
[AIRFLOW-1028] Databricks Operator for Airflow
[AIRFLOW-1075] Security docs cleanup
[AIRFLOW-1033][AIFRLOW-1033] Fix ti_deps for no schedule dags
[AIRFLOW-1016] Allow HTTP HEAD request method on HTTPSensor
[AIRFLOW-970] Load latest_runs on homepage async
[AIRFLOW-111] Include queued tasks in scheduler concurrency check
[AIRFLOW-1001] Fix landing times if there is no following schedule
[AIRFLOW-1065] Add functionality for Azure Blob Storage over wasb://
[AIRFLOW-947] Improve exceptions for unavailable Presto cluster
[AIRFLOW-1067] use example.com in examples
[AIRFLOW-1064] Change default sort to job_id for TaskInstanceModelView
[AIRFLOW-1030][AIRFLOW-1] Fix hook import for HttpSensor
[AIRFLOW-1051] Add a test for resetdb to CliTests
[AIRFLOW-1004][AIRFLOW-276] Fix `airflow webserver -D` to run in background
[AIRFLOW-1062] Fix DagRun#find to return correct result
[AIRFLOW-1011] Fix bug in BackfillJob._execute() for SubDAGs
[AIRFLOW-1038] Specify celery serialization options explicitly
[AIRFLOW-1054] Fix broken import in test_dag
[AIRFLOW-1007] Use Jinja sandbox for chart_data endpoint
[AIRFLOW-719] Fix race condition in ShortCircuit, Branch and LatestOnly
[AIRFLOW-1043] Fix doc strings of operators
[AIRFLOW-840] Make ticket renewer python3 compatible
[AIRFLOW-985] Extend the sqoop operator and hook
[AIRFLOW-1034] Make it possible to connect to S3 in sigv4 regions
[AIRFLOW-1045] Make log level configurable via airflow.cfg
[AIRFLOW-1047] Sanitize strings passed to Markup
[AIRFLOW-1040] Fix some small typos in comments and docstrings
[AIRFLOW-1017] get_task_instance shouldn't throw exception when no TI
[AIRFLOW-1006] Add config_templates to MANIFEST
[AIRFLOW-999] Add support for Redis database
[AIRFLOW-1009] Remove SQLOperator from Concepts page
[AIRFLOW-1006] Move config templates to separate files
[AIRFLOW-1005] Improve Airflow startup time
[AIRFLOW-1010] Add convenience script for signing releases
[AIRFLOW-995] Remove reference to actual Airflow issue
[AIRFLOW-681] homepage doc link should pointing to apache repo not airbnb repo
[AIRFLOW-705][AIRFLOW-706] Fix run_command bugs
[AIRFLOW-990] Fix Py27 unicode logging in DockerOperator
[AIRFLOW-963] Fix non-rendered code examples
[AIRFLOW-969] Catch bad python_callable argument
[AIRFLOW-984] Enable subclassing of SubDagOperator
[AIRFLOW-997] Update setup.cfg to point to Apache
[AIRFLOW-994] Add MiNODES to the official airflow user list
[AIRFLOW-995][AIRFLOW-1] Update GitHub PR Template
[AIRFLOW-989] Do not mark dag run successful if unfinished tasks
[AIRFLOW-903] New configuration setting for the default dag view
[AIRFLOW-979] Add GovTech GDS
[AIRFLOW-933] Replace eval with literal_eval to prevent RCE
[AIRFLOW-974] Fix mkdirs race condition
[AIRFLOW-917] Fix formatting of error message
[AIRFLOW-770] Refactor BaseHook so env vars are always read
[AIRFLOW-900] Double trigger should not kill original task instance
[AIRFLOW-900] Fixes bugs in LocalTaskJob for double run protection
[AIRFLOW-932][AIRFLOW-932][AIRFLOW-921][AIRFLOW-910] Do not mark tasks removed when backfilling[
[AIRFLOW-961] run onkill when SIGTERMed
[AIRFLOW-910] Use parallel task execution for backfills
[AIRFLOW-967] Wrap strings in native for py2 ldap compatibility
[AIRFLOW-958] Improve tooltip readability
AIRFLOW-959 Cleanup and reorganize .gitignore
AIRFLOW-960 Add .editorconfig file
[AIRFLOW-931] Do not set QUEUED in TaskInstances
[AIRFLOW-956] Get docs working on readthedocs.org
[AIRFLOW-954] Fix configparser ImportError
[AIRFLOW-941] Use defined parameters for psycopg2
[AIRFLOW-943] Update Digital First Media in users list
[AIRFLOW-942] Add mytaxi to Airflow users
[AIRFLOW-939] add .swp to gitginore
[AIRFLOW-719] Prevent DAGs from ending prematurely
[AIRFLOW-938] Use test for True in task_stats queries
[AIRFLOW-937] Improve performance of task_stats
[AIRFLOW-933] use ast.literal_eval rather eval because ast.literal_eval does not execute input.
[AIRFLOW-925] Revert airflow.hooks change that cherry-pick picked
[AIRFLOW-919] Running tasks with no start date shouldn't break a DAGs UI
[AIRFLOW-802][AIRFLOW-1] Add spark-submit operator/hook
[AIRFLOW-725] Use keyring to store credentials for JIRA
[AIRFLOW-916] Remove deprecated readfp function
[AIRFLOW-911] Add coloring and timing to tests
[AIRFLOW-906] Update Code icon from lightning bolt to file
[AIRFLOW-897] Prevent dagruns from failing with unfinished tasks
[AIRFLOW-896] Remove unicode to 8-bit conversion in BigQueryOperator
[AIRFLOW-899] Tasks in SCHEDULED state should be white in the UI instead of black
[AIRFLOW-895] Address Apache release incompliancies
[AIRFLOW-893][AIRFLOW-510] Fix crashing webservers when a dagrun has no start date
[AIRFLOW-880] Make webserver serve logs in a sane way for remote logs
[AIRFLOW-889] Fix minor error in the docstrings for BaseOperator
[AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When Testing Booleans
[AIRFLOW-875] Add template to HttpSensor params
[AIRFLOW-866] Add FTPSensor
[AIRFLOW-881] Check if SubDagOperator is in DAG context manager
[AIRFLOW-885] Add change.org to the users list
[AIRFLOW-836] Use POST and CSRF for state changing endpoints
[AIRFLOW-862] Fix Unit Tests for DaskExecutor
[AIRFLOW-887] Support future v0.16
[AIRFLOW-886] Pass result to post_execute() hook
[AIRFLOW-871] change logging.warn() into warning()
[AIRFLOW-882] Remove unnecessary dag>>op assignment in docs
[AIRFLOW-861] make pickle_info endpoint be login_required
[AIRFLOW-869] Refactor mark success functionality
[AIRFLOW-877] Remove .sql template extension from GCS download operator
[AIRFLOW-826] Add Zendesk hook
[AIRFLOW-842] do not query the DB with an empty IN clause
[AIRFLOW-834] change raise StopIteration into return
[AIRFLOW-832] Let debug server run without SSL
[AIRFLOW-862] Add DaskExecutor
[AIRFLOW-858] Configurable database name for DB operators
[AIRFLOW-863] Example DAGs should have recent start dates
[AIRFLOW-853] use utf8 encoding for stdout line decode
[AIRFLOW-857] Use library assert statements instead of conditionals
[AIRFLOW-856] Make sure execution date is set for local client
[AIRFLOW-854] Add OKI as Airflow user
[AIRFLOW-830][AIRFLOW-829][AIRFLOW-88] Reduce Travis log verbosity
[AIRFLOW-814] Fix Presto*CheckOperator.__init__
[AIRFLOW-793] Enable compressed loading in S3ToHiveTransfer
[AIRFLOW-844] Fix cgroups directory creation
[AIRFLOW-831] Restore import to fix broken tests
[AIRFLOW-794] Access DAGS_FOLDER and SQL_ALCHEMY_CONN exclusively from settings
[AIRFLOW-694] Fix config behaviour for empty envvar
[AIRFLOW-365] Set dag.fileloc explicitly and use for Code view
[AIRFLOW-781] Allow DataFlowOperators to accept jobs stored in GCS
AIRFLOW 1.8.2, 2017-09-04
-------------------------
9a53e66 [AIRFLOW-809][AIRFLOW-1] Use __eq__ ColumnOperator When Testing Booleans
333e0b3 [AIRFLOW-1296] Propagate SKIPPED to all downstream tasks
93825d5 Re-enable caching for hadoop components
33a9dcb Pin Hive and Hadoop to a specific version and create writable warehouse dir
7cff6cd [AIRFLOW-1308] Disable nanny usage for Dask
c6a09c4 Updating CHANGELOG for 1.8.2rc1
570b2ed [AIRFLOW-1294] Backfills can loose tasks to execute
3f48d48 [AIRFLOW-1291] Update NOTICE and LICENSE files to match ASF requirements
e10af9a [AIRFLOW-XXX] Set version to 1.8.2rc1
69bd269 [AIRFLOW-1160] Update Spark parameters for Mesos
9692510 [AIRFLOW 1149][AIRFLOW-1149] Allow for custom filters in Jinja2 templates
6de5330 [AIRFLOW-1119] Fix unload query so headers are on first row[]
b4e9eb8 [AIRFLOW-1089] Add Spark application arguments
a4083f3 [AIRFLOW-1078] Fix latest_runs endpoint for old flask versions
7a02841 [AIRFLOW-1074] Don't count queued tasks for concurrency limits
a2c18a5 [AIRFLOW-1064] Change default sort to job_id for TaskInstanceModelView
d1c64ab [AIRFLOW-1038] Specify celery serialization options explicitly
b4ee88a [AIRFLOW-1036] Randomize exponential backoff
9fca409 [AIRFLOW-993] Update date inference logic
272c2f5 [AIRFLOW-1167] Support microseconds in FTPHook modification time
c7c0b72 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change
acd0166 [AIRFLOW-1263] Dynamic height for charts
7f33f6e [AIRFLOW-1266] Increase width of gantt y axis
fc33c04 [AIRFLOW-1290] set docs author to 'Apache Airflow'
2e9eee3 [AIRFLOW-1282] Fix known event column sorting
2389a8a [AIRFLOW-1166] Speed up _change_state_for_tis_without_dagrun
bf966e6 [AIRFLOW-1192] Some enhancements to qubole_operator
57d5bcd [AIRFLOW-1281] Sort variables by key field by default
802fc15 [AIRFLOW-1244] Forbid creation of a pool with empty name
1232b6a [AIRFLOW-1243] DAGs table has no default entries to show
b0ba3c9 [AIRFLOW-1227] Remove empty column on the Logs view
c406652 [AIRFLOW-1226] Remove empty column on the Jobs view
51a83cc [AIRFLOW-1199] Fix create modal
cac7d4c [AIRFLOW-1200] Forbid creation of a variable with an empty key
5f3ee52 [AIRFLOW-1186] Sort dag.get_task_instances by execution_date
f446c08 [AIRFLOW-1145] Fix closest_date_partition function with before set to True If we're looking for the closest date before, we should take the latest date in the list of date before.
93b8e96 [AIRFLOW-1180] Fix flask-wtf version for test_csrf_rejection
bb56805 [AIRFLOW-1170] DbApiHook insert_rows inserts parameters separately
093b2f0 [AIRFLOW-1150] Fix scripts execution in sparksql hook[]
777f181 [AIRFLOW-1168] Add closing() to all connections and cursors
bc8e912 [AIRFLOW-XXX] Updating CHANGELOG, README, and UPDATING after 1.8.1 release
AIRFLOW 1.8.1, 2017-05-09
-------------------------
[AIRFLOW-1142] SubDAG Tasks Not Executed Even Though All Dependencies Met
[AIRFLOW-1138] Add licenses to files in scripts directory
[AIRFLOW-1127] Move license notices to LICENSE instead of NOTICE
[AIRFLOW-1124] Do not set all task instances to scheduled on backfill
[AIRFLOW-1120] Update version view to include Apache prefix
[AIRFLOW-1062] DagRun#find returns wrong result if external_trigger=False is specified
[AIRFLOW-1054] Fix broken import on test_dag
[AIRFLOW-1050] Retries ignored - regression
[AIRFLOW-1033] TypeError: can't compare datetime.datetime to None
[AIRFLOW-1017] get_task_instance should return None instead of throw an exception for non-existent TIs
[AIRFLOW-1011] Fix bug in BackfillJob._execute() for SubDAGs
[AIRFLOW-1004] `airflow webserver -D` runs in foreground
[AIRFLOW-1001] Landing Time shows "unsupported operand type(s) for -: 'datetime.datetime' and 'NoneType'" on example_subdag_operator
[AIRFLOW-1000] Rebrand to Apache Airflow instead of Airflow
[AIRFLOW-989] Clear Task Regression
[AIRFLOW-974] airflow.util.file mkdir has a race condition
[AIRFLOW-906] Update Code icon from lightning bolt to file
[AIRFLOW-858] Configurable database name for DB operators
[AIRFLOW-853] ssh_execute_operator.py stdout decode default to ASCII
[AIRFLOW-832] Fix debug server
[AIRFLOW-817] Trigger dag fails when using CLI + API
[AIRFLOW-816] Make sure to pull nvd3 from local resources
[AIRFLOW-815] Add previous/next execution dates to available default variables.
[AIRFLOW-813] Fix unterminated unit tests in tests.job (tests/job.py)
[AIRFLOW-812] Scheduler job terminates when there is no dag file
[AIRFLOW-806] UI should properly ignore DAG doc when it is None
[AIRFLOW-794] Consistent access to DAGS_FOLDER and SQL_ALCHEMY_CONN
[AIRFLOW-785] ImportError if cgroupspy is not installed
[AIRFLOW-784] Cannot install with funcsigs > 1.0.0
[AIRFLOW-780] The UI no longer shows broken DAGs
[AIRFLOW-777] dag_is_running is initlialized to True instead of False
[AIRFLOW-719] Skipped operations make DAG finish prematurely
[AIRFLOW-694] Empty env vars do not overwrite non-empty config values
[AIRFLOW-492] Insert into dag_stats table results into failed task while task itself succeeded
[AIRFLOW-139] Executing VACUUM with PostgresOperator
[AIRFLOW-111] DAG concurrency is not honored
[AIRFLOW-88] Improve clarity Travis CI reports
AIRFLOW 1.8.0, 2017-03-12
-------------------------
[AIRFLOW-900] Double trigger should not kill original task instance
[AIRFLOW-900] Fixes bugs in LocalTaskJob for double run protection
[AIRFLOW-932] Do not mark tasks removed when backfilling
[AIRFLOW-961] run onkill when SIGTERMed
[AIRFLOW-910] Use parallel task execution for backfills
[AIRFLOW-967] Wrap strings in native for py2 ldap compatibility
[AIRFLOW-941] Use defined parameters for psycopg2
[AIRFLOW-719] Prevent DAGs from ending prematurely
[AIRFLOW-938] Use test for True in task_stats queries
[AIRFLOW-937] Improve performance of task_stats
[AIRFLOW-933] use ast.literal_eval rather eval because ast.literal_eval does not execute input.
[AIRFLOW-925] Revert airflow.hooks change that cherry-pick picked
[AIRFLOW-919] Running tasks with no start date shouldn't break a DAGs UI
[AIRFLOW-802] Add spark-submit operator/hook
[AIRFLOW-897] Prevent dagruns from failing with unfinished tasks
[AIRFLOW-861] make pickle_info endpoint be login_required
[AIRFLOW-853] use utf8 encoding for stdout line decode
[AIRFLOW-856] Make sure execution date is set for local client
[AIRFLOW-830][AIRFLOW-829][AIRFLOW-88] Reduce Travis log verbosity
[AIRFLOW-831] Restore import to fix broken tests
[AIRFLOW-794] Access DAGS_FOLDER and SQL_ALCHEMY_CONN exclusively from settings
[AIRFLOW-694] Fix config behaviour for empty envvar
[AIRFLOW-365] Set dag.fileloc explicitly and use for Code view
[AIRFLOW-931] Do not set QUEUED in TaskInstances
[AIRFLOW-899] Tasks in SCHEDULED state should be white in the UI instead of black
[AIRFLOW-895] Address Apache release incompliancies
[AIRFLOW-893][AIRFLOW-510] Fix crashing webservers when a dagrun has no start date
[AIRFLOW-793] Enable compressed loading in S3ToHiveTransfer
[AIRFLOW-863] Example DAGs should have recent start dates
[AIRFLOW-869] Refactor mark success functionality
[AIRFLOW-856] Make sure execution date is set for local client
[AIRFLOW-814] Fix Presto*CheckOperator.__init__
[AIRFLOW-844] Fix cgroups directory creation
[AIRFLOW-816] Use static nvd3 and d3
[AIRFLOW-821] Fix py3 compatibility
[AIRFLOW-817] Check for None value of execution_date in endpoint
[AIRFLOW-822] Close db before exception
[AIRFLOW-815] Add prev/next execution dates to template variables
[AIRFLOW-813] Fix unterminated unit tests in SchedulerJobTest
[AIRFLOW-813] Fix unterminated scheduler unit tests
[AIRFLOW-806] UI should properly ignore DAG doc when it is None
[AIRFLOW-812] Fix the scheduler termination bug.
[AIRFLOW-780] Fix dag import errors no longer working
[AIRFLOW-783] Fix py3 incompatibility in BaseTaskRunner
[AIRFLOW-810] Correct down_revision dag_id/state index creation
[AIRFLOW-807] Improve scheduler performance for large DAGs
[AIRFLOW-798] Check return_code before forcing termination
[AIRFLOW-139] Let psycopg2 handle autocommit for PostgresHook
[AIRFLOW-776] Add missing cgroups devel dependency
[AIRFLOW-777] Fix expression to check if a DagRun is in running state
[AIRFLOW-785] Don't import CgroupTaskRunner at global scope
[AIRFLOW-784] Pin funcsigs to 1.0.0
[AIRFLOW-624] Fix setup.py to not import airflow.version as version
[AIRFLOW-779] Task should fail with specific message when deleted
[AIRFLOW-778] Fix completey broken MetastorePartitionSensor
[AIRFLOW-739] Set pickle_info log to debug
[AIRFLOW-771] Make S3 logs append instead of clobber
[AIRFLOW-773] Fix flaky datetime addition in api test
[AIRFLOW-219][AIRFLOW-398] Cgroups + impersonation
[AIRFLOW-683] Add jira hook, operator and sensor
[AIRFLOW-762] Add Google DataProc delete operator
[AIRFLOW-760] Update systemd config
[AIRFLOW-759] Use previous dag_run to verify depend_on_past
[AIRFLOW-757] Set child_process_log_directory default more sensible
[AIRFLOW-692] Open XCom page to super-admins only
[AIRFLOW-737] Fix HDFS Sensor directory.
[AIRFLOW-747] Fix retry_delay not honoured
[AIRFLOW-558] Add Support for dag.catchup=(True|False) Option
[AIRFLOW-489] Allow specifying execution date in trigger_dag API
[AIRFLOW-738] Commit deleted xcom items before insert
[AIRFLOW-729] Add Google Cloud Dataproc cluster creation operator
[AIRFLOW-728] Add Google BigQuery table sensor
[AIRFLOW-741] Log to debug instead of info for app.py
[AIRFLOW-731] Fix period bug for NamedHivePartitionSensor
[AIRFLOW-740] Pin jinja2 to < 2.9.0
[AIRFLOW-663] Improve time units for task performance charts
[AIRFLOW-665] Fix email attachments
[AIRFLOW-734] Fix SMTP auth regression when not using user/pass
[AIRFLOW-702] Fix LDAP Regex Bug
[AIRFLOW-717] Add Cloud Storage updated sensor
[AIRFLOW-695] Retries do not execute because dagrun is in FAILED state
[AIRFLOW-673] Add operational metrics test for SchedulerJob
[AIRFLOW-727] try_number is not increased
[AIRFLOW-715] A more efficient HDFS Sensor:
[AIRFLOW-716] Allow AVRO BigQuery load-job without schema
[AIRFLOW-718] Allow the query URI for DataProc Pig
Log needs to be part of try/catch block
[AIRFLOW-721] Descendant process can disappear before termination
[AIRFLOW-403] Bash operator's kill method leaves underlying processes running
[AIRFLOW-657] Add AutoCommit Parameter for MSSQL
[AIRFLOW-641] Improve pull request instructions
[AIRFLOW-685] Add test for MySqlHook.bulk_load()
[AIRFLOW-686] Match auth backend config section
[AIRFLOW-691] Add SSH KeepAlive option to SSH_hook
[AIRFLOW-709] Use same engine for migrations and reflection
[AIRFLOW-700] Update to reference to web authentication documentation
[AIRFLOW-649] Support non-sched DAGs in LatestOnlyOp
[AIRFLOW-712] Fix AIRFLOW-667 to use proper HTTP error properties
[AIRFLOW-710] Add OneFineStay as official user
[AIRFLOW-703][AIRFLOW-1] Stop Xcom being cleared too early
[AIRFLOW-679] Stop concurrent task instances from running
[AIRFLOW-704][AIRFLOW-1] Fix invalid syntax in BQ hook
[AIRFLOW-667] Handle BigQuery 503 error
[AIRFLOW-680] Disable connection pool for commands
[AIRFLOW-678] Prevent scheduler from double triggering TIs
[AIRFLOW-677] Kill task if it fails to heartbeat
[AIRFLOW-674] Ability to add descriptions for DAGs
[AIRFLOW-682] Bump MAX_PERIODS to make mark_success work for large DAGs
Use jdk selector to set required jdk
[AIRFLOW-647] Restore dag.get_active_runs
[AIRFLOW-662] Change seasons to months in project description
[AIRFLOW-656] Add dag/task/date index to xcom table
[AIRFLOW-658] Improve schema_update_options in GCP
[AIRFLOW-41] Fix pool oversubscription
[AIRFLOW-489] Add API Framework
[AIRFLOW-653] Add some missing endpoint tests
[AIRFLOW-652] Remove obsolete endpoint
[AIRFLOW-345] Add contrib ECSOperator
[AIRFLOW-650] Adding Celect to user list
[AIRFLOW-510] Filter Paused Dags, show Last Run & Trigger Dag
[AIRFLOW-643] Improve date handling for sf_hook
[AIRFLOW-638] Add schema_update_options to GCP ops
[AIRFLOW-640] Install and enable nose-ignore-docstring
[AIRFLOW-639]AIRFLOW-639] Alphasort package names
[AIRFLOW-375] Fix pylint errors
[AIRFLOW-347] Show empty DAG runs in tree view
[AIRFLOW-628] Adding SalesforceHook to contrib/hooks
[AIRFLOW-514] hive hook loads data from pandas DataFrame into hive and infers types
[AIRFLOW-565] Fixes DockerOperator on Python3.x
[AIRFLOW-635] Encryption option for S3 hook
[AIRFLOW-137] Fix max_active_runs on clearing tasks
[AIRFLOW-343] Fix schema plumbing in HiveServer2Hook
[AIRFLOW-130] Fix ssh operator macosx
[AIRFLOW-633] Show TI attributes in TI view
[AIRFLOW-626][AIRFLOW-1] HTML Content does not show up when sending email with attachment
[AIRFLOW-533] Set autocommit via set_autocommit
[AIRFLOW-629] stop pinning lxml
[AIRFLOW-464] Add setdefault method to Variable
[AIRFLOW-626][AIRFLOW-1] HTML Content does not show up when sending email with attachment
[AIRFLOW-591] Add datadog hook & sensor
[AIRFLOW-561] Add RedshiftToS3Transfer operator
[AIRFLOW-570] Pass root to date form on gantt
[AIRFLOW-504] Store fractional seconds in MySQL tables
[AIRFLOW-623] LDAP attributes not always a list
[AIRFLOW-611] source_format in BigQueryBaseCursor
[AIRFLOW-619] Fix exception in Gannt chart
[AIRFLOW-618] Cast DateTimes to avoid sqllite errors
[AIRFLOW-422] Add JSON endpoint for task info
[AIRFLOW-616][AIRFLOW-617] Minor fixes to PR tool UX
[AIRFLOW-179] Fix DbApiHook with non-ASCII chars
[AIRFLOW-566] Add timeout while fetching logs
[AIRFLOW-615] Set graph glyphicon first
[AIRFLOW-609] Add application_name to PostgresHook
[AIRFLOW-604] Revert .first() to .one()
[AIRFLOW-370] Create AirflowConfigException in exceptions.py
[AIRFLOW-582] Fixes TI.get_dagrun filter (removes start_date)
[AIRFLOW-568] Fix double task_stats count if a DagRun is active
[AIRFLOW-585] Fix race condition in backfill execution loop
[AIRFLOW-580] Prevent landscape warning on .format
[AIRFLOW-597] Check if content is None, not false-equivalent
[AIRFLOW-586] test_dag_v1 fails from 0 to 3 a.m.
[AIRFLOW-453] Add XCom Admin Page
[AIRFLOW-588] Add Google Cloud Storage Object sensor[]
[AIRFLOW-592] example_xcom import Error
[AIRFLOW-587] Fix incorrect scope for Google Auth[]
[AIRFLOW-589] Add templatable job_name[]
[AIRFLOW-227] Show running config in config view
[AIRFLOW-319]AIRFLOW-319] xcom push response in HTTP Operator
[AIRFLOW-385] Add symlink to latest scheduler log directory
[AIRFLOW-583] Fix decode error in gcs_to_bq
[AIRFLOW-96] s3_conn_id using environment variable
[AIRFLOW-575] Clarify tutorial and FAQ about `schedule_interval` always inheriting from DAG object
[AIRFLOW-577] Output BigQuery job for improved debugging
[AIRFLOW-560] Get URI & SQLA engine from Connection
[AIRFLOW-518] Require DataProfilingMixin for Variables CRUD
[AIRFLOW-553] Fix load path for filters.js
[AIRFLOW-554] Add Jinja support to Spark-sql
[AIRFLOW-550] Make ssl config check empty string safe
[AIRFLOW-500] Use id for github allowed teams
[AIRFLOW-556] Add UI PR guidelines
[AIRFLOW-358][AIRFLOW-430] Add `connections` cli
[AIRFLOW-548] Load DAGs immediately & continually
[AIRFLOW-539] Updated BQ hook and BQ operator to support Standard SQL.
[AIRFLOW-378] Add string casting to params of spark-sql operator
[AIRFLOW-544] Add Pause/Resume toggle button
[AIRFLOW-333][AIRFLOW-258] Fix non-module plugin components
[AIRFLOW-542] Add tooltip to DAGs links icons
[AIRFLOW-530] Update docs to reflect connection environment var has to be in uppercase
[AIRFLOW-525] Update template_fields in Qubole Op
[AIRFLOW-480] Support binary file download from GCS
[AIRFLOW-198] Implement latest_only_operator
[AIRFLOW-91] Add SSL config option for the webserver
[AIRFLOW-191] Fix connection leak with PostgreSQL backend
[AIRFLOW-512] Fix 'bellow' typo in docs & comments
[AIRFLOW-509][AIRFLOW-1] Create operator to delete tables in BigQuery
[AIRFLOW-498] Remove hard-coded gcp project id
[AIRFLOW-505] Support unicode characters in authors' names
[AIRFLOW-494] Add per-operator success/failure metrics
[AIRFLOW-488] Fix test_simple fail
[AIRFLOW-468] Update Panda requirement to 0.17.1
[AIRFLOW-159] Add cloud integration section + GCP documentation
[AIRFLOW-477][AIRFLOW-478] Restructure security section for clarity
[AIRFLOW-467] Allow defining of project_id in BigQueryHook
[AIRFLOW-483] Change print to logging statement
[AIRFLOW-475] make the segment granularity in Druid hook configurable
AIRFLOW 1.7.2
-------------
[AIRFLOW-463] Link Airflow icon to landing page
[AIRFLOW-149] Task Dependency Engine + Why Isn't My Task Running View
[AIRFLOW-361] Add default failure handler for the Qubole Operator
[AIRFLOW-353] Fix dag run status update failure
[AIRFLOW-447] Store source URIs in Python 3 compatible list
[AIRFLOW-443] Make module names unique when importing
[AIRFLOW-444] Add Google authentication backend
[AIRFLOW-446][AIRFLOW-445] Adds missing dataproc submit options
[AIRFLOW-431] Add CLI for CRUD operations on pools
[AIRFLOW-329] Update Dag Overview Page with Better Status Columns
[AIRFLOW-360] Fix style warnings in models.py
[AIRFLOW-425] Add white fill for null state tasks in tree view.
[AIRFLOW-69] Use dag runs in backfill jobs
[AIRFLOW-415] Make dag_id not found error clearer
[AIRFLOW-416] Use ordinals in README's company list
[AIRFLOW-369] Allow setting default DAG orientation
[AIRFLOW-410] Add 2 Q/A to the FAQ in the docs
[AIRFLOW-407] Add different colors for some sensors
[AIRFLOW-414] Improve error message for missing FERNET_KEY
[AIRFLOW-406] Sphinx/rst fixes
[AIRFLOW-412] Fix lxml dependency
[AIRFLOW-413] Fix unset path bug when backfilling via pickle
[AIRFLOW-78] Airflow clear leaves dag_runs
[AIRFLOW-402] Remove NamedHivePartitionSensor static check, add docs
[AIRFLOW-394] Add an option to the Task Duration graph to show cumulative times
[AIRFLOW-404] Retry download if unpacking fails for hive
[AIRFLOW-276] Gunicorn rolling restart
[AIRFLOW-399] Remove dags/testdruid.py
[AIRFLOW-400] models.py/DAG.set_dag_runs_state() does not correctly set state
[AIRFLOW-395] Fix colon/equal signs typo for resources in default config
[AIRFLOW-397] Documentation: Fix typo "instatiating" to "instantiating"
[AIRFLOW-395] Remove trailing commas from resources in config
[AIRFLOW-388] Add a new chart for Task_Tries for each DAG
[AIRFLOW-322] Fix typo in FAQ section
[AIRFLOW-375] Pylint fixes
limit scope to user email only AIRFLOW-386
[AIRFLOW-383] Cleanup example qubole operator dag
[AIRFLOW-160] Parse DAG files through child processes
[AIRFLOW-381] Manual UI Dag Run creation: require dag_id field
[AIRFLOW-373] Enhance CLI variables functionality
[AIRFLOW-379] Enhance Variables page functionality: import/export variables
[AIRFLOW-331] modify the LDAP authentication config lines in 'Security' sample codes
[AIRFLOW-356][AIRFLOW-355][AIRFLOW-354] Replace nobr, enable DAG only exists locally message, change edit DAG icon
[AIRFLOW-362] Import __future__ division
[AIRFLOW-359] Pin flask-login to 0.2.11
[AIRFLOW-261] Add bcc and cc fields to EmailOperator
[AIRFLOW-348] Fix code style warnings
[AIRFLOW-349] Add metric for number of zombies killed
[AIRFLOW-340] Remove unused dependency on Babel
[AIRFLOW-339]: Ability to pass a flower conf file
[AIRFLOW-341][operators] Add resource requirement attributes to operators
[AIRFLOW-335] Fix simple style errors/warnings
[AIRFLOW-337] Add __repr__ to VariableAccessor and VariableJsonAccessor
[AIRFLOW-334] Fix using undefined variable
[AIRFLOW-315] Fix blank lines code style warnings
[AIRFLOW-306] Add Spark-sql Hook and Operator
[AIRFLOW-327] Add rename method to the FTPHook
[AIRFLOW-321] Fix a wrong code example about tests/dags
[AIRFLOW-316] Always check DB state for Backfill Job execution
[AIRFLOW-264] Adding workload management for Hive
[AIRFLOW-297] support exponential backoff option for retry delay
[AIRFLOW-31][AIRFLOW-200] Add note to updating.md
[AIRFLOW-307] There is no __neq__ python magic method.
[AIRFLOW-309] Add requirements of develop dependencies to docs
[AIRFLOW-307] Rename __neq__ to __ne__ python magic method.
[AIRFLOW-313] Fix code style for sqoop_hook.py
[AIRFLOW-311] Fix wrong path in CONTRIBUTING.md
[AIRFLOW-24] DataFlow Java Operator
[AIRFLOW-308] Add link to refresh DAG within DAG view header
[AIRFLOW-314] Fix BigQuery cursor run_table_upsert method
[AIRFLOW-298] fix incubator diclaimer in docs
[AIRFLOW-284] HiveServer2Hook fix for cursor scope for get_results
[AIRFLOW-260] More graceful exit when issues can't be closed
[AIRFLOW-260] Handle case when no version is found
[AIRFLOW-228] Handle empty version list in PR tool
[AIRFLOW-302] Improve default squash commit message
[AIRFLOW-187] Improve prompt styling
[AIRFLOW-187] Fix typo in argument name
[AIRFLOW-187] Move "Close XXX" message to end of squash commit
[AIRFLOW-247] Add EMR hook, operators and sensors. Add AWS base hook
[AIRFLOW-301] Fix broken unit test
[AIRFLOW-100] Add execution_date_fn to ExternalTaskSensor
[AIRFLOW-282] Remove PR Tool logic that depends on version formatting
[AIRFLOW-291] Add index for state in TI table
[AIRFLOW-269] Add some unit tests for PostgreSQL
[AIRFLOW-296] template_ext is being treated as a string rather than a tuple in qubole operator
[AIRFLOW-286] Improve FTPHook to implement context manager interface
[AIRFLOW-243] Create NamedHivePartitionSensor
[AIRFLOW-246] Improve dag_stats endpoint query
[AIRFLOW-189] Highlighting of Parent/Child nodes in Graphs
[ARFLOW-255] Check dagrun timeout when comparing active runs
[AIRFLOW-281] Add port to mssql_hook
[AIRFLOW-285] Use Airflow 2.0 style imports for all remaining hooks/operators
[AIRFLOW-40] Add LDAP group filtering feature.
[AIRFLOW-277] Multiple deletions does not work in Task Instances view if using SQLite backend
[AIRFLOW-200] Make hook/operator imports lazy, and print proper exceptions
[AIRFLOW-283] Make store_to_xcom_key a templated field in GoogleCloudStorageDownloadOperator
[AIRFLOW-278] Support utf-8 ecoding for SQL
[AIRFLOW-280] clean up tmp druid table no matter if an ingestion job succeeds or not
[AIRFLOW-274] Add XCom functionality to GoogleCloudStorageDownloadOperator
[AIRFLOW-273] Create an svg version of the airflow logo.
[AIRFLOW-275] Update contributing guidelines
[AIRFLOW-244] Modify hive operator to inject analysis data
[AIRFLOW-162] Allow variable to be accessible into templates
[AIRFLOW-248] Add Apache license header to all files
[AIRFLOW-263] Remove temp backtick file
[AIRFLOW-252] Raise Sqlite exceptions when deleting tasks instance in WebUI
[AIRFLOW-180] Fix timeout behavior for sensors
[AIRFLOW-262] Simplify commands in MANIFEST.in
[AIRFLOW-31] Add zope dependency
[AIRFLOW-6] Remove dependency on Highcharts
[AIRFLOW-234] make task that aren't `running` self-terminate
[AIRFLOW-256] Fix test_scheduler_reschedule heartrate
Add Python 3 compatibility fix
[AIRFLOW-31] Use standard imports for hooks/operators
[AIRFLOW-173] Initial implementation of FileSensor
[AIRFLOW-224] Collect orphaned tasks and reschedule them
[AIRFLOW-239] Fix tests indentation
[AIRFLOW-225] Better units for task duration graph
[AIRFLOW-241] Add testing done section to PR template
[AIRFLOW-222] Show duration of task instances in ui
[AIRFLOW-231] Do not eval user input in PrestoHook
[AIRFLOW-216] Add Sqoop Hook and Operator
[AIRFLOW-171] Add upgrade notes on email and S3 to 1.7.1.2
[AIRFLOW-238] Make compatible with flask-admin 1.4.1
[AIRFLOW-230] [HiveServer2Hook] adding multi statements support
[AIRFLOW-142] setup_env.sh doesn't download hive tarball if hdp is specified as distro
[AIRFLOW-223] Make parametrable the IP on which Flower binds to
[AIRFLOW-218] Added option to enable webserver gunicorn access/err logs
[AIRFLOW-213] Add "Closes #X" phrase to commit messages
[AIRFLOW-68] Align start_date with the schedule_interval
[AIRFLOW-9] Improving docs to meet Apache's standards
[AIRFLOW-131] Make XCom.clear more selective
[AIRFLOW-214] Fix occasion of detached taskinstance
[AIRFLOW-206] Add commit to close PR
[AIRFLOW-206] Always load local log files if they exist
[AIRFLOW-211] Fix JIRA "resolve" vs "close" behavior
[AIRFLOW-64] Add note about relative DAGS_FOLDER
[AIRFLOW-114] Sort plugins dropdown
[AIRFLOW-209] Add scheduler tests and improve lineage handling
[AIRFLOW-207] Improve JIRA auth workflow
[AIRFLOW-187] Improve PR tool UX
[AIRFLOW-155] Documentation of Qubole Operator
Optimize and refactor process_dag
[AIRFLOW-185] Handle empty versions list
[AIRFLOW-201] Fix for HiveMetastoreHook + kerberos
[AIRFLOW-202]: Fixes stray print line
[AIRFLOW-196] Fix bug that exception is not handled in HttpSensor
[AIRFLOW-195] : Add toggle support to subdag clearing in the CLI
[AIRFLOW-23] Support for Google Cloud DataProc
[AIRFLOW-25] Configuration for Celery always required
[AIRFLOW-190] Add codecov and remove download count
[AIRFLOW-168] Correct evaluation of @once schedule
[AIRFLOW-183] Fetch log from remote when worker returns 4xx/5xx response
[AIRFLOW-181] Fix failing unpacking of hadoop by redownloading
[AIRFLOW-176] remove unused formatting key
[AIRFLOW-167]: Add dag_state option in cli
[AIRFLOW-178] Fix bug so that zip file is detected in DAG folder
[AIRFLOW-176] Improve PR Tool JIRA workflow
AIRFLOW-45: Support Hidden Airflow Variables
[AIRFLOW-175] Run git-reset before checkout in PR tool
[AIRFLOW-157] Make PR tool Py3-compat; add JIRA command
[AIRFLOW-170] Add missing @apply_defaults
AIRFLOW 1.7.1, 2016-05-19
-------------------------
- Fix : Don't treat premature tasks as could_not_run tasks
- AIRFLOW-92 Avoid unneeded upstream_failed session closes apache/incubator-airflow#1485
- Add logic to lock DB and avoid race condition
- Handle queued tasks from multiple jobs/executors
- AIRFLOW-52 Warn about overwriting tasks in a DAG
- Fix corner case with joining processes/queues (#1473)
- [AIRFLOW-52] Fix bottlenecks when working with many tasks
- Add columns to toggle extra detail in the connection list view.
- Log the number of errors when importing DAGs
- Log dagbag metrics dupplicate messages in queue into Statsd (#1406)
- Clean up issue template (#1419)
- correct missed arg.foreground to arg.daemon in cli
- Reinstate imports for github enterprise auth
- Use os.execvp instead of subprocess.Popen for the webserver
- Revert from using "--foreground" to "--daemon"
- Implement a Cloudant hook
- Add missing args to `airflow clear`
- Fixed a bug in the scheduler: num_runs used where runs intended
- Add multiprocessing support to the scheduler
- Partial fix to make sure next_run_date cannot be None
- Support list/get/set variables in the CLI
- Properly handle BigQuery booleans in BigQuery hook.
- Added the ability to view XCom variables in webserver
- Change DAG.tasks from a list to a dict
- Add support for zipped dags
- Stop creating hook on instantiating of S3 operator
- User subquery in views to find running DAGs
- Prevent DAGs from being reloaded on every scheduler iteration
- Add a missing word to docs
- Document the parameters of `DbApiHook`
- added oracle operator with existing oracle hook
- Add PyOpenSSL to Google cloud gcp_api.
- Remove executor error unit test
- Add DAG inference, deferral, and context manager
- Don't return error when writing files to Google cloud storage.
- Fix GCS logging for gcp_api.
- Ensure attr is in scope for error message
- Fixing misnamed PULL_REQUEST_TEMPLATE
- Extract non_pooled_task_slot_count into a configuration param
- Update plugins.rst for clarity on the example (#1309)
- Fix s3 logging issue
- Add twitter feed example dag
- Github ISSUE_TEMPLATE & PR_TEMPLATE cleanup
- Reduce logger verbosity
- Adding a PR Template
- Add Lucid to list of users
- Fix usage of asciiart
- Use session instead of outdated main_session for are_dependencies_met
- Fix celery flower port allocation
- Fix for missing edit actions due to flask-admin upgrade
- Fix typo in comment in prioritize_queued method
- Add HipchatOperator
- Include all example dags in backfill unit test
- Make sure skipped jobs are actually skipped
- Fixing a broken example dag, example_skip_dag.py
- Add consistent and thorough signal handling and logging
- Allow Operators to specify SKIPPED status internally
- Update docstring for executor trap unit test
- Doc: explain the usage of Jinja templating for templated params
- Don't schedule runs before the DAG's start_date
- Fix infinite retries with pools, with test
- Fix handling of deadlocked jobs
- Show only Airflow's deprecation warnings
- Set DAG_FOLDER for unit tests
- Missing comma in setup.py
- Deprecate *args and **kwargs in BaseOperator
- Raise deep scheduler exceptions to force a process restart.
- Change inconsistent example DAG owners
- Fix module path of send_email_smtp in configuration
- added Gentner Lab to list of users
- Increase timeout time for unit test
- Fix reading strings from conf
- CHORE - Remove Trailing Spaces
- Fix SSHExecuteOperator crash when using a custom ssh port
- Add note about airflow components to template
- Rewrite BackfillJob logic for clarity
- Add unit tests
- Fix miscellaneous bugs and clean up code
- Fix logic for determining DagRun states
- Make SchedulerJob not run EVERY queued task
- Improve BackfillJob handling of queued/deadlocked tasks
- Introduce ignore_depends_on_past parameters
- Use Popen with CeleryExecutor
- Rename user table to users to avoid conflict with postgres
- Beware of negative pool slots.