Code

15 gtfs validation pipeline#20

Code

Merged

ethan-moss merged 98 commits intodevdatasciencecampus/transport-network-performance:devfrom 15-gtfs-validation-pipelinedatasciencecampus/transport-network-performance:15-gtfs-validation-pipelineCopy head branch name to clipboardAug 16, 2023

+2,012-5

Commits

Commits on Jun 21, 2023

• chore: Add ipykernel to deps

  

  r-leyshon

  committed
  7b138f1

  Copy full SHA for 7b138f1

Commits on Jun 28, 2023

• chore: Add reqs for scraping GTFS lookup table

  

  r-leyshon

  committed
  6b29978

  Copy full SHA for 6b29978

• feat: Func to ingest GTFS route_type table from urls

  

  r-leyshon

  committed
  7eaffe9

  Copy full SHA for 7eaffe9

• refactor: Dynamic column name lookup

  

  r-leyshon

  committed
  5bf382f

  Copy full SHA for 5bf382f

• refactor: Reduce num of variables

  

  r-leyshon

  committed
  ac0ae02

  Copy full SHA for ac0ae02

• feat: Defensive checking & refactoring for complexity hook

  

  r-leyshon

  committed
  2a4a433

  Copy full SHA for 2a4a433

Commits on Jun 29, 2023

• docs: Add flag guidance to test module

  

  r-leyshon

  committed
  f7c2844

  Copy full SHA for f7c2844

• docs: Typos

  

  r-leyshon

  committed
  e8f5607

  Copy full SHA for e8f5607

• chore: Test fixture for route_type lookup

  

  r-leyshon

  committed
  5acc6e4

  Copy full SHA for 5acc6e4

• chore: Adjust pickle ignore rules for single named exception

  

  r-leyshon

  committed
  a29ea96

  Copy full SHA for a29ea96

• chore: Setup runinteg pytest flag

  

  r-leyshon

  committed
  bcfd84c

  Copy full SHA for bcfd84c

• test: Integration test check scraped html table is stable

  

  r-leyshon

  committed
  f91f508

  Copy full SHA for f91f508

• test: Defensive checks raise as expected

  

  r-leyshon

  committed
  ac0c179

  Copy full SHA for ac0c179

• chore: Add test flag, Sergio suggested fix

  

  r-leyshon

  committed
  ca73900

  Copy full SHA for ca73900

• Merge branch 'dev' into 15-gtfs-validation-pipeline

  

  r-leyshon

  committed
  3652ae7

  Copy full SHA for 3652ae7

• refactor: Helper function easier to mock requests

  

  r-leyshon

  committed
  ddfe587

  Copy full SHA for ddfe587

• chore: Add mocking dependencies

  

  r-leyshon

  committed
  62d2925

  Copy full SHA for 62d2925

• test: Check expected lookup is returned

  

  r-leyshon

  committed
  fbf32d8

  Copy full SHA for fbf32d8

• refactor: Use class to handle mocked return values

  

  r-leyshon

  committed
  a63a51c

  Copy full SHA for a63a51c

• fix: Catch the final gtfs description as not succeeded by br tag

  

  r-leyshon

  committed
  c50c266

  Copy full SHA for c50c266

• fix: Update response value for new func logic

  

  r-leyshon

  committed
  c168573

  Copy full SHA for c168573

• test: Table scraped correctly with extended schema format

  

  r-leyshon

  committed
  ab205bf

  Copy full SHA for ab205bf

• fix: Update test fixture lookup with corrected format

  

  r-leyshon

  committed
  40186dc

  Copy full SHA for 40186dc

• chore: Delete placeholder script

  

  r-leyshon

  committed
  c20a94a

  Copy full SHA for c20a94a

Commits on Jul 3, 2023

• chore: Add toml to deps

  

  r-leyshon

  committed
  2ed8ec3

  Copy full SHA for 2ed8ec3

• feat: First iteration of validation class

  

  r-leyshon

  committed
  75f8c77

  Copy full SHA for 75f8c77

• style: Consistent use of speechmarks

  

  r-leyshon

  committed
  953e1b2

  Copy full SHA for 953e1b2

• feat: Pipeline to check, clean, visualise GTFS

  

  r-leyshon

  committed
  26dc30c

  Copy full SHA for 26dc30c

• feat: Helper print statements in pipeline

  

  r-leyshon

  committed
  3d443ec

  Copy full SHA for 3d443ec

• test: Test class init defensive behaviour

  

  r-leyshon

  committed
  1650c90

  Copy full SHA for 1650c90

• feat: Class defence implementation

  

  r-leyshon

  committed
  dfc3b21

  Copy full SHA for dfc3b21

• test: Assertions about feed instantiated attribute

  

  r-leyshon

  committed
  1c06fd0

  Copy full SHA for 1c06fd0

• test: Passes on conversion from UK meters to m

  

  r-leyshon

  committed
  5a56e6f

  Copy full SHA for 5a56e6f

• test: Check dimensions of validity_df attribute

  

  r-leyshon

  committed
  8555382

  Copy full SHA for 8555382

• feat: Defensive checks for print_alerts method

  

  r-leyshon

  committed
  37cb97b

  Copy full SHA for 37cb97b

• test: Defensive behaviour of print_alerts method

  

  r-leyshon

  committed
  ad778bb

  Copy full SHA for ad778bb

• fix: Use hasattr to check for attribute existence

  

  r-leyshon

  committed
  a39fc34

  Copy full SHA for a39fc34

• test: print_alert prints single error without truncation

  

  r-leyshon

  committed
  a987d72

  Copy full SHA for a987d72

• test: print_alerts prints multiple alerts without truncation

  

  r-leyshon

  committed
  3568572

  Copy full SHA for 3568572

• refactor: Tests use function scope fixture

  

  r-leyshon

  committed
  e895a95

  Copy full SHA for e895a95

Commits on Jul 4, 2023

• fix: Filter pkg_resources warnings

  

  r-leyshon

  committed
  e2b031f

  Copy full SHA for e2b031f

• refactor: Rm unused **kwargs

  

  r-leyshon

  committed
  61f9afa

  Copy full SHA for 61f9afa

• test: Defensive behaviour for viz_stops

  

  r-leyshon

  committed
  7fbcd49

  Copy full SHA for 7fbcd49

• refactor: Helper function checks path-likes

  

  r-leyshon

  committed
  04dd63f

  Copy full SHA for 04dd63f

• viz_stops defense implementation

  

  r-leyshon

  committed
  15c8ef1

  Copy full SHA for 15c8ef1

• refactor: Unified save statement

  

  r-leyshon

  committed
  fabbbbe

  Copy full SHA for fabbbbe

• test: viz_stops behaviour when mapping points

  

  r-leyshon

  committed
  c5fd6c1

  Copy full SHA for c5fd6c1

• test: Fileops on viz_plot when plot convex hull

  

  r-leyshon

  committed
  e593b3d

  Copy full SHA for e593b3d

• test: Internal helper renders text as required

  

  r-leyshon

  committed
  9d57d9f

  Copy full SHA for 9d57d9f

• refactor: Minor text formatting

  

  r-leyshon

  committed
  42f7180

  Copy full SHA for 42f7180

• test: Expected table format for get_route_modes

  

  r-leyshon

  committed
  7320cee

  Copy full SHA for 7320cee

• feat: Defensive checks for summarise_weekday

  

  r-leyshon

  committed
  3778229

  Copy full SHA for 3778229

• test: Defensive behaviour for summarise_weekday

  

  r-leyshon

  committed
  e773ca4

  Copy full SHA for e773ca4

• feat: Defensive implementation for summarise_weekday

  

  r-leyshon

  committed
  a2278a3

  Copy full SHA for a2278a3

• refactor: Move integration test into more appropriate test module

  

  r-leyshon

  committed
  6484092

  Copy full SHA for 6484092

• test: Expensive test for test_summarise_weekday_defence with pytest flag

  

  r-leyshon

  committed
  de72c6b

  Copy full SHA for de72c6b

Commits on Jul 5, 2023

• test: Check mockers called as expected

  

  r-leyshon

  committed
  fc0dfc6

  Copy full SHA for fc0dfc6

• refactor: Mock scrape call when test get_route_modes

  

  r-leyshon

  committed
  102f812

  Copy full SHA for 102f812

• feat: Pipeline optionally prints performance profiling

  

  r-leyshon

  committed
  a4ddb9e

  Copy full SHA for a4ddb9e

Commits on Jul 6, 2023

• refactor: Move utility defensive checkers to dedicated module

  

  r-leyshon

  committed
  1e83b01

  Copy full SHA for 1e83b01

• feat: Added optional performance testing to pipeline

  

  r-leyshon

  committed
  10bacf6

  Copy full SHA for 10bacf6

• refactor: Move more utilities to defence module

  

  r-leyshon

  committed
  dbf76cc

  Copy full SHA for dbf76cc

Commits on Jul 7, 2023

• test: Func returns None on pass

  

  r-leyshon

  committed
  7fc2cd9

  Copy full SHA for 7fc2cd9

• test: bbox_filter behaviour on pass

  

  r-leyshon

  committed
  9d9881e

  Copy full SHA for 9d9881e

• feat: Func filters GTFS based on bbox

  

  r-leyshon

  committed
  184ccde

  Copy full SHA for 184ccde

• feat: Class now prints calendar date range

  

  r-leyshon

  committed
  b0aa256

  Copy full SHA for b0aa256

• feat: Defence check list & elements optionally

  

  r-leyshon

  committed
  62b60ab

  Copy full SHA for 62b60ab

• feat: Pipeline checks GTFS archive exists & extracts available dates in calendar

  

  r-leyshon

  committed
  f00dabb

  Copy full SHA for f00dabb

• refactor: Minor string styling in STOP_HULL_PTH

  

  r-leyshon

  committed
  50a8987

  Copy full SHA for 50a8987

Commits on Jul 10, 2023

• test: Defensive behaviour for utility func

  

  r-leyshon

  committed
  703140c

  Copy full SHA for 703140c

• refactor: Avoid complexity hook

  

  r-leyshon

  committed
  de17234

  Copy full SHA for de17234

• test: Assert refcatored message calls

  

  r-leyshon

  committed
  85c771f

  Copy full SHA for 85c771f

• feat: Handle attribute error

  

  r-leyshon

  committed
  67f1b4e

  Copy full SHA for 67f1b4e

• test: Exception handle when no alerts exist

  

  r-leyshon

  committed
  c2554fd

  Copy full SHA for c2554fd

• test: Raises error on missing parent dir when create=False

  

  r-leyshon

  committed
  691d88d

  Copy full SHA for 691d88d

• test: Simulated condition where shape_id missing from shapes.txt

  

  r-leyshon

  committed
  b2ac376

  Copy full SHA for b2ac376

• test: viz_stops handles cases with missing stops_id

  

  r-leyshon

  committed
  5341729

  Copy full SHA for 5341729

• refactor: Add assertion fail messages to all tests

  

  r-leyshon

  committed
  212be70

  Copy full SHA for 212be70

• refactor: Add print statements to assertion fails

  

  r-leyshon

  committed
  cae3757

  Copy full SHA for cae3757

• chore: Add more assertion fail print statements

  

  r-leyshon

  committed
  f9308cf

  Copy full SHA for f9308cf

• refactor: Test expected columns

  

  r-leyshon

  committed
  616c3e7

  Copy full SHA for 616c3e7

• chore: Add assertion failure message

  

  r-leyshon

  committed
  c17668e

  Copy full SHA for c17668e

• chore: Add pyrojroot to deps

  

  r-leyshon

  committed
  23467cd

  Copy full SHA for 23467cd

• refactor: Run pytest with no runsetup check

  

  r-leyshon

  committed
  b2fff03

  Copy full SHA for b2fff03

• chore: Try fixing np version

  

  r-leyshon

  committed
  e52a6c4

  Copy full SHA for e52a6c4

Commits on Jul 11, 2023

• refactor: Class name syntax

  

  r-leyshon

  committed
  bb886d5

  Copy full SHA for bb886d5

Commits on Jul 19, 2023

• chore: Merge dev breaking changes & update import statements

  

  r-leyshon

  committed
  8b809e8

  Copy full SHA for 8b809e8

Commits on Aug 3, 2023

• Refactored daily summaries (#35)

  Show description for 364cd03
  

  CBROWN-ONS

  and

  r-leyshon

  authored
  364cd03

  Copy full SHA for 364cd03

Commits on Aug 7, 2023

• refactor: Use gtfskit get_dates

  

  r-leyshon

  committed
  7a82780

  Copy full SHA for 7a82780

• refactor: Rm shallow wrapper around gtfs_kit.get_dates

  

  r-leyshon

  committed
  dce4bc9

  Copy full SHA for dce4bc9

• docs: Update mthd return type

  

  r-leyshon

  committed
  3d04cda

  Copy full SHA for 3d04cda

• chore: Bring up to date with dev branch

  

  r-leyshon

  committed
  57ee8c7

  Copy full SHA for 57ee8c7

• validation that checks are not needed for non matching GTFS IDs

  CBROWN-ONS

  committed
  4ee328d

  Copy full SHA for 4ee328d

• refactor: Condition check compatible with numpy >=1.25.0

  

  r-leyshon

  committed
  362219d

  Copy full SHA for 362219d

• refactor: Test returns min and max columns instead of amin & amax in np>=1.25.0

  

  r-leyshon

  committed
  52b683e

  Copy full SHA for 52b683e

• chore: Pin minimum version of numpy to 1.25.0

  

  r-leyshon

  committed
  32be07c

  Copy full SHA for 32be07c

• fix: Typo in reqs

  

  r-leyshon

  committed
  a37df94

  Copy full SHA for a37df94

Commits on Aug 16, 2023

• merged with dev; resolved requirements conflict

  ethan-moss

  committed
  053fcde

  Copy full SHA for 053fcde