Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

some initial doc on netfile v2 migration #360

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

some initial doc on netfile v2 migration #360

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
4642c27
some initial doc on netfile v2 migration
ChenglimEar Dec 12, 2024
d9484a4
Run `make clean download import process`
Dec 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion build/_data/stats.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
{
"date_processed": "2024-12-10 00:12:32 -0800"
"date_processed": "2024-12-11 00:13:07 -0800"
}
17 changes: 17 additions & 0 deletions docs/netfile_v2.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Netfile API v2 Migration Plan

## Makefile changes to support new download

The Makefile will be updated to also download Netfile v2 files by running `python download/main.py` if the local environment is set up with credentials to access the files on Google Drive.

Two variables control whether the new Netfile v2 download and import will occur when `make download` and `make import` are run:
* NETFILE_V2_DOWNLOAD
* NETFILE_V2_IMPORT

The variables above will only be set if the `SERVICE_ACCOUNT_KEY_JSON` environment variable is set or a file exists at `.local/SERVICE_ACCOUNT_KEY_JSON.json` . If the variables above aren't set at the top of the Makefile, the new downloads will not occur. The two variables set targets that run during download and import.

The new download will download files to `.local/download`.

## Updating python package requirements

During development, a separate requirements.txt file was maintained in the `download` directory. When we finally merge the code into main, this file will be deprecated. All required packages must be specified in the top level `requirements.txt` file that is used by the existing download code. In other words, we have to make sure that the same versions of packages work for both the old and new download code.