Skip to content
This repository has been archived by the owner on Oct 16, 2024. It is now read-only.

Update Elasticsearch #2186

Merged
merged 2 commits into from
Jan 8, 2024
Merged

Update Elasticsearch #2186

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3fca3d1
Update Elasticsearch
bensteinberg Jan 8, 2024
4c7a358
Handle date_added the same as last_updated
bensteinberg Jan 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion capstone/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ services:
redis:
image: redis:6.0.15
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.17.15
image: docker.elastic.co/elasticsearch/elasticsearch:7.17.16
environment:
- node.name=es01
- bootstrap.memory_lock=true
Expand Down
6 changes: 5 additions & 1 deletion capstone/scripts/tests/test_cap_static.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,16 @@ def test_export_cap_static(reset_sequences, case_factory, jurisdiction_factory,
with django_assert_num_queries(select=8):
summarize_cap_static(str(tmp_path))

# fix up last_updated timestamps for consistent comparison
# fix up last_updated and date_added timestamps for consistent comparison
for path in tmp_path.rglob('*/*/*/**/*.json'):
path.write_text(re.sub(
r'"last_updated": "[^"]+"',
'"last_updated": "2023-12-01T01:01:01.0000001+00:00"',
path.read_text()))
path.write_text(re.sub(
r'"date_added": "[^"]+"',
'"date_added": "2023-12-01"',
path.read_text()))

# compare temp dir to test_data/cap_static
cap_static_dir = Path(settings.BASE_DIR, 'test_data/cap_static')
Expand Down
6 changes: 3 additions & 3 deletions capstone/test_data/cap_static/redacted/us/1/CasesMetadata.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
"analysis": {},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down Expand Up @@ -76,7 +76,7 @@
"analysis": {},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down Expand Up @@ -118,7 +118,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
2 changes: 1 addition & 1 deletion capstone/test_data/cap_static/redacted/us/1/cases/0001-01.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
"analysis": {},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
2 changes: 1 addition & 1 deletion capstone/test_data/cap_static/redacted/us/1/cases/0002-01.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
"analysis": {},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
2 changes: 1 addition & 1 deletion capstone/test_data/cap_static/redacted/us/1/cases/0002-02.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
6 changes: 3 additions & 3 deletions capstone/test_data/cap_static/redacted/us/2/CasesMetadata.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
"analysis": {},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down Expand Up @@ -76,7 +76,7 @@
"analysis": {},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down Expand Up @@ -118,7 +118,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
2 changes: 1 addition & 1 deletion capstone/test_data/cap_static/redacted/us/2/cases/0001-01.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
"analysis": {},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
2 changes: 1 addition & 1 deletion capstone/test_data/cap_static/redacted/us/2/cases/0002-01.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
"analysis": {},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
2 changes: 1 addition & 1 deletion capstone/test_data/cap_static/redacted/us/2/cases/0002-02.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
6 changes: 3 additions & 3 deletions capstone/test_data/cap_static/unredacted/us/1/CasesMetadata.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down Expand Up @@ -76,7 +76,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down Expand Up @@ -118,7 +118,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
2 changes: 1 addition & 1 deletion capstone/test_data/cap_static/unredacted/us/1/cases/0001-01.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
2 changes: 1 addition & 1 deletion capstone/test_data/cap_static/unredacted/us/1/cases/0002-01.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
2 changes: 1 addition & 1 deletion capstone/test_data/cap_static/unredacted/us/1/cases/0002-02.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
6 changes: 3 additions & 3 deletions capstone/test_data/cap_static/unredacted/us/2/CasesMetadata.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down Expand Up @@ -76,7 +76,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down Expand Up @@ -118,7 +118,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
2 changes: 1 addition & 1 deletion capstone/test_data/cap_static/unredacted/us/2/cases/0001-01.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
2 changes: 1 addition & 1 deletion capstone/test_data/cap_static/unredacted/us/2/cases/0002-01.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
2 changes: 1 addition & 1 deletion capstone/test_data/cap_static/unredacted/us/2/cases/0002-02.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
},
"last_updated": "2023-12-01T01:01:01.0000001+00:00",
"provenance": {
"date_added": "2023-12-05",
"date_added": "2023-12-01",
"source": "Harvard",
"batch": "2018"
},
Expand Down
Loading