Skip to content

Commit

Permalink
Download data in the Python script.
Browse files Browse the repository at this point in the history
I was downloading the data in a separate hook but there is a test that just runs all of the Python scripts in src/python and apparently it's not trivial to get the data download to trigger just once, when the appropriate Python script test is running, so it's just easier to move the data download to the script that actually uses it. We also add a comment to the Rust script to direct users to the Python script for the data location.
  • Loading branch information
rodrigogiraoserrao committed Oct 10, 2024
1 parent a5df31e commit 2d8c3c7
Show file tree
Hide file tree
Showing 4 changed files with 28 additions and 29 deletions.
28 changes: 0 additions & 28 deletions docs/source/_build/scripts/prep_data.py

This file was deleted.

25 changes: 25 additions & 0 deletions docs/source/src/python/user-guide/transformations/joins.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,28 @@
import requests


DATA = [
(
"https://raw.githubusercontent.com/pola-rs/polars-static/refs/heads/master/data/monopoly_props_groups.csv",
"docs/assets/data/monopoly_props_groups.csv",
),
(
"https://raw.githubusercontent.com/pola-rs/polars-static/refs/heads/master/data/monopoly_props_prices.csv",
"docs/assets/data/monopoly_props_prices.csv",
),
]


for url, dest in DATA:
with open(dest, "wb") as f:
try:
f.write(requests.get(url, timeout=10).content)
except Exception as e:
print(f"WARNING: failed to download file {dest} ({e})")
else:
print(f"INFO: downloaded {dest}")


# --8<-- [start:props_groups]
import polars as pl

Expand Down
3 changes: 3 additions & 0 deletions docs/source/src/rust/user-guide/transformations/joins.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ use polars::prelude::*;
// --8<-- [end:setup]

fn main() -> Result<(), Box<dyn std::error::Error>> {
// NOTE: This assumes the data has been downloaded and is available.
// See the corresponding Python script for the remote location of the data.

// --8<-- [start:props_groups]
let props_groups = CsvReadOptions::default()
.with_has_header(true)
Expand Down
1 change: 0 additions & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,6 @@ markdown_extensions:

hooks:
- docs/source/_build/scripts/people.py
- docs/source/_build/scripts/prep_data.py

plugins:
- search:
Expand Down

0 comments on commit 2d8c3c7

Please sign in to comment.