Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
1295e91
start adding functions
ehinman Aug 8, 2025
c4b0b9a
start adding documentation and going through functions
ehinman Aug 27, 2025
c32ded5
adjust date function
ehinman Aug 28, 2025
99e949c
fix dates function
ehinman Aug 29, 2025
1641e85
keep working out issues with api calls
ehinman Aug 29, 2025
7bc6c6f
add documentation
ehinman Aug 29, 2025
1b29d6a
adjust how response is handled and edit walk pages, fix API limit print
ehinman Sep 19, 2025
3289982
add documentation
ehinman Sep 19, 2025
867d728
add more documentation, correct waterdata module
ehinman Sep 19, 2025
44213b5
allow post and get calls in recursive walk pages, fix typo where firs…
ehinman Sep 19, 2025
4affa2f
add in all possible arguments
ehinman Sep 19, 2025
21691d0
trying to get cql2 query correct, will keep at it
ehinman Sep 19, 2025
4c2a3ee
correct cql2 queries
ehinman Sep 22, 2025
14f2830
simplify syntax, remove unneeded dependencies
ehinman Sep 22, 2025
d25f854
start adding function documentation
ehinman Sep 24, 2025
7fe486a
add link urls
ehinman Sep 25, 2025
fad9ce0
fix date formatting function
ehinman Sep 25, 2025
a33d201
make waterdata outputs geopandas if geometry included
ehinman Sep 25, 2025
bd82c49
make gpd an optional dependency and change returns accordingly
ehinman Sep 25, 2025
06b0e69
incorporate geopandas boolean into function arguments and ensure user…
ehinman Sep 25, 2025
253da79
clean up some documentation and comments
ehinman Sep 25, 2025
f5cca07
add optional dependency to pyproject.toml
ehinman Sep 25, 2025
5c546e7
set convertType to default or user specification
ehinman Sep 25, 2025
e9221ac
start unit tests on new functions
ehinman Sep 25, 2025
b1436db
update README and add a NEWS markdown in which to place past updates
ehinman Sep 26, 2025
dc24658
make a few small changes to names and documentation
ehinman Sep 26, 2025
89b960c
define max_results when it is an input
ehinman Sep 26, 2025
1237777
comment out code that wasn't doing the correct thing with max_results
ehinman Sep 26, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
**10/01/2025:** `dataretrieval` is pleased to offer a new module, `waterdata`, which gives users access USGS's modernized [Water Data APIs](https://api.waterdata.usgs.gov/). The Water Data API endpoints include daily values, instantaneous values, field measurements (modernized groundwater levels service), time series metadata, and discrete water quality data from the Samples database. Though there will be a period of overlap, the functions within `waterdata` will eventually replace the `nwis` module, which currently provides access to the legacy [NWIS Water Services](https://waterservices.usgs.gov/). More example workflows and functions coming soon. Check `help(waterdata)` for more information.

**09/03/2024:** The groundwater levels service has switched endpoints, and `dataretrieval` was updated accordingly in [`v1.0.10`](https://github.com/DOI-USGS/dataretrieval-python/releases/tag/v1.0.10). Older versions using the discontinued endpoint will return 503 errors for `nwis.get_gwlevels` or the `service='gwlevels'` argument. Visit [Water Data For the Nation](https://waterdata.usgs.gov/blog/wdfn-waterservices-2024/) for more information.

**03/01/2024:** USGS data availability and format have changed on Water Quality Portal (WQP). Since March 2024, data obtained from WQP legacy profiles will not include new USGS data or recent updates to existing data. All USGS data (up to and beyond March 2024) are available using the new WQP beta services. You can access the beta services by setting `legacy=False` in the functions in the `wqp` module.

To view the status of changes in data availability and code functionality, visit: https://doi-usgs.github.io/dataRetrieval/articles/Status.html
25 changes: 14 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,19 @@
![Conda Version](https://img.shields.io/conda/v/conda-forge/dataretrieval)
![Downloads](https://static.pepy.tech/badge/dataretrieval)

:warning: USGS data availability and format have changed on Water Quality Portal (WQP). Since March 2024, data obtained from WQP legacy profiles will not include new USGS data or recent updates to existing data. All USGS data (up to and beyond March 2024) are available using the new WQP beta services. You can access the beta services by setting `legacy=False` in the functions in the `wqp` module.
## Latest Announcements

To view the status of changes in data availability and code functionality, visit: https://doi-usgs.github.io/dataRetrieval/articles/Status.html
:mega: **10/01/2025:** `dataretrieval` is pleased to offer a new, *in-development* module, `waterdata`, which gives users access USGS's modernized [Water Data APIs](https://api.waterdata.usgs.gov/). The Water Data API endpoints include daily values, instantaneous values, field measurements (modernized groundwater levels service), time series metadata, and discrete water quality data from the Samples database. Though there will be a period of overlap, the functions within `waterdata` will eventually replace the `nwis` module, which currently provides access to the legacy [NWIS Water Services](https://waterservices.usgs.gov/). More example workflows and functions coming soon. Check `help(waterdata)` for more information.

:mega: **09/03/2024:** The groundwater levels service has switched endpoints, and `dataretrieval` was updated accordingly in [`v1.0.10`](https://github.com/DOI-USGS/dataretrieval-python/releases/tag/v1.0.10). Older versions using the discontinued endpoint will return 503 errors for `nwis.get_gwlevels` or the `service='gwlevels'` argument. Visit [Water Data For the Nation](https://waterdata.usgs.gov/blog/wdfn-waterservices-2024/) for more information.
**Important:** Users of the Water Data APIs are strongly encouraged to obtain an API key, which gives users higher rate limits and thus greater access to USGS data. [Register for an API key](https://api.waterdata.usgs.gov/signup/) and then place that API key in your python environment as an environment variable named "API_USGS_PAT". One option is to set the variable as follows:

```python
import os
os.environ["API_USGS_PAT"] = "your_api_key_here"
```
Note that you may need to restart your python session for the environment variable to be recognized.

Check out the [NEWS](NEWS.md) file for all updates and announcements, or track updates to the package via the GitHub releases.

## What is dataretrieval?
`dataretrieval` was created to simplify the process of loading hydrologic data into the Python environment.
Expand All @@ -20,8 +28,7 @@ Environmental Protection Agency (EPA), U.S. Department of Agriculture
(USDA), and USGS. Direct USGS data is obtained from a service called the
National Water Information System (NWIS).

Note that the python version is not a direct port of the original: it attempts to reproduce the functionality of the R package,
though its organization and interface often differ.
Note that the python version is not a direct port of the original: it attempts to reproduce the functionality of the R package, though its organization and interface often differ.

If there's a hydrologic or environmental data portal that you'd like dataretrieval to
work with, raise it as an [issue](https://github.com/USGS-python/dataretrieval/issues).
Expand Down Expand Up @@ -53,7 +60,7 @@ Water quality data are available from:
- [Samples](https://waterdata.usgs.gov/download-samples/#dataProfile=site) - Discrete USGS water quality data only
- [Water Quality Portal](https://www.waterqualitydata.us/) - Discrete water quality data from USGS and EPA. Older data are available in the legacy WQX version 2 format; all data are available in the beta WQX3.0 format.

To access the full functionality available from NWIS web services, nwis.get record appends any additional kwargs into the REST request. For example, this function call:
To access the full functionality available from NWIS web services, `nwis.get_record()` appends any additional kwargs into the REST request. For example, this function call:
```python
nwis.get_record(sites='03339000', service='dv', start='2017-12-31', parameterCd='00060')
```
Expand All @@ -67,8 +74,6 @@ For example
nwis.get_record(sites='05404147',service='iv', start='2021-01-01', end='2021-3-01', access='3')
```

More services and documentation to come!

## Quick start

dataretrieval can be installed using pip:
Expand Down Expand Up @@ -99,13 +104,11 @@ For more details, see the file [CONTRIBUTING.md](CONTRIBUTING.md).

## Need help?

The Water Mission Area of the USGS supports the development and maintenance of `dataretrieval`. Any questions can be directed to the Computational Tools team at
[email protected].
The Water Mission Area of the USGS supports the development and maintenance of `dataretrieval`. Any questions can be directed to the Computational Tools team at [email protected].

Resources are available primarily for maintenance and responding to user questions.
Priorities on the development of new features are determined by the `dataretrieval` development team.


## Acknowledgments
This material is partially based upon work supported by the National Science Foundation (NSF) under award 1931297.
Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF.
Expand Down
Loading