Merge branch 'main' into ixmp4/auth

IAMconsortium · Aug 22, 2023 · c85d17a · c85d17a
2 parents 715d78c + e70a0b7
commit c85d17a
Show file tree

Hide file tree

Showing 54 changed files with 619 additions and 506 deletions.
diff --git a/.github/workflows/pytest.yml b/.github/workflows/pytest.yml
@@ -47,7 +47,7 @@ jobs:
 
     - name: Upload coverage report to Codecov
       if: ${{ matrix.os == 'ubuntu-latest' && matrix.python-version == '3.10' }}
-      uses: codecov/codecov-action@v1
+      uses: codecov/codecov-action@v3
       with:
         file: ./coverage.xml
         env_vars: ${{ matrix.os }} py${{ matrix.python-version }}
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
@@ -14,20 +14,29 @@ Credentials to access the IIASA scenario database infrastructure should now be m
 using the **ixmp4** package
 (see [here](https://pyam-iamc.readthedocs.io/en/stable/api/iiasa.html)).
 
-The column *exclude* of the `meta` attribute was refacored to a new attribute `exclude`.
+The column *exclude* of the `meta` indicators was moved to a new attribute `exclude`.
 All validation methods are refactored such that the argument `exclude_on_fail` changes
 this new attribute (see PR [#759](https://github.com/IAMconsortium/pyam/pull/759)).
 
 The term "exclude" is now an illegal column name for (timeseries) data and meta tables.
 When importing an xlsx file created with pyam < 2.0, which has an "exclude" column in
 "meta", that column is moved to the new exclude attribute with a log message.
 
+PR [#764](https://github.com/IAMconsortium/pyam/pull/764) implemented a more restrictive
+approach to exposing pyam-internals at the package level, requiring a more explicit
+import of these methods. For example, use `pyam.utils.to_list()`
+instead of `pyam.to_list()`.
+
 ## Individual updates
 
+
+- [#772](https://github.com/IAMconsortium/pyam/pull/772) Show all missing rows for `require_data()`
+- [#771](https://github.com/IAMconsortium/pyam/pull/771) Refactor to start a separate validation module
 - [#766](https://github.com/IAMconsortium/pyam/pull/766) Use **ixmp4** for credentials to access a Scenario Explorer database
+- [#764](https://github.com/IAMconsortium/pyam/pull/764) Clean-up exposing internal methods and attributes
 - [#763](https://github.com/IAMconsortium/pyam/pull/763) Implement a fix against carrying over unused levels when initializing from an indexed pandas object
 - [#759](https://github.com/IAMconsortium/pyam/pull/759) Excise "exclude" column from meta and add a own attribute
-- [#747](https://github.com/IAMconsortium/pyam/pull/747) Drop support for Python 3.7 #747
+- [#747](https://github.com/IAMconsortium/pyam/pull/747) Drop support for Python 3.7
 
 # Release v1.9.0
 

diff --git a/docs/R_tutorials/pyam_R_tutorial.ipynb b/docs/R_tutorials/pyam_R_tutorial.ipynb
@@ -456,7 +456,7 @@
     "## Query data from an IIASA data resource\n",
     "\n",
     "The IIASA *Energy, Climate, and Environment* Program hosts a suite of **Scenario Explorer** instances and related infrastructure to support analysis of integrated-assessment pathways in IPCC reports and model comparison projects. \n",
-    "High-profile use cases include the [IAMC 1.5°C Scenario Explorer hosted by IIASA](https://data.ene.iiasa.ac.at/iamc-1.5c-explorer) supporting the *IPCC Special Report on Global Warming of 1.5°C* (SR15) and the Horizon 2020 project [CD-LINKS](https://data.ene.iiasa.ac.at/cd-links).\n",
+    "High-profile use cases include the [IAMC 1.5°C Scenario Explorer hosted by IIASA](https://data.ece.iiasa.ac.at/iamc-1.5c-explorer) supporting the *IPCC Special Report on Global Warming of 1.5°C* (SR15) and the Horizon 2020 project [CD-LINKS](https://data.ece.iiasa.ac.at/cd-links).\n",
     "\n",
     "The **pyam** package can retrieve scenario data directly from any **Scenario Explorer** database instance\n",
     "hosted by IIASA for use in your processing and analysis workflows."

diff --git a/docs/api/compute.rst b/docs/api/compute.rst
@@ -1,4 +1,4 @@
-.. currentmodule:: pyam
+.. currentmodule:: pyam.compute
 
 Derived timeseries data
 =======================

diff --git a/docs/api/slice.rst b/docs/api/slice.rst
@@ -1,4 +1,4 @@
-.. currentmodule:: pyam
+.. currentmodule:: pyam.slice
 
 The **IamSlice** class
 ======================

diff --git a/docs/api/timeseries.rst b/docs/api/timeseries.rst
@@ -1,4 +1,4 @@
-.. currentmodule:: pyam
+.. currentmodule:: pyam.timeseries
 
 Timeseries functions
 ====================

diff --git a/docs/api/variables.rst b/docs/api/variables.rst
@@ -1,12 +1,11 @@
-.. currentmodule:: pyam
+.. currentmodule:: pyam.str
 
 Variables utilities
 ===================
 
-The **variable** dimension of the |pyam| data format implements
-implements a "semi-hierarchical" structure using the :code:`|` character
-(*pipe*, not l or i) to indicate the *depth*.
-Read the `data model documentation`_ for more information.
+The **variable** dimension of the |pyam| data format implements implements a
+"semi-hierarchical" structure using the :code:`|` character (*pipe*, not l or i)
+to indicate the *depth*. Read the `data model documentation`_ for more information.
 
 .. _`data model documentation`: ../data.html#the-variable-column
 
@@ -16,6 +15,8 @@ The package provides several functions to work with such strings.
 
 .. autofunction:: find_depth
 
+.. autofunction:: get_variable_components
+
 .. autofunction:: reduce_hierarchy
 
-.. autofunction:: get_variable_components
+.. autofunction:: is_str
diff --git a/docs/conf.py b/docs/conf.py
@@ -87,7 +87,7 @@
 
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
-exclude_patterns = []
+exclude_patterns = ["_build", "gallery"]
 
 # The reST default role (used for this markup: `text`) to use for all
 # documents.

diff --git a/docs/data.rst b/docs/data.rst
@@ -28,7 +28,7 @@ of any scenario data provided in this format.
 .. figure:: _static/iamc_template.png
 
    Illustrative example of IAMC-format timeseries data |br|
-   via the `IAMC 1.5°C Scenario Explorer`_ (:cite:`Huppmann:2019:scenario-data`)
+   via the `IAMC 1.5°C Scenario Explorer`_
 
 .. _`IAMC 1.5°C Scenario Explorer`: https://data.ece.iiasa.ac.at/iamc-1.5c-explorer
 

diff --git a/docs/examples/plot_sankey.py b/docs/examples/plot_sankey.py
@@ -18,9 +18,9 @@
 #
 # .. _SR15: http://ipcc.ch/sr15/
 #
-# .. _`IAMC 1.5°C Scenario Explorer and Data hosted by IIASA` : https://data.ene.iiasa.ac.at/iamc-1.5c-explorer
+# .. _`IAMC 1.5°C Scenario Explorer and Data hosted by IIASA` : https://data.ece.iiasa.ac.at/iamc-1.5c-explorer
 #
-# .. _License : https://data.ene.iiasa.ac.at/iamc-1.5c-explorer/#/license
+# .. _License : https://data.ece.iiasa.ac.at/iamc-1.5c-explorer/#/license
 #
 # If you haven't cloned the **pyam** GitHub repository to your machine,
 # you can download the data file from

diff --git a/docs/examples/plot_scatter.py b/docs/examples/plot_scatter.py
@@ -57,7 +57,7 @@
 )
 
 df.set_meta(
-    meta=co2.apply(pyam.cumulative, first_year=2020, last_year=2100, axis=1),
+    meta=co2.apply(pyam.timeseries.cumulative, first_year=2020, last_year=2100, axis=1),
     name="cumulative_co2",
 )
 

diff --git a/docs/index.rst b/docs/index.rst
@@ -84,9 +84,9 @@ a timeseries format for *yearly data* developed and used by the
 .. figure:: _static/iamc_template.png
 
    Illustrative example of IAMC-format timeseries data |br|
-   via the `IAMC 1.5°C Scenario Explorer`_ (:cite:`Huppmann:2019:scenario-data`)
+   via the `IAMC 1.5°C Scenario Explorer`_
 
-.. _`IAMC 1.5°C Scenario Explorer`: https://data.ene.iiasa.ac.at/iamc-1.5c-explorer
+.. _`IAMC 1.5°C Scenario Explorer`: https://data.ece.iiasa.ac.at/iamc-1.5c-explorer
 
 Subannual time resolution
 ^^^^^^^^^^^^^^^^^^^^^^^^^

diff --git a/docs/tutorials/aggregating_downscaling_consistency.ipynb b/docs/tutorials/aggregating_downscaling_consistency.ipynb
@@ -55,7 +55,7 @@
    "outputs": [],
    "source": [
     "import pandas as pd\n",
-    "import pyam"
+    "from pyam import IamDataFrame"
    ]
   },
   {
@@ -73,7 +73,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "df = pyam.IamDataFrame(data='tutorial_data_aggregating_downscaling.csv')"
+    "df = IamDataFrame(data='tutorial_data_aggregating_downscaling.csv')"
    ]
   },
   {
@@ -421,7 +421,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "tutorial_df = pyam.IamDataFrame(pd.DataFrame([\n",
+    "tutorial_df = IamDataFrame(pd.DataFrame([\n",
     "    ['World', 'Primary Energy', 'EJ/yr', 7, 15],\n",
     "    ['World', 'Primary Energy|Coal', 'EJ/yr', 4, 11],\n",
     "    ['World', 'Primary Energy|Wind', 'EJ/yr', 2, 4],\n",
@@ -483,7 +483,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -497,7 +497,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.3"
+   "version": "3.10.0"
   }
  },
  "nbformat": 4,

diff --git a/docs/tutorials/aggregating_variables_and_plotting_with_negative_values.ipynb b/docs/tutorials/aggregating_variables_and_plotting_with_negative_values.ipynb
@@ -18,7 +18,7 @@
     "import numpy as np\n",
     "import pandas as pd\n",
     "\n",
-    "import pyam"
+    "from pyam import IamDataFrame"
    ]
   },
   {
@@ -46,7 +46,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "df = pyam.IamDataFrame(pd.DataFrame([\n",
+    "df = IamDataFrame(pd.DataFrame([\n",
     "    ['IMG', 'a_scen', 'World', 'Emissions|CO2|Energy|Oil', 'Mt CO2/yr', 2, 3.2, 2.0, 1.8],\n",
     "    ['IMG', 'a_scen', 'World', 'Emissions|CO2|Energy|Gas', 'Mt CO2/yr', 1.3, 1.6, 1.0, 0.7],\n",
     "    ['IMG', 'a_scen', 'World', 'Emissions|CO2|Energy|BECCS', 'Mt CO2/yr', 0.0, 0.4, -0.4, 0.3],\n",
@@ -164,7 +164,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "df = pyam.IamDataFrame(pd.DataFrame([\n",
+    "df = IamDataFrame(pd.DataFrame([\n",
     "    ['IMG', 'a_scen', 'World', 'Emissions|CO2', 'Mt CO2/yr', 4.6, 5.3, 5.5, 4.3],\n",
     "    ['IMG', 'a_scen', 'World', 'Emissions|CO2|Fossil', 'Mt CO2/yr', 4.0, 4.6, 4.9, 4.1],\n",
     "    ['IMG', 'a_scen', 'World', 'Emissions|CO2|AFOLU', 'Mt CO2/yr', 0.6, 0.7, 0.6, 0.2],\n",
@@ -235,7 +235,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Even if there are sectors which are defined only at the world level (e.g. `Emissions|CO2|Fossil|Aviation` in our example), Pyam will find them and include them when calculating the regional total if we specify `components=True` when using `aggregate_region`."
+    "Even if there are sectors which are defined only at the world level (e.g. `Emissions|CO2|Fossil|Aviation` in our example), **pyam** will find them and include them when calculating the regional total if we specify `components=True` when using `aggregate_region`."
    ]
   },
   {
@@ -300,7 +300,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -314,7 +314,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.6"
+   "version": "3.10.0"
   }
  },
  "nbformat": 4,

diff --git a/docs/tutorials/algebraic_operations.ipynb b/docs/tutorials/algebraic_operations.ipynb
@@ -50,7 +50,7 @@
    "outputs": [],
    "source": [
     "import pandas as pd\n",
-    "import pyam"
+    "from pyam import IamDataFrame"
    ]
   },
   {
@@ -68,7 +68,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "df = pyam.IamDataFrame(data='tutorial_data_aggregating_downscaling.csv')\n",
+    "df = IamDataFrame(data='tutorial_data_aggregating_downscaling.csv')\n",
     "df"
    ]
   },
@@ -334,7 +334,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -348,7 +348,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.3"
+   "version": "3.10.0"
   }
  },
  "nbformat": 4,