Excise "exclude" column from meta and add a own attribute (#759)

RELEASE_NOTES.md

@@ -1,5 +1,20 @@
 # Next Release
 
+The next release must bump the major version number.
+
+## API changes
+
+The column *exclude* of the `meta` attribute was refacored to a new attribute `exclude`.
+All validation methods are refactored such that the argument `exclude_on_fail` changes
+this new attribute (see PR [#759](https://github.com/IAMconsortium/pyam/pull/759)).
+
+The term "exclude" is now an illegal column name for (timeseries) data and meta tables.
+When importing an xlsx file created with pyam < 2.0, which has an "exclude" column in
+"meta", that column is moved to the new exclude attribute with a log message.
+
+## Individual updates
+
+- [#759](https://github.com/IAMconsortium/pyam/pull/759) Excise "exclude" column from meta and add a own attribute
 - [#747](https://github.com/IAMconsortium/pyam/pull/747) Drop support for Python 3.7 #747
 
 # Release v1.9.0

docs/tutorials/pyam_first_steps.ipynb

@@ -101,9 +101,9 @@
     "\n",
     "<div class=\"alert alert-info\">\n",
     "\n",
-    "If you haven't cloned the **pyam** GitHub repository to your machine, you can download the file\n",
-    "from the folder [doc/source/tutorials](https://github.com/IAMconsortium/pyam/tree/master/doc/source/tutorials).  \n",
-    "Make sure to place the file in the same folder as this notebook.\n",
+    "If you haven't cloned the **pyam** GitHub repository to your machine, you can download the data file\n",
+    "from the folder [docs/tutorials](https://github.com/IAMconsortium/pyam/tree/main/docs/tutorials).  \n",
+    "Make sure to place the data file in the same folder as this notebook.\n",
     "\n",
     "</div>"
    ]
@@ -465,9 +465,9 @@
     "When analyzing scenario results, it is often useful to check whether certain timeseries data exist or the values are within a specific range.\n",
     "For example, it may make sense to ensure that reported data for historical periods are close to established reference data or that near-term developments are reasonable.\n",
     "\n",
-    "Before diving into the diagnostics and validation features, we need to briefly introduce the 'meta' table.\n",
+    "Before diving into the diagnostics and validation features, we need to briefly introduce the **meta** attribute.\n",
     "This attribute of an **IamDataFrame** is a [pandas.DataFrame](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html), which can be used to store categorization information and quantitative indicators of each model-scenario.\n",
-    "Per default, a new **IamDataFrame** will contain a column `exclude`, which is set to `False` for all model-scenarios.\n",
+    "In addition, an **IamDataFrame** has an attribubte **exclude**, which is a [pandas.Series](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.html) set to `False` for all model-scenario combinations.\n",
     "\n",
     "The next cell shows the first 10 rows of the 'meta' table."
    ]
@@ -518,7 +518,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "df_world.require_variable(variable='Primary Energy')"
+    "df_world.require_data(variable='Primary Energy')"
    ]
   },
   {
@@ -527,7 +527,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "df_world.require_variable(variable='Primary Energy', year=2100)"
+    "df_world.require_data(variable='Primary Energy', year=2100)"
    ]
   },
   {
@@ -864,8 +864,8 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "As a last step of this illustrative example, we again display the first 10 rows of the 'meta' table for the scenarios in the **IamDataFrame**.\n",
-    "In addition to the `exclude` column seen in cell 20, this table now also includes columns with the three quantitative indicators."
+    "As a last step of this illustrative example, we again display the first 10 rows of the **meta** attribute of the **IamDataFrame**.\n",
+    "The **meta** column seen in cell 20 now includes columns with the three quantitative indicators."
    ]
   },
   {

pyam/_debiasing.py

@@ -2,7 +2,7 @@ def _compute_bias(df, name, method, axis):
     """Internal implementation for computing bias weights"""
     if method == "count":
         # invert from the count to obtain the weighting factor
-        count = 1 / df.meta.groupby(axis).count().exclude
+        count = 1 / df.exclude.groupby(axis).count()
         count.name = name
         df.meta = df.meta.join(count, on=axis, how="outer")
     else: