Cascading and spillover of exception_traceback info all over other expectations

[indresh-singh]

Bug: If one expectation fails (e.g., due to a data type mismatch), the failure prevents the rest of the expectations from being executed successfully. All the expectations in my Great Expectations results object show the same error traceback, which actually corresponds to only one expectation (expect_column_values_to_match_strftime_format)

To Reproduce
Here's the Expectation_suite:

{
"name": "my_expectation_suite",
"id": "ac5bc98b-a950-4111-8fb2-c0e76d8eaf06",
"expectations": [
{
"type": "expect_column_values_to_not_be_null",
"kwargs": {
"catch_exceptions": false,
"column": "sku_id"
},
"meta": {},
"id": "39ce50e4-57d2-4791-909d-c43a0d1d0417"
},
{
"type": "expect_column_values_to_not_be_null",
"kwargs": {
"catch_exceptions": false,
"column": "location_id"
},
"meta": {},
"id": "ded50f35-018e-4e7a-a2f3-b27adc64ba2f"
},
{
"type": "expect_column_values_to_match_regex",
"kwargs": {
"catch_exceptions": false,
"column": "forecast_date",
"regex": "@gmail.com"
},
"meta": {},
"id": "a36daa82-8637-45e0-b3c5-9bb8fefedbab"
},
{
"type": "expect_column_values_to_match_strftime_format",
"kwargs": {
"catch_exceptions": false,
"column": "snapshotdate",
"strftime_format": "%Y-%m-%d"
},
"meta": {},
"id": "58df67c0-77bf-4e2e-bda3-0e07f622baf6"
}
],
"meta": {
"great_expectations_version": "1.2.4"
},
"notes": null
}

Here's the output received (notice how exception_traceback is the same for every expectation, ideally this exception_traceback should be only contained for expectation - expect_column_values_to_match_strftime_format

{
"success": false,
"results": [
{
"success": false,
"expectation_config": {
"type": "expect_column_values_to_not_be_null",
"kwargs": {
"catch_exceptions": false,
"column": "sku_id",
"batch_id": "bronze_wc_forecast-my_dataframe_data_asset"
},
"meta": {},
"id": "39ce50e4-57d2-4791-909d-c43a0d1d0417"
},
"result": {},
"meta": {},
"exception_info": {
"('column_values.nonnull.unexpected_count', 'e8bf7e0f55c1614aa69feae117c3a61e', ())": {
"exception_traceback": "Traceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 545, in _process_direct_and_bundled_metric_computation_configurations\n self.resolve_metric_bundle(metric_fn_bundle=metric_fn_bundle_configurations)\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/sparkdf_execution_engine.py", line 904, in resolve_metric_bundle\n res = df.agg(*aggregate["column_aggregates"]).collect()\n File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 1218, in collect\n sock_info = self._jdf.collectToPython()\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/py4j/java_gateway.py", line 1322, in call\n return_value = get_return_value(\n File "/opt/spark/python/lib/pyspark.zip/pyspark/errors/exceptions/captured.py", line 175, in deco\n raise converted from None\npyspark.errors.exceptions.captured.PythonException: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/validator/validation_graph.py", line 276, in _resolve\n self._execution_engine.resolve_metrics(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 279, in resolve_metrics\n return self._process_direct_and_bundled_metric_computation_configurations(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 549, in _process_direct_and_bundled_metric_computation_configurations\n raise gx_exceptions.MetricResolutionError(\ngreat_expectations.exceptions.exceptions.MetricResolutionError: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n",
"exception_message": "\n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n",
"raised_exception": true
}
}
},
{
"success": false,
"expectation_config": {
"type": "expect_column_values_to_not_be_null",
"kwargs": {
"catch_exceptions": false,
"column": "location_id",
"batch_id": "bronze_wc_forecast-my_dataframe_data_asset"
},
"meta": {},
"id": "ded50f35-018e-4e7a-a2f3-b27adc64ba2f"
},
"result": {},
"meta": {},
"exception_info": {
"('column_values.nonnull.unexpected_count', '5830051c38d53711d10f1b4d04301826', ())": {
"exception_traceback": "Traceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 545, in _process_direct_and_bundled_metric_computation_configurations\n self.resolve_metric_bundle(metric_fn_bundle=metric_fn_bundle_configurations)\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/sparkdf_execution_engine.py", line 904, in resolve_metric_bundle\n res = df.agg(*aggregate["column_aggregates"]).collect()\n File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 1218, in collect\n sock_info = self._jdf.collectToPython()\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/py4j/java_gateway.py", line 1322, in call\n return_value = get_return_value(\n File "/opt/spark/python/lib/pyspark.zip/pyspark/errors/exceptions/captured.py", line 175, in deco\n raise converted from None\npyspark.errors.exceptions.captured.PythonException: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/validator/validation_graph.py", line 276, in _resolve\n self._execution_engine.resolve_metrics(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 279, in resolve_metrics\n return self._process_direct_and_bundled_metric_computation_configurations(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 549, in _process_direct_and_bundled_metric_computation_configurations\n raise gx_exceptions.MetricResolutionError(\ngreat_expectations.exceptions.exceptions.MetricResolutionError: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n",
"exception_message": "\n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n",
"raised_exception": true
}
}
},
{
"success": false,
"expectation_config": {
"type": "expect_column_values_to_match_regex",
"kwargs": {
"catch_exceptions": false,
"column": "forecast_date",
"regex": "@gmail.com",
"batch_id": "bronze_wc_forecast-my_dataframe_data_asset"
},
"meta": {},
"id": "a36daa82-8637-45e0-b3c5-9bb8fefedbab"
},
"result": {},
"meta": {},
"exception_info": {
"('column_values.match_regex.unexpected_count', 'eb97dc1c342c4a0c0b4028e3b0a435d9', '[email protected]')": {
"exception_traceback": "Traceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 545, in _process_direct_and_bundled_metric_computation_configurations\n self.resolve_metric_bundle(metric_fn_bundle=metric_fn_bundle_configurations)\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/sparkdf_execution_engine.py", line 904, in resolve_metric_bundle\n res = df.agg(*aggregate["column_aggregates"]).collect()\n File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 1218, in collect\n sock_info = self._jdf.collectToPython()\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/py4j/java_gateway.py", line 1322, in call\n return_value = get_return_value(\n File "/opt/spark/python/lib/pyspark.zip/pyspark/errors/exceptions/captured.py", line 175, in deco\n raise converted from None\npyspark.errors.exceptions.captured.PythonException: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/validator/validation_graph.py", line 276, in _resolve\n self._execution_engine.resolve_metrics(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 279, in resolve_metrics\n return self._process_direct_and_bundled_metric_computation_configurations(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 549, in _process_direct_and_bundled_metric_computation_configurations\n raise gx_exceptions.MetricResolutionError(\ngreat_expectations.exceptions.exceptions.MetricResolutionError: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n",
"exception_message": "\n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n",
"raised_exception": true
},
"('column_values.nonnull.unexpected_count', 'eb97dc1c342c4a0c0b4028e3b0a435d9', ())": {
"exception_traceback": "Traceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 545, in _process_direct_and_bundled_metric_computation_configurations\n self.resolve_metric_bundle(metric_fn_bundle=metric_fn_bundle_configurations)\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/sparkdf_execution_engine.py", line 904, in resolve_metric_bundle\n res = df.agg(*aggregate["column_aggregates"]).collect()\n File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 1218, in collect\n sock_info = self._jdf.collectToPython()\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/py4j/java_gateway.py", line 1322, in call\n return_value = get_return_value(\n File "/opt/spark/python/lib/pyspark.zip/pyspark/errors/exceptions/captured.py", line 175, in deco\n raise converted from None\npyspark.errors.exceptions.captured.PythonException: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/validator/validation_graph.py", line 276, in _resolve\n self._execution_engine.resolve_metrics(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 279, in resolve_metrics\n return self._process_direct_and_bundled_metric_computation_configurations(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 549, in _process_direct_and_bundled_metric_computation_configurations\n raise gx_exceptions.MetricResolutionError(\ngreat_expectations.exceptions.exceptions.MetricResolutionError: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n",
"exception_message": "\n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n",
"raised_exception": true
}
}
},
{
"success": false,
"expectation_config": {
"type": "expect_column_values_to_match_strftime_format",
"kwargs": {
"catch_exceptions": false,
"column": "snapshotdate",
"strftime_format": "%Y-%m-%d",
"batch_id": "bronze_wc_forecast-my_dataframe_data_asset"
},
"meta": {},
"id": "58df67c0-77bf-4e2e-bda3-0e07f622baf6"
},
"result": {},
"meta": {},
"exception_info": {
"('column_values.match_strftime_format.unexpected_values', 'e15033a11125fa0bba03afdbfd94e870', '71ebb60c1d1cd456f6ac2098748e502f')": {
"exception_traceback": "Traceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 532, in _process_direct_and_bundled_metric_computation_configurations\n metric_computation_configuration.metric_fn( # type: ignore[misc] # F not callable\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/map_metric_provider/column_map_condition_auxilliary_methods.py", line 309, in _spark_column_map_condition_values\n return [row[column_name] for row in query.collect()]\n File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 1218, in collect\n sock_info = self._jdf.collectToPython()\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/py4j/java_gateway.py", line 1322, in call\n return_value = get_return_value(\n File "/opt/spark/python/lib/pyspark.zip/pyspark/errors/exceptions/captured.py", line 175, in deco\n raise converted from None\npyspark.errors.exceptions.captured.PythonException: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/validator/validation_graph.py", line 276, in _resolve\n self._execution_engine.resolve_metrics(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 279, in resolve_metrics\n return self._process_direct_and_bundled_metric_computation_configurations(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 537, in _process_direct_and_bundled_metric_computation_configurations\n raise gx_exceptions.MetricResolutionError(\ngreat_expectations.exceptions.exceptions.MetricResolutionError: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n",
"exception_message": "\n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n",
"raised_exception": true
},
"('column_values.nonnull.unexpected_count', 'e15033a11125fa0bba03afdbfd94e870', ())": {
"exception_traceback": "Traceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 545, in _process_direct_and_bundled_metric_computation_configurations\n self.resolve_metric_bundle(metric_fn_bundle=metric_fn_bundle_configurations)\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/sparkdf_execution_engine.py", line 904, in resolve_metric_bundle\n res = df.agg(*aggregate["column_aggregates"]).collect()\n File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 1218, in collect\n sock_info = self._jdf.collectToPython()\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/py4j/java_gateway.py", line 1322, in call\n return_value = get_return_value(\n File "/opt/spark/python/lib/pyspark.zip/pyspark/errors/exceptions/captured.py", line 175, in deco\n raise converted from None\npyspark.errors.exceptions.captured.PythonException: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/validator/validation_graph.py", line 276, in _resolve\n self._execution_engine.resolve_metrics(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 279, in resolve_metrics\n return self._process_direct_and_bundled_metric_computation_configurations(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 549, in _process_direct_and_bundled_metric_computation_configurations\n raise gx_exceptions.MetricResolutionError(\ngreat_expectations.exceptions.exceptions.MetricResolutionError: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n",
"exception_message": "\n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n",
"raised_exception": true
},
"('column_values.match_strftime_format.unexpected_count', 'e15033a11125fa0bba03afdbfd94e870', 'strftime_format=%Y-%m-%d')": {
"exception_traceback": "Traceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 545, in _process_direct_and_bundled_metric_computation_configurations\n self.resolve_metric_bundle(metric_fn_bundle=metric_fn_bundle_configurations)\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/sparkdf_execution_engine.py", line 904, in resolve_metric_bundle\n res = df.agg(*aggregate["column_aggregates"]).collect()\n File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 1218, in collect\n sock_info = self._jdf.collectToPython()\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/py4j/java_gateway.py", line 1322, in call\n return_value = get_return_value(\n File "/opt/spark/python/lib/pyspark.zip/pyspark/errors/exceptions/captured.py", line 175, in deco\n raise converted from None\npyspark.errors.exceptions.captured.PythonException: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/validator/validation_graph.py", line 276, in _resolve\n self._execution_engine.resolve_metrics(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 279, in resolve_metrics\n return self._process_direct_and_bundled_metric_computation_configurations(\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/execution_engine/execution_engine.py", line 549, in _process_direct_and_bundled_metric_computation_configurations\n raise gx_exceptions.MetricResolutionError(\ngreat_expectations.exceptions.exceptions.MetricResolutionError: \n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n\n",
"exception_message": "\n An exception was thrown from the Python worker. Please see the stack trace below.\nTraceback (most recent call last):\n File "/home/trusted-service-user/cluster-env/clonedenv/lib/python3.10/site-packages/great_expectations/expectations/metrics/column_map_metrics/column_values_match_strftime_format.py", line 55, in is_parseable_by_format\n raise TypeError( # noqa: TRY003\nTypeError: Values passed to expect_column_values_to_match_strftime_format must be of type string.\nIf you want to validate a column of dates or timestamps, please call the expectation before converting from string format.\n",
"raised_exception": true
}
}
}
],
"suite_name": "my_expectation_suite",
"suite_parameters": {},
"statistics": {
"evaluated_expectations": 4,
"successful_expectations": 0,
"unsuccessful_expectations": 4,
"success_percent": 0.0
},
"meta": {
"great_expectations_version": "1.2.4",
"batch_spec": {
"batch_data": "SparkDataFrame"
},
"batch_markers": {
"ge_load_time": "20241129T085618.749891Z"
},
"active_batch_definition": {
"datasource_name": "bronze_wc_forecast",
"data_connector_name": "fluent",
"data_asset_name": "my_dataframe_data_asset",
"batch_identifiers": {
"dataframe": ""
}
},
"validation_id": "287d27f5-2581-4471-a0f8-541013889773",
"checkpoint_id": null,
"batch_parameters": {
"dataframe": ""
}
},
"id": null
}

Expected behavior
Exception_traceback with the errors should be contained only to the erroneous expectation: column_values_match_strftime_format instead of spilling over to exception_traceback of all other expectations as well.

Environment:

• Operating System: MacOS
• Great Expectations Version: 1.2.4
• Data Source: Dataframe
• Cloud environment: MS Fabric

[adeola-ak]

hello, thank you for bringing this to our attention. I've shared it with the team and I will follow up on this issue with any updates

[indresh-singh]

Hey @adeola-ak , is there an update on the above issue ?

[adeola-ak]

Hi @indresh-singh,

We’ve identified that this issue is related to how our metrics are computed. I can confirm that it’s on our backlog to address. Unfortunately, I don’t have a specific timeline at the moment, but I’ll follow up with my team and keep you updated.

[victorgrcp]

I found the same error, but in my case the error was caused by ExpectColumnValuesToBeInSet and propagate over the rest of the Expectations of the Suite but for UnexpectedRowsExpectation ones.

I will be attentive to the update.