Errors around Datalabs

[odashi]

Encountered following test errors just a minutes ago. @pfliu-nlp

======================================================================
ERROR: test_datalab_loader (integration_tests.argument_pair_extraction_test.ArgumentPairExtractionTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "~/ExplainaBoard/integration_tests/argument_pair_extraction_test.py", line 24, in test_datalab_loader
    data = loader.load()
  File "~/ExplainaBoard/explainaboard/loaders/loader.py", line 154, in load
    dataset_loaded_data = self._dataset_file_loader.load(
  File "~/ExplainaBoard/explainaboard/loaders/file_loader.py", line 335, in load
    raw_data = self.load_raw(data, source)
  File "~/ExplainaBoard/explainaboard/loaders/file_loader.py", line 567, in load_raw
    dataset = load_dataset(
  File "~/ExplainaBoard/venv/lib/python3.10/site-packages/datalabs/load.py", line 2093, in load_dataset
    builder_instance = load_dataset_builder(
  File "~/ExplainaBoard/venv/lib/python3.10/site-packages/datalabs/load.py", line 1888, in load_dataset_builder
    builder_instance: DatasetBuilder = builder_cls(
  File "~/ExplainaBoard/venv/lib/python3.10/site-packages/datalabs/builder.py", line 1273, in __init__
    super(GeneratorBasedBuilder, self).__init__(*args, **kwargs)
  File "~/ExplainaBoard/venv/lib/python3.10/site-packages/datalabs/builder.py", line 331, in __init__
    info.update(self._info())
  File "~/.cache/expressai/modules/datasets_modules/datalab/ape/6a47b551465788cfbfe4158930982d8dc72e77f95d45301a122e560683f1a9b9/ape.py", line 67, in _info
    get_task(TaskType.argument_pair_extraction)(
  File "~/.pyenv/versions/3.10.6/lib/python3.10/enum.py", line 437, in __getattr__
    raise AttributeError(name) from None
AttributeError: argument_pair_extraction

======================================================================
ERROR: test_datalab_loader (integration_tests.qa_table_text_hybrid_test.QATableTextHybridTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "~/ExplainaBoard/integration_tests/qa_table_text_hybrid_test.py", line 23, in test_datalab_loader
    data = loader.load()
  File "~/ExplainaBoard/explainaboard/loaders/loader.py", line 154, in load
    dataset_loaded_data = self._dataset_file_loader.load(
  File "~/ExplainaBoard/explainaboard/loaders/file_loader.py", line 335, in load
    raw_data = self.load_raw(data, source)
  File "~/ExplainaBoard/explainaboard/loaders/file_loader.py", line 567, in load_raw
    dataset = load_dataset(
  File "~/ExplainaBoard/venv/lib/python3.10/site-packages/datalabs/load.py", line 2093, in load_dataset
    builder_instance = load_dataset_builder(
  File "~/ExplainaBoard/venv/lib/python3.10/site-packages/datalabs/load.py", line 1888, in load_dataset_builder
    builder_instance: DatasetBuilder = builder_cls(
  File "~/ExplainaBoard/venv/lib/python3.10/site-packages/datalabs/builder.py", line 1273, in __init__
    super(GeneratorBasedBuilder, self).__init__(*args, **kwargs)
  File "~/ExplainaBoard/venv/lib/python3.10/site-packages/datalabs/builder.py", line 331, in __init__
    info.update(self._info())
  File "~/.cache/expressai/modules/datasets_modules/datalab/tat_qa/044657d0b6a5d09233115746f2037ec9db48204e0f7b238cf965e19fced7052c/tat_qa.py", line 130, in _info
    task_templates=get_task(TaskType.qa_table_text_hybrid)(
  File "~/.pyenv/versions/3.10.6/lib/python3.10/enum.py", line 437, in __getattr__
    raise AttributeError(name) from None
AttributeError: qa_table_text_hybrid

----------------------------------------------------------------------

[pfliu-nlp]

I will take a look

[pfliu-nlp]

Hi, @odashi could you check along following steps:

(1) check the version of datalabs first: pip show dastalabs
these two tasks are supported by datalabs >= 0.4.9

(2) check if the dataset fromargument_pair_extraction could be loaded locally

from datalabs import load_dataset
dataset = load_dataset("ape")

If yes, I think (3) should work.

(3)

python -m unittest integration_tests.argument_pair_extraction_test.ArgumentPairExtractionTest

[odashi]

@pfliu-nlp Thanks, it looks the tests now works as expected. Let me close this issue.

these two tasks are supported by datalabs >= 0.4.9

It seems to cause a versioning problem (also discussed in #403): ExplainaBoard cannot add datalab-specific tasks without upgrading the library and code.

Since you are leading Datalab, I think you may also need to introduce some version agnostic way to manage supported tasks in that library.

[pfliu-nlp]

"Since you are leading Datalab, I think you may also need to introduce some version agnostic way to manage supported tasks in that library."

this is a good question. I have created an issue and will think about it.