Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

多线程频繁进行ts pro初始化概率返回EmptyDataError,建议pandas操作文件能够主动释放文件句柄 #1727

Open
samuel-lyu opened this issue Jan 22, 2024 · 0 comments

Comments

@samuel-lyu
Copy link

samuel-lyu commented Jan 22, 2024

当频繁多线程操作时,存在一定概率消耗完句柄返回异常:pandas.errors.EmptyDataError: No columns to parse from file

  File "/usr/local/lib/python3.10/site-packages/tushare/pro/data_pro.py", line 36, in pro_api
    token = upass.get_token()
  File "/usr/local/lib/python3.10/site-packages/tushare/util/upass.py", line 27, in get_token
    df = pd.read_csv(fp)
  File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 912, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 577, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1407, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1679, in _make_engine
    return mapping[engine](f, **self.options)
  File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 93, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 557, in pandas._libs.parsers.TextReader.__cinit__  File "/usr/local/lib/python3.10/site-packages/tushare/pro/data_pro.py", line 36, in pro_api
    token = upass.get_token()
  File "/usr/local/lib/python3.10/site-packages/tushare/util/upass.py", line 27, in get_token
    df = pd.read_csv(fp)
  File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 912, in read_csv
    return _read(filepath_or_buffer, kwds)
  File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 577, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1407, in __init__
    self._engine = self._make_engine(f, self.engine)
  File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/readers.py", line 1679, in _make_engine
    return mapping[engine](f, **self.options)
  File "/usr/local/lib/python3.10/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 93, in __init__
    self._reader = parsers.TextReader(src, **kwds)
  File "pandas/_libs/parsers.pyx", line 557, in pandas._libs.parsers.TextReader.__cinit__

建议upass模块中to_csv和read_csv操作建议增加文件with上下文管理主动关闭句柄
例如:

with open(fp, mode='r') as f:
    df = pd.read_csv(p)
@samuel-lyu samuel-lyu changed the title 多线程频繁进行ts pro初始化概率返回EmptyDataError,建议pandas操作文件能够主动关闭文件句柄 多线程频繁进行ts pro初始化概率返回EmptyDataError,建议pandas操作文件能够主动释放文件句柄 Jan 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant