Skip to content

Commit

Permalink
Use resfo for parsing headers in UNRST file
Browse files Browse the repository at this point in the history
Using resdata to ask for headers implies reading the entire file which
requires excessive memory.

Porting more of res2df to use resfo is postponed to a future issue.

Before this commit, obtaining the RST information for one particular
date in a 118GB UNRST file was not possible (requires at least more than
118GB of available memory). With this commit, the same operation can be
done in arund 6 GB of used memory.
  • Loading branch information
berland committed Nov 27, 2024
1 parent 31ee271 commit a303383
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 4 deletions.
3 changes: 3 additions & 0 deletions mypy.ini
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
[mypy-resdata.*]
ignore_missing_imports = True

[mypy-resfo.*]
ignore_missing_imports = True

[mypy-opm.*]
ignore_missing_imports = True

Expand Down
10 changes: 6 additions & 4 deletions res2df/grid.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import pandas as pd
import pyarrow
import pyarrow.feather
import resfo
from resdata.resfile import ResdataFile

from .__version__ import __version__
Expand Down Expand Up @@ -199,11 +200,12 @@ def rst2df(
# for all active cells:
activecells = resdatafiles.get_egrid().getNumActive()
rstvectors = []
for vec in resdatafiles.get_rstfile().headers:
if vec[1] == activecells and any(
fnmatch.fnmatch(vec[0], key) for key in vectors
for vec in resfo.lazy_read(resdatafiles.get_rstfilename()):
keyword_name = vec.read_keyword().strip()
if vec.read_length() == activecells and any(
fnmatch.fnmatch(keyword_name, key) for key in vectors
):
rstvectors.append(vec[0])
rstvectors.append(keyword_name)
rstvectors = list(set(rstvectors)) # Make unique list
# Note that all of these might not exist at all timesteps.

Expand Down
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
SETUP_REQUIREMENTS = ["setuptools>=28", "setuptools_scm"]
REQUIREMENTS = [
"resdata>=5.0.0-b0",
"resfo",
"numpy",
"opm>=2020.10.2",
"pandas",
Expand Down

0 comments on commit a303383

Please sign in to comment.