You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
When executing a blockwise operations in mars which have many setitem/getitem nodes, mars will take about 1 minutes, which is too long.
To Reproduce
To help us reproducing this bug, please provide information below:
Your Python version: 3.7.9
The version of Mars you use: master
Versions of crucial packages, such as numpy, scipy and pandas
Full stack of the error.
Minimized code to reproduce the error.
importmathdf=md.DataFrame(
mt.random.rand(120_0000, 70, chunk_size=5000),
columns=[f"col{i}"foriinrange(70)])
forcinrange(70):
df[f"col{i+70}"] =df[f"col{i}"].fillna(0)
df[f"col{i+140}"] =df[f"col{i}"].fillna(0)
forcinrange(70):
df[f"col{i}"] =df[f"col{i}"]/100df=df.fillna(0)
cols=df.columns.to_pandas().valuesdf=df[cols[:-1]]
df=df.apply(lambdax: x, axis=1)
df=df.replace('NaN', np.nan) # replace string NaN to numpydf=df.replace(math.nan, np.nan) # replace string NaN to numpydf=df.fillna(value=np.nan) # replace none, null to numpydf.map_chunk(lambdax:x).execute()
Expected behavior
The graph building time should be less than 3 seconds
The text was updated successfully, but these errors were encountered:
Describe the bug
When executing a blockwise operations in mars which have many setitem/getitem nodes, mars will take about 1 minutes, which is too long.
To Reproduce
To help us reproducing this bug, please provide information below:
Your Python version: 3.7.9
The version of Mars you use: master
Versions of crucial packages, such as numpy, scipy and pandas
Full stack of the error.
Minimized code to reproduce the error.
Expected behavior
The graph building time should be less than 3 seconds
The text was updated successfully, but these errors were encountered: