Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【Codon】 When a process that consumes a large amount of memory is executed, it is killed and the process is stopped. #633

Open
icst005 opened this issue Mar 5, 2025 · 1 comment

Comments

@icst005
Copy link

icst005 commented Mar 5, 2025

I was not sure if there was a report applicable to the same case, so I am reporting it just in case.

When a process that consumes a large amount of memory (test04.py) is executed, it is killed and the process stops.
When the same code is executed in Python, the process completes normally.

[Execution Result]
-test04.py Execution result with Codon
ic@DESKTOP-C83DAGF:/mnt/c/Users/ic$ codon run -release /mnt/z/koko/ps3/cd/test04_codon.py

kaishi
test04_codon
from python import numpy as np
Killed
ic@DESKTOP-C83DAGF:/mnt/c/Users/ic$

-test04.py Execution result with Python

kaishi
test04_codon
from python import numpy as np
0
1
2
3

 ・・・

9997
9998
2367.8824689388275
test04_python
import numpy as np
owari

[Execution Procedure]
Execute the following from Windows PowerShell Terminal on Windows in the following order.

wsl -d Ubuntu-22.04
codon run -release /mnt/z/koko/ps3/cd/test04.py
(Python, with Anaconda, running on JupyterNotebook)

[Execution environment]
Codon Ver: 0.18
Python Ver: 3.10.8
Windows 10 Pro 22H2(WSL Ubuntu 22.04)
Windows Feature Experience Pack 1000.19055.1000.0
Core(TM) i7-7700HQ
Memory 64GB

[Sampl Code]
-test04.py
from python import numpy as np #------------①
from python import pandas as pd
from python import csv
import time
from python import random

print("kaishi")
print("test04_codon")
print("from python import numpy as np")

t00 = time.time()

for aa0 in range(9999):

print(aa0)

m0 = 9000000
np1 = np.zeros((m0, 1), dtype=np.int64)

np.random.seed()
np1[0: m0, 0] = np.random.randint(0, 99999, m0)

np1 = None

t01 = time.time()
t02 = t01 - t00

print(t02)
print("test04_codon")
print("from python import numpy as np")
print("owari")

@ghuls
Copy link

ghuls commented Mar 5, 2025

I assume this is because the python objects still exsit and each time your assign a numpy expression to np1, a different python object gets bound to np1 (which is only a variable in codon and not in python).

Putting this code in a python function, does not blow up memory.

from python import numpy as np #------------①
from python import pandas as pd
from python import csv
import time
from python import random

@python
def random_int(aa0):
    import numpy as np
    print(aa0)

    m0 = 9000000
    np1 = np.zeros((m0, 1), dtype=np.int64)

    np.random.seed()
    np1[0: m0, 0] = np.random.randint(0, 99999, m0)

    np1 = None

print("kaishi")
print("test04_codon")
print("from python import numpy as np")

t00 = time.time()

for aa0 in range(9999):
    random_int(aa0)

t01 = time.time()
t02 = t01 - t00

print(t02)
print("test04_codon")
print("from python import numpy as np")

On a side note, calling del on the object seems not supported:

from python import numpy as np #------------①
from python import pandas as pd
from python import csv
import time
from python import random

print("kaishi")
print("test04_codon")
print("from python import numpy as np")

t00 = time.time()

for aa0 in range(9999):
    print(aa0)

    m0 = 9000000
    np1 = np.zeros((m0, 1), dtype=np.int64)

    np.random.seed()
    np1[0: m0, 0] = np.random.randint(0, 99999, m0)

    del np1
    #np1 = None

t01 = time.time()
t02 = t01 - t00

print(t02)
print("test04_codon")
print("from python import numpy as np")
print("owari")
$ codon run --release test_memory_codon.py
test04.py:22:5-12: error: 'pyobj' object has no method '__init__' with arguments (pyobj)
Command exited with non-zero status 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants