Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can we have better errors for cross locale Python interop #26710

Open
jabraham17 opened this issue Feb 14, 2025 · 1 comment
Open

Can we have better errors for cross locale Python interop #26710

jabraham17 opened this issue Feb 14, 2025 · 1 comment

Comments

@jabraham17
Copy link
Member

jabraham17 commented Feb 14, 2025

The new Python interop module allows users to call Python code from Chapel, but only on a single locale. Users can pass remote values to Python functions and they will be copied to python, but they can't use the python module cross-locale. This is because Python has no concept of locality.

If users want python on multiple locales, they have to have multiple interpreters per locale, and not share pythons between locales. If users do this, the code will not work (and probably will segfault).

However, because of Chapel's "if its in scope you can use it" model, its really easy to accidentally do this.

The following two programs demonstrate this, where both segfault inside the on block

use Python;
var interp = new Interpreter();
on Locales[1] {
  interp.run("print('Hello, World!')");
}
use Python;
var interp = new Interpreter();
var v = interp.importModule("sys");
writeln(v.str()); // fine
on Locales[1] {
  writeln(v.str()); // not fine
}

This issue is to capture the desire that the Python module help users catch these kinds of mistakes by adding checking that the python values are only used on the locale on which they are created. An alternative is for the Python module to just implicitly move computation to the right locale, but I think this is more likely to cause surprising behavior than be helpful.

@bradcray
Copy link
Member

This is thematically very similar to the case of #21220 in which a user unwittingly used a remote c_string (or, now it would be a c_ptr(c_char) and got a segfault.

My thought/hypothesis here is that we could/should mark certain [C pointer-oriented?] types as being "usable on their home locale only" and to generate an error for them if someone tries to do any sort of dereference of them on a remote locale. Specifically, I wouldn't prevent references to the variable on the remote locale (i.e., it should be legal for me to copy the pointer around), I'm just imagining that anything actually using the value would do a locality check (imagining that the value itself would be wrapped into a wide pointer in such cases) and do some sort of assertion failure if var.locale != here in such cases.

I expect that we'd also need to disable rvf for such types to make sure that remote references were always wide refs and not optimized into local copies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants