Add compile_and_run convenience function #54
Comments
|
I guess you'd prefer a single function, but currently two functions can compile and run a parsed A possible way to use the functions is in the updated version of the mccode-antlr/tests/runtime/compiled.py Lines 85 to 97 in 99a5cd9 Would this work for you? If so, I can pull together a release for PyPI. |
|
For sure, that version of compile_and_run is much shorter, so for the workshop next week it should be good enough! But thinking aloud... perhaps I would still love to be spoiled a bit more at some point in the future, i.e. have a "mcstas_run" function, which would automatically use the MCSTAS generator, and accept an "instrument" (i.e. a string, file, or an already-parsed instrument), and also accept a working directory argument (since in general the results should probably not be cleaned up before the function exits... i guess). So Ideally I would love to have code like: from mccode_antler.whatever import mcstas_run
myinstr="""
BLA SOME INSTRUMENT CODE HERE
"""
mcstas_run(myinstr,'./somedir')
#or:
with TemporaryDirectory() as d:
mcstas_run(myinstr,d.joinpath('somedir'))
#analyse output here
Does that make sense? However, there might be stuff I am not thinking about and so forth, so perhaps it is best that for the short term I simply try and use the few functions you already provide? |
|
If you can come up with a sensible way to handle directories, and want to write the logic to take 3+ types of input as a positional argument, this could be a useful addition. The directory scheme should probably match (or at least resemble) the McCode behavior, where the working directory is always the current directory, the binary ends up named after the instrument file, and the runtime output artifacts end up in a newly-created folder either specified as a command line option or named after the instrument file and current time. |
|
Sounds good to me, let us discuss it in person next time we have a chance! In the meantime, for my workshop, I was trying to add a small utility function at the top of my google colab notebooks: def mcstas_run( instr, rundir = None ):
import pathlib
import tempfile
from mccode_antlr.translators.target import MCSTAS_GENERATOR as GEN
from mccode_antlr.run import mccode_compile, mccode_run_compiled
rundir = pathlib.Path( tempfile.mkdtemp('mcstas_rundir_')
if rundir is None else rundir )
if not rundir.is_dir():
raise FileNotFoundError("Directory not found: %s"%rundir_base)
binary, target = mccode_compile(instr, rundir, generator=GEN)
mccode_run_compiled(binary, target, rundir)
return rundirHowever, I didn't get to really test it yet since I get the error: Does that mean that I have to wait for you to make a new PyPI package first? |
|
Actually, I guess I can just pip install straight from your repo to try it out. |
|
Ok, this seemed to work: I will try it out now. |
|
Ok, so this went better. I had issues with the def mcstas_run( instr, rundir = None ):
if hasattr(instr,'__fspath__') or not hasattr(instr,'name'):
from mccode_antlr.loader import parse_mcstas_instr
instr = parse_mcstas_instr(instr)
import pathlib
import tempfile
from mccode_antlr.translators.target import MCSTAS_GENERATOR as GEN
from mccode_antlr.run import mccode_compile, mccode_run_compiled
rundir = pathlib.Path( tempfile.mkdtemp('mcstas_rundir_')
if rundir is None else rundir )
if not rundir.is_dir():
raise FileNotFoundError("Directory not found: %s"%rundir_base)
binary, target = mccode_compile(instr, rundir, generator=GEN)
mccode_run_compiled(binary, target, rundir)
return rundirNext problem is that it seems to hang after the following output. Do I need to somehow pass in a different number of sr neutrons? Actually, how DO I pass in parameters? Do you have an example? |
|
For reference, this was my instrument file (actually, in a string): |
|
It's good that installing from the git branch of the repository works 😆
The binary runtime input parameters should be specified as a string, as in the test helper function I'm a bit surprised that your call to At the moment it's up to you to know what parameters your binary will accept, and the standard McCode compiled instrument parameters are of course allowed. Note You probably don't want to specify Warning Your snippet will raise an error because you're passing the working directory |
|
@tkittel You should now be able to install |
|
Great! I will most likely have to continue my attempts tomorrow. But when you say that McStas refuses something, isn't that in your code now? Or is it in the c-files? |
|
So I still can't get this to run. It seems to hang during compilation (or just takes longer than my attention span). For reference, classic mcrun parses, compiles, and runs in <1sec. Here is a standalone script showing my problem: #!/usr/bin/env python3
#Needs "pip install mccode_antlr ncrystal"
_test_instr="""
DEFINE INSTRUMENT example2()
TRACE
COMPONENT origin = Progress_bar()
AT (0, 0, 0) RELATIVE ABSOLUTE
COMPONENT source = Source_div(lambda0=1.539739, dlambda=0.01, xwidth=0.001, yheight=0.001, focus_aw=1, focus_ah=1)
AT (0, 0, 0.3) RELATIVE origin
COMPONENT mysample = NCrystal_sample(cfg="Al2O3_sg167_Corundum.ncmat",yheight=0.01,radius=0.01)
AT (0, 0, 0) RELATIVE PREVIOUS
COMPONENT powder_pattern_detc = Monitor_nD(
options = "banana, angle limits=[10 170], bins=500",
radius = 0.05, yheight = 0.1)
AT (0, 0, 0) RELATIVE PREVIOUS
END
"""
def mcstas_run( instr, rundir = None ):
if hasattr(instr,'__fspath__') or not hasattr(instr,'name'):
from mccode_antlr.loader import parse_mcstas_instr
instr = parse_mcstas_instr(instr)
import pathlib
import tempfile
from mccode_antlr.translators.target import MCSTAS_GENERATOR as GEN
from mccode_antlr.run import mccode_compile, mccode_run_compiled
rundir = pathlib.Path( tempfile.mkdtemp('mcstas_rundir_')
if rundir is None else rundir )
if not rundir.is_dir():
raise FileNotFoundError("Directory not found: %s"%rundir)
for i in range(1,1000000):
#Quick and dirty solution, to be changed:
actual_rd = rundir / f'mcstasrun_{i}'
if not actual_rd.exists():
break
binary, target = mccode_compile(instr, rundir, generator=GEN)
mccode_run_compiled(binary, target, actual_rd, parameters='-n 1000')
return actual_rd
def main():
mcstas_run( _test_instr, './lala' )
if __name__=='__main__':
main() |
|
Hmm... investigating a bit further, it seems to only hang when NCrystal_sample is present. Could it be the FUNNEL (non-gpu) mode? |
The runtime refuses, see if(mkdir(dirname, 0777)) {
#ifndef DANSE
fprintf(stderr, "Error: unable to create directory '%s' (mcuse_dir)\n", dir);
fprintf(stderr, "(Maybe the directory already exists?)\n"); |
|
I haven't used |
|
I thought maybe it could be stuck working-out circular includes of, e.g., The |
|
Yes, and I see the compilation command picks up the correct flags from it... |
|
It's somehow related to Using the cc -g -O2 -D_POSIX_C_SOURCE -std=c99 example2.c -lm -Wl,-rpath,$(ncrystal-config --show libdir) $(ncrystal-config --show libpath) -I$(ncrystal-config --show includedir) -DFUNNELand cc -g -O2 -D_POSIX_C_SOURCE -std=c99 -x c - -lm -Wl,-rpath,$(ncrystal-config --show libdir) $(ncrystal-config --show libpath) -I$(ncrystal-config --show includedir) -DFUNNEL < example2.cThe first works fine, the second will fill your terminal with all sorts of warnings about stray undefined escape sequences and null characters in what looks like a binary blob in your shared library. I don't know whether using a stdin pipe for the source code is prone to this sort of problem, or if there is something special about the NCrystal shared library. Have you seen this before? It is somehow independent of whether the library is even used. With a very simple #include <stdio.h>
int main(){
printf("Hello world\n");
return 0;
}cc -g -O2 -D_POSIX_C_SOURCE -std=c99 -x c - -lm -Wl,-rpath,$(ncrystal-config --show libdir) $(ncrystal-config --show libpath) -I$(ncrystal-config --show includedir) -DFUNNEL < main.cproduces exactly the same output to stderr |
|
The use of $ ncrystal-config --show libdir
./lib
$ ncrystal-config --show libpath
./lib/libNCrystal.so.3.9.5
$ ncrystal-config --show includedir
./includeand dropping the compiler flags for the moment, the compilation command looks like $ cc ... -x c - -lm -Wl,-rpath,./lib ./lib/libNCrystal.so.3.9.5 -I./include -DFUNNEL < main.cWe can, for the moment, behave like gcc and ignore any $ cc ... -x c - ./lib/libNCrystal.so.3.9.5 -I./include -DFUNNEL < main.cso its parsing the shared object file because it appears like a source on the command line. Usefully, it's possible to specify the location $ cc ... -x c - -lm -Wl,-rpath,./lib -L./lib -l:libNCrystal.so.3.9.5 -I./include -DFUNNEL < main.cand then it compiles as expected without any attempted parsing of the shared library as code. Is there a reason that the NCrystal |
|
Ah, so it was compiling the shared library! Sick that it didn't just error out :-) So I am a bit confused about the relative paths you get from Now, on one hand should you not try to mimic the classic mccode order? But on the other hand, I agree we should try to make it more robust in any case... So |
|
So for testing, how does I tell mccode_antlr to use a locally edited |
Drop it in the same directory where you call mccode-antlr/mccode_antlr/commands.py Lines 43 to 74 in a56e48c And how its passed to the parser mccode-antlr/mccode_antlr/loader/loader.py Lines 16 to 28 in a56e48c |
It is absolute, I just shortened the paths for clarity.
No, the order is the same. The difference is I pass the C source to the compiler via a pipe instead of writing a file (and then, currently, also write a file for debugging)
I just wasn't sure if, say clang, or MSVC, would handle this too |
|
Clang is very similar to gcc, but MSVC probably in any case needs a completely different way of putting the flags. I actually remembered, the solution is most likely just that I prepend I was trying now with a local |
As we discussed, it would be great to be essentially able to parse, compile, and run a sample .instr file in a few lines of code (i.e. in a notebook as part of a tutorial). The function
compile_and_runthat I was kindly pointed at in https://github.com/McStasMcXtrace/mccode-antlr/blob/main/tests/runtime/compiled.py#L85 looks like something that would be useful as a general utility function.But possibly taking the directory as an argument, instead of the TemporaryDirectory thing (which the user could do in their calling code if needed)?
The text was updated successfully, but these errors were encountered: