You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/using_eessi/building_on_eessi.md
+27-63Lines changed: 27 additions & 63 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -114,38 +114,22 @@ This means you'll _always_ need to load the `EESSI-extend` module if you want to
114
114
115
115
!!! warning
116
116
117
-
We are working on a module file that should make building on top of EESSI (without using EasyBuild)
118
-
more straightforward, particularly when using `Autotools` or `CMake`. Right now, it is a little convoluted
119
-
and requires you to have a decent grasp of
120
-
* What a runtime dynamic linker (`ld-linux*.so`) is and does
121
-
* How to influence the behaviour of the runtime linker with `LD_LIBRARY_PATH`
122
-
* The difference between `LIBRARY_PATH` and `LD_LIBRARY_PATH`
123
-
124
-
As such, this documentation is intended for "experts" in the runtime linker and it's behaviour,
117
+
Even with the help of the `buildenv` module, this documentation is intended for "experts" in the runtime linker and it's behaviour,
125
118
and most cases are untested. Any feedback on this topic is highly appreciated.
126
119
127
-
Building and running software on top of EESSI without EasyBuild is not straightforward and requires some considerations to take care of.
128
-
129
-
It is expected that you will have loaded all of your required dependencies as modules from the EESSI environment. Since EESSI sets
130
-
`LIBRARY_PATH` for all of the modules and the `GCC` compiler is configured to use the compat layer, there should be no additional configuration
131
-
required to execute a standard build process. On the other hand, EESSI does not set `LD_LIBRARY_PATH` so, _at runtime_, the executable will need help
132
-
finding the libraries that it needs to actually execute. The easiest way to circumvent this requirement is by setting the environment variable `LD_RUN_PATH`
133
-
during compile time as well. With `LD_RUN_PATH` set, the program will be able to tell the dynamic linker to search in those paths when the program is being
134
-
executed.
120
+
Building and running software on top of EESSI without EasyBuild is now more straightforward thanks to the freshly developed `buildenv`module.
135
121
136
-
EESSI uses a [compatibility layer](../compatibility_layer.md) to ensure that it takes as few libraries from the host as possible. The safest way to make sure
137
-
all libraries will point to the required locations in the compatibility layer (and do not leak in from the host operating system) is starting an EESSI prefix
138
-
shell before building. To do this:
122
+
EESSI uses a [compatibility layer](../compatibility_layer.md) to ensure that it takes as few libraries from the host as possible. Starting an EESSI prefix
123
+
shell before building and loading any of the `buildenv` modules (one per `foss` toolchain), this is the safest way to make sure all libraries will point to the required locations in the compatibility layer and do not leak in from the host operating system. A step by step:
139
124
140
125
* First of all, load the environment by starting an EESSI shell as described [here](https://www.eessi.io/docs/using_eessi/setting_up_environment).
141
-
* Load all dependencies you need to build your software. You must use at least a toolchain from EESSI to compile it (`foss` is a good option as it will also
* Load one of the `buildenv` modules, there's one per `foss` toolchain available in EESSI. (`foss` is a good option as it will also
142
130
include MPI with OpenMPI and math libraries via FlexiBLAS/FFTW).
143
-
* Set manually `LD_RUN_PATH` to resolve libraries at runtime. `LIBRARY_PATH` should contain all the paths we need, and we also need to include the path to
144
-
`libstdc++` from our GCC installation to avoid picking up the one from the host:
* Compile and make sure the library resolution points to the EESSI stack. For this, `ldd` from compatibility layer and **not**`/usr/bin/ldd` should be used
131
+
* Compile as you would do as usual. The module includes wrappers for gcc, gfortran and linkers.
132
+
* Make sure the library resolution points to the EESSI stack. For this, `ldd` from compatibility layer and **not**`/usr/bin/ldd` should be used
149
133
when checking the binary.
150
134
151
135
* Run!
@@ -159,7 +143,7 @@ To exemplify this, take the classic MPI Hello World example code:
159
143
160
144
int main(int argc, char **argv)
161
145
{
162
-
int node;
146
+
int rank;
163
147
164
148
MPI_Init(&argc,&argv);
165
149
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
@@ -171,22 +155,19 @@ int main(int argc, char **argv)
171
155
172
156
```
173
157
174
-
As described in the steps above, prepare the environment and load the required dependencies. For this case, we will use `gompi/2023b`as the toolchain to compile it.
158
+
As described in the steps above, prepare the environment and load the required dependencies. For this case, we will use `buildenv/default-foss-2023b`module.
@@ -199,47 +180,30 @@ Look at the difference on the library solving when using the compatibility layer
199
180
200
181
# ldd from the compatibility layer, notice how all libraries are resolved from the software layer -> "/cvmfs/software.eessi.io/versions/2023.06/software", the libc and the interpreter point to the compatibility layer, so all good to go!
# ldd from the host, even though the libraries point to the software layer, now the linker ld-linux-x86-64.so.2 from the compat layer directly points to "/lib64/ld-linux-x86-64.so.2" from the host to do the resolving, resulting in the GLIBC mismatch as libc is also resolved in the host and not the compat layer
216
-
217
-
{EESSI 2023.06} $ /usr/bin/ldd HelloWorld
218
-
./HelloWorld: /lib64/libc.so.6: version `GLIBC_2.36' not found (required by /cvmfs/software.eessi.io/versions/2023.06/software/linux/x86_64/intel/skylake_avx512/software/libevent/2.1.12-GCCcore-13.2.0/lib64/libevent_core-2.1.so.7)
219
-
./HelloWorld: /lib64/libc.so.6: version `GLIBC_ABI_DT_RELR' not found (required by /cvmfs/software.eessi.io/versions/2023.06/compat/linux/x86_64/lib/../lib64/libm.so.6)
Now is the moment of truth, if everything looks right when checking with ldd, you should be fine to run the program:
230
195
231
196
```
232
197
{EESSI 2023.06} $ mpirun -n 2 HelloWorld
233
-
Hello World from Node 0
234
-
Hello World from Node 1
198
+
Hello World from rank 0
199
+
Hello World from rank 1
235
200
236
201
```
237
-
Even when closing the shell and restarting the environment, the libraries should point to the directories we setin`LD_RUN_PATH`, still, remember to load the required dependencies before running the binary.
238
202
239
203
!!! warning
240
204
241
205
RPATH should never point to a compatibility layer directory, only to software layer ones, as all resolving is done via the runtime linker (`ld-linux*.so`)
242
206
that is shipped with EESSI, which automatically searches these locations.
243
207
244
-
The biggest downside of this approach is that your executable becomes bound to the architecture you linked your libraries for, i.e., if you add to your executable RPATH a `libhdf5.so`compiled for`intel_avx512`, you will not be able to run that binary on a machine with a different architecture. If this is an issue for you, you should look into how EESSI itself organises the location of binaries and perhaps leverage the relevant environment variables (e.g., `EESSI_SOFTWARE_SUBDIR`).
0 commit comments