Skip to content

Commit

Permalink
v2024.04.26 (#50)
Browse files Browse the repository at this point in the history 
- Add VPKT_WRITE_CONTRIBS option to save all virtual packets
contributions for later time and wavelength binning, with
emission/absorption types by artistools.
- Improve performance by enabling fast-math compiler options, constexpr
radfield bins, improving bound-free estimator accumulation, removing
storage for empty cells, and using node shared memory for
initmassfracstable, elem_meanweight.

- 3D AD2 (Shingles et al. 2023) 1e8pkt 1920 core JUWELS runtime
decreased from 210k core hrs to 134k core hrs (1.56x faster). Memory per
core decreased from 2.8GB to 1.5GB.
- W7 (Shingles et al. 2022) nebular 960 core 2e9pkt 150-410d JUWELS
runtime decreased from 13.6k core hrs (20230811 version) to 10.4k core
hrs (1.31x faster). Compared to the 2022 published version with 18.6k
core hrs, this is 1.79x faster.
- With a Simpson rule integrator in place of GSL's adaptive method
(producing identical observables) runtime decreases to 6.83k core hours,
1.99x faster than 20230811 and 2.72x faster than the 2022 published
version.
- subMch Shen+2018 nebular 960 core 1e9pkt 150-410d runtime decreased
from 13.14k core hrs (20210417 version) to 7.30 kCore hours (1.80x
faster). With Simpson rule integrator, runtime decreased to 6.07 k core
hrs (2.16x faster)
- sim2010 classic mode 2-120 days 2e7 960 core JUWELS runtime decreased
from 3.81k core hrs (20230526 version) to 1.87k core hrs (2.04x faster).
Compared to ARTIS classic with 11.45k core hrs, this version is 6.13x
faster.

- Reduce memory per core to enable future GPU support. All threads now
share a single cell cache and operate on packets within the same cell.
This may later be expanded to multiple cell caches to increase thread
occupancy as memory allows.
- Fix virtual packets for models with internal empty cells. Passing into
an empty cell previously triggered instant escape.
- Fix a 1e5 factor in free-free opacity for opacity case 5 (Tanaka
Ye-dependent grey mode and line-by-line non-grey mode for kilonovae).
This increased the 3D AD2 (Shingles et al. 2023) early luminosity by
about 20%. The spectra are almost unchanged (except for being brighter)
- Reduce noise in particle deposition rate estimators by contributing
partial deposition of packet energy prior to Monte Carlo transition
event (similar to gamma deposition rates from path estimators).
- Add support for calculating and using expansion opacities.
- Add option for #52 by Gerrit Leck
- #47 by Josh Pollin
- Add some fixes to hybrid LTE-NLTE mode by Christine Collins

---------

Co-authored-by: gleck97 <[email protected]>
Co-authored-by: Gerrit Leck <[email protected]>
Co-authored-by: Josh Pollin <[email protected]>
Co-authored-by: Christine Collins <[email protected]>
  • Loading branch information
@lukeshingles @gleck97
@jpollin98 @ccollins22
5 people authored Apr 26, 2024
1 parent d1eec12 commit d9e7d90
Show file tree
Hide file tree
Showing 108 changed files with 13,899 additions and 8,184 deletions.
11 changes: 7 additions & 4 deletions .clang-tidy
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ Checks: >
-cert-err34-c,
-cert-err58-cpp,
-clang-analyzer-deadcode.DeadStores,
-clang-diagnostic-unused-macros,
-cppcoreguidelines-avoid-magic-numbers,
-cppcoreguidelines-avoid-non-const-global-variables,
-cppcoreguidelines-macro-usage,
Expand All @@ -27,7 +26,6 @@ Checks: >
-cppcoreguidelines-pro-bounds-constant-array-index,
-cppcoreguidelines-pro-bounds-pointer-arithmetic,
-cppcoreguidelines-pro-type-cstyle-cast,
-cppcoreguidelines-pro-type-reinterpret-cast,
-cppcoreguidelines-pro-type-vararg,
-fuchsia-*,
-google-objc*,
Expand All @@ -37,7 +35,6 @@ Checks: >
-hicpp-no-array-decay,
-hicpp-no-malloc,
-hicpp-signed-bitwise,
-llvm-header-guard,
-misc-use-anonymous-namespace,
-misc-no-recursion,
-misc-non-private-member-variables-in-classes,
Expand All @@ -54,11 +51,17 @@ CheckOptions:
value: google
- key: cppcoreguidelines-init-variables.MathHeader
value: <cmath>
- key: modernize-use-std-print.PrintfLikeFunctions # replace printout macro with a function when we want to autofix to use std::format gcc >=13
value: printf; absl::PrintF; printout
- key: modernize-use-std-print.FprintfLikeFunctions
value: fprintf; absl::FPrintF
- key: cppcoreguidelines-narrowing-conversions.IgnoreConversionFromTypes
value: size_t;ptrdiff_t;size_t;difference_type;time_t;MPI_Aint
value: size_t;ptrdiff_t;size_type;difference_type;time_t;MPI_Aint
- key: cppcoreguidelines-narrowing-conversions.WarnOnFloatingPointNarrowingConversion
value: 'false'
- key: cppcoreguidelines-narrowing-conversions.WarnOnIntegerToFloatingPointNarrowingConversion
value: 'false'
- key: cppcoreguidelines-pro-type-member-init.IgnoreArrays
value: 'true'
- key: readability-identifier-naming.NamespaceCase
value: lower_case
184 changes: 133 additions & 51 deletions .github/workflows/ci-checks.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ on:

jobs:
cppcheck:
runs-on: macos-13
runs-on: macos-14
steps:
- uses: actions/checkout@v4

Expand All @@ -25,12 +25,12 @@ jobs:
- name: run cppcheck and check for errors
run: |
cppcheck --version
cppcheck --force --error-exitcode=1 --language=c++ --std=c++20 --enable=warning,performance,portability .
cppcheck --force --error-exitcode=1 --language=c++ --std=c++20 --enable=warning,performance,portability --suppress=memleak --check-level=exhaustive .
- name: show cppcheck style suggestions
run: |
cppcheck --version
cppcheck --force --language=c++ --std=c++20 --enable=style --suppress=knownConditionTrueFalse .
cppcheck --force --language=c++ --std=c++20 --enable=style --suppress=knownConditionTrueFalse --check-level=exhaustive .
clang-format:
runs-on: ubuntu-22.04
Expand All @@ -44,7 +44,7 @@ jobs:
check-path: .

clang-tidy:
runs-on: macos-13
runs-on: macos-14
steps:
- uses: actions/checkout@v4

Expand All @@ -61,26 +61,22 @@ jobs:
- name: install llvm
run: |
brew install llvm || true
echo "/usr/local/opt/llvm/bin" >> $GITHUB_PATH
echo "/opt/homebrew/opt/llvm/bin" >> $GITHUB_PATH
echo "CXX=clang++" >> $GITHUB_ENV
echo "OMPI_CXX=clang++" >> $GITHUB_ENV
echo "LDFLAGS=-L/usr/local/opt/llvm/lib" >> $GITHUB_ENV
echo "CPPFLAGS=-I/usr/local/opt/llvm/include" >> $GITHUB_ENV
echo "LDFLAGS=-L/opt/homebrew/opt/llvm/lib" >> $GITHUB_ENV
echo "CPPFLAGS=-I/opt/homebrew/opt/llvm/include" >> $GITHUB_ENV
echo "LDFLAGS=-L/usr/local/opt/libomp/lib" >> $GITHUB_ENV
echo "CXXFLAGS=-I/usr/local/opt/libomp/include" >> $GITHUB_ENV
echo "CXXFLAGS=-I/opt/homebrew/opt/libomp/include" >> $GITHUB_ENV
echo "LIBRARY_PATH=/usr/local/opt/lib" >> $GITHUB_ENV
echo "LD_LIBRARY_PATH=/usr/local/opt/lib" >> $GITHUB_ENV
echo "LIBRARY_PATH=/opt/homebrew/opt/lib" >> $GITHUB_ENV
echo "LD_LIBRARY_PATH=/opt/homebrew/opt/lib" >> $GITHUB_ENV
echo "CPATH=/usr/local/opt/include" >> $GITHUB_ENV
echo "CPATH=/opt/homebrew/opt/include" >> $GITHUB_ENV
echo "compiledb" > requirements.txt
- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
cache: pip
python-version: '3.12'
Expand All @@ -106,51 +102,76 @@ jobs:
CXX: g++
strategy:
matrix:
compiler:
[
compiler: [
{name: gcc, ver: 11},
{name: gcc, ver: 12},
{name: gcc, ver: 13},
{name: clang, ver: 14},
{name: clang, ver: 15},
# {name: gcc, ver: 14},
{name: nvc++, ver: '24.3'},
]
mpi: [ON, OFF]
openmp: [ON, OFF]
stdpar: [ON, OFF]
exclude:
- compiler: {name: clang, ver: 15}
openmp: ON
- openmp: ON
stdpar: ON
fail-fast: false

name: ${{ matrix.compiler.name }}-${{ matrix.compiler.ver }}${{ matrix.mpi == 'ON' && ' MPI' || ''}}${{ matrix.openmp == 'ON' && ' OpenMP' || ''}}
name: ${{ matrix.compiler.name }} ${{ matrix.compiler.ver }}${{ matrix.mpi == 'ON' && ' MPI' || ''}}${{ matrix.openmp == 'ON' && ' OpenMP' || ''}}${{ matrix.stdpar == 'ON' && ' stdpar' || ''}}
steps:
- uses: actions/checkout@v4

- name: CPU type and core count
id: cpu-count
run: |
g++ -march=native -Q --help=target | grep -- '-march= ' | cut -f3
# echo "count=$(python3 -c 'import psutil; print(int(psutil.cpu_count(logical=False)))')" >> $GITHUB_OUTPUT
python3 -c 'import multiprocessing; print(f"CPU count: {multiprocessing.cpu_count()}")'
echo "count=$(python3 -c 'import multiprocessing; print(multiprocessing.cpu_count())')" >> $GITHUB_OUTPUT
- name: apt update
- name: Install gcc
if: matrix.compiler.name == 'gcc'
uses: fortran-lang/setup-fortran@v1
# continue-on-error: true
with:
compiler: gcc
version: ${{ matrix.compiler.ver }}

- name: Install gcc for nvc++
if: matrix.compiler.name == 'nvc++'
uses: fortran-lang/setup-fortran@v1
continue-on-error: true
with:
compiler: gcc
version: 13

# - name: Install cuda-toolkit
# if: matrix.compiler.name == 'nvc++'
# run: |
# sudo apt-get -y install nvidia-cuda-toolkit
# echo "/usr/local/cuda-12.4/bin" >> $GITHUB_PATH

- name: Install nvc++
if: matrix.compiler.name == 'nvc++'
uses: fortran-lang/setup-fortran@v1
# continue-on-error: true
with:
compiler: nvidia-hpc
version: ${{ matrix.compiler.ver }}

- name: Set nvc++ as compiler
if: matrix.compiler.name == 'nvc++'
run: |
sudo add-apt-repository main
sudo add-apt-repository universe
sudo add-apt-repository restricted
sudo add-apt-repository multiverse
sudo apt-get update
echo "CXX=nvc++" >> $GITHUB_ENV
echo "OMPI_CXX=nvc++" >> $GITHUB_ENV
- name: Install gcc-${{ matrix.compiler.ver }}
if: matrix.compiler.name == 'gcc'
- name: install gsl
run: |
sudo apt install -y gcc-${{ matrix.compiler.ver }} g++-${{ matrix.compiler.ver }}
echo "CXX=g++-${{ matrix.compiler.ver }}" >> $GITHUB_ENV
sudo apt install libgsl-dev
- name: Install clang-${{ matrix.compiler.ver }}
if: matrix.compiler.name == 'clang'
- name: install tbb
if: matrix.compiler.name == 'gcc' && matrix.stdpar == 'ON'
run: |
sudo apt install -y clang-${{ matrix.compiler.ver }} --install-suggests
echo "CXX=clang++-${{ matrix.compiler.ver }}" >> $GITHUB_ENV
sudo apt install libtbb-dev
- name: install openmpi
if: matrix.mpi == 'ON'
Expand All @@ -162,30 +183,91 @@ jobs:
run: |
sudo apt-get install -y libomp5-14 libomp-dev
- name: install gsl
run: sudo apt install libgsl-dev
- name: Compile classic mode
run: |
$CXX -v
cp -v -p artisoptions_classic.h artisoptions.h
make clean
make MPI=${{matrix.mpi}} OPENMP=${{matrix.openmp}} ${{ matrix.stdpar == 'ON' && ' STDPAR=ON' || ''}} -j${{ steps.cpu-count.outputs.count}} sn3d exspec
- name: Compile nebular mode
run: |
cp -v -p artisoptions_classic.h artisoptions.h
make clean
make MPI=${{matrix.mpi}} OPENMP=${{matrix.openmp}} ${{ matrix.stdpar == 'ON' && ' STDPAR=ON' || ''}} -j${{ steps.cpu-count.outputs.count}} sn3d exspec
mac-llvm:
runs-on: macos-14
env:
CXX: clang++
strategy:
matrix:
mpi: [ON, OFF]
openmp: [ON, OFF]
stdpar: [ON, OFF]
exclude:
- openmp: ON
stdpar: ON
fail-fast: false

# - name: Set compiler environment variables (MPI off)
# if: matrix.mpi == 'OFF'
# run: echo "CXX=${{ matrix.compiler.cmd }}" >> $GITHUB_ENV
name: macOS brew clang ${{ matrix.mpi == 'ON' && ' MPI' || ''}}${{ matrix.openmp == 'ON' && ' OpenMP' || ''}}${{ matrix.stdpar == 'ON' && ' stdpar' || ''}}

- name: Set compiler environment variables (MPI on)
steps:
- uses: actions/checkout@v4

- name: CPU core count
id: cpu-count
run: |
# echo "count=$(python3 -c 'import psutil; print(int(psutil.cpu_count(logical=False)))')" >> $GITHUB_OUTPUT
echo "count=$(python3 -c 'import multiprocessing; print(multiprocessing.cpu_count())')" >> $GITHUB_OUTPUT
- uses: maxim-lobanov/setup-xcode@v1
with:
xcode-version: 'latest-stable'

- name: install dependencies
run: |
brew update
brew install gsl || true
- name: install llvm
run: |
brew install llvm || true
echo "/opt/homebrew/opt/llvm/bin" >> $GITHUB_PATH
echo "LDFLAGS=-L/opt/homebrew/opt/llvm/lib/c++ -Wl,-rpath,/opt/homebrew/opt/llvm/lib/c++ $LDFLAGS" >> $GITHUB_ENV
echo "CXX=clang++" >> $GITHUB_ENV
- name: install OpenMP
if: matrix.openmp == 'ON'
run: |
brew install libomp || true
brew link --force libomp
echo "LDFLAGS=-L/opt/homebrew/opt/libomp/lib $LDFLAGS" >> $GITHUB_ENV
echo "CXXFLAGS=-I/opt/homebrew/opt/libomp/include $CXXFLAGS" >> $GITHUB_ENV
- name: install OpenMPI
if: matrix.mpi == 'ON'
run: |
echo "OMPI_CXX=$CXX" >> $GITHUB_ENV
echo "CXX=mpicxx" >> $GITHUB_ENV
mpicxx --showme:version
brew install openmpi || true
- name: Show compiler version and CPU type
- name: install onedpl and tbb
if: matrix.stdpar == 'ON'
run: |
${{ env.CXX }} --version
brew install onedpl tbb
echo "LIBRARY_PATH=$HOMEBREW_PREFIX/lib:$LIBRARY_PATH" >> $GITHUB_ENV
echo "LD_LIBRARY_PATH=$HOMEBREW_PREFIX/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
echo "CPATH=$HOMEBREW_PREFIX/include:$CPATH" >> $GITHUB_ENV
- name: Compile classic mode
run: |
cp -v -p artisoptions_classic.h artisoptions.h
make MPI=${{matrix.mpi}} OPENMP=${{matrix.openmp}} -j${{ steps.cpu-count.outputs.count}} sn3d exspec
make clean
make MPI=${{matrix.mpi}} OPENMP=${{matrix.openmp}} ${{ matrix.stdpar == 'ON' && ' STDPAR=ON' || ''}} -j${{ steps.cpu-count.outputs.count}} sn3d exspec
- name: Compile nebular mode
run: |
cp -v -p artisoptions_classic.h artisoptions.h
make MPI=${{matrix.mpi}} OPENMP=${{matrix.openmp}} -j${{ steps.cpu-count.outputs.count}} sn3d exspec
make clean
make MPI=${{matrix.mpi}} OPENMP=${{matrix.openmp}} ${{ matrix.stdpar == 'ON' && ' STDPAR=ON' || ''}} -j${{ steps.cpu-count.outputs.count}} sn3d exspec
Loading

0 comments on commit d9e7d90

Please sign in to comment.