Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial MIR component #1438

Open
wants to merge 356 commits into
base: develop
Choose a base branch
from
Open

Initial MIR component #1438

wants to merge 356 commits into from

Conversation

BradWhitlock
Copy link
Member

@BradWhitlock BradWhitlock commented Oct 3, 2024

This PR is adds an initial MIR component to Axom. EquiZ MIR was implemented as the first algorithm because it was more familiar and my first goal was to shake out problems with infrastructure for writing algorithms against Blueprint data. The MIR algorithm takes Blueprint input and generates Blueprint output.

This PR does the following:

  • Supercedes previous MIR work (previous work was either adapted or moved to "reference" directory)
  • Implements EquiZ (without iteration for now) that runs on CPU/GPU and works on 2D/3D meshes with various topology and shape types.
  • Provides views to simplify writing algorithms that use Blueprint data.
  • Provides several utility classes that perform mesh operations (extract zones, merge meshes, etc.)
  • Adds documentation
  • Adds example programs
  • Adds a little core infrastructure

NOTE:

  • The Elvira algorithm will be added in a future PR.
  • The previous serial MIR implementation was moved to a "reference" directory that we can remove once EquiZAlgorithm has iteration support (future work).
  • The component requires Conduit, RAJA, Umpire - I'll probably have to iterate with the CI some to make that work everywhere.
  • There are some files that changed because of "make style"

@BradWhitlock
Copy link
Member Author

BradWhitlock commented Nov 9, 2024

I added a script that will make BASH scripts to run the mir_concentric_circles example program on various platforms and gather the caliper data results. I plotted the MIR runtime and some things look good, some very bad at the moment. I think SEQ mode could be optimized more to bring it down, likely bringing down time in the other modes too. OMP has inexplicably bad performance. I've seen some hints of this before during development but focused more on GPU so far. I'll need to profile OMP and see what's the matter.

image

rzansel42{whitlocb}144: env OMP_NUM_THREADS=44 ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                 17.601138     17.601138     17.601138 88.542790 
rzansel42{whitlocb}145: env OMP_NUM_THREADS=88 ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                 20.347344     20.347344     20.347344 89.191530 
rzansel42{whitlocb}146: env OMP_NUM_THREADS=32 ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                 16.865811     16.865811     16.865811 88.723097 
rzansel42{whitlocb}147: env OMP_NUM_THREADS=22 ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                 12.256273     12.256273     12.256273 85.147964 
rzansel42{whitlocb}148: env OMP_NUM_THREADS=2 ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                  6.044385      6.044385      6.044385 73.873691 
rzansel42{whitlocb}149: env OMP_NUM_THREADS=1 ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                  3.880441      3.880441      3.880441 64.487702 
rzansel42{whitlocb}150: ./examples/mir_concentric_circles --gridsize 1000 --numcircles 5 --policy omp | grep runMIR
runMIR                                 30.048744     30.048744     30.048744 90.253846 

void conduit_debug_err_handler(const std::string &s1, const std::string &s2, int i1)
{
std::cout << "s1=" << s1 << ", s2=" << s2 << ", i1=" << i1 << std::endl;
// This is on purpose.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you: this is a necessary comment.

inline int getAllocatorIDForAddress(void* ptr)
{
umpire::ResourceManager& rm = umpire::ResourceManager::getInstance();
return rm.getAllocator(ptr).getId();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if ptr was not Umpire-allocated? A check that the allocator could be found would be helpful to avoid crashing. Would it make sense to return INVALID_ALLOCATOR_ID in that case?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion. I added a test that calls getAllocatorIDForAddress() with a pointer not allocated by Umpire, which made Umpire throw an exception. I added exception handling and return INVALID_ALLOCATOR_ID in that case.

*
* \note The coordset view must agree with the coordset in n_input. We pass both
* a view and the coordset node since the view may not be able to contain
* come coordset metadata and remain trivially copyable.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"come coordset metadata" Should this be "some coordset metadata"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed it.

@gunney1
Copy link
Contributor

gunney1 commented Jan 15, 2025

I'm struggling to understand what "blending" is. Can't find it in the code or the docs. (Maybe it's in the referenced publications.) It seems like an important concept. Can you define it, at least in the documentation?

@BradWhitlock
Copy link
Member Author

Blending takes tuples of ids and weights applies them to an input coordset or field to make a new coordset or field. The inputs are blended according to the weights to make the new item. https://github.com/LLNL/axom/pull/1438/files#diff-fb605bba545a37f0257de0b2acdc23ec71f15af3ab1cf907ba5a580e635cb754

Copy link
Member

@kennyweiss kennyweiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks you @BradWhitlock for this comprehensive and well-documented new feature!
Sorry it took so long to review -- there is a lot here!

The new blueprint/view utilities are going to be extremely useful. As we discussed offline, we'll likely want to move them outside the MIR component in the future. Either as a new folder in sidre or as a separate component.


void add_distance(conduit::Node &mesh, float dist = 6.5f)
{
// Make a new distance field.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please enhance the comment -- I think it's the distance to a sphere of radius dist ?

Comment on lines +35 to +41
for(int index = 0; index < nnodes; index++)
{
const auto pt = coordsetView[index];
float norm2 = 0.f;
for(int i = 0; i < pt.DIMENSION; i++) norm2 += pt[i] * pt[i];
valuesPtr[index] = sqrt(norm2) - dist;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: Would it be clearer to use a primal::Sphere and it's computeSignedDistance() member function?

Perhaps not, b/c the latter is templated on the dimension (?)

add_distance(mesh);
}

void make_unibuffer(const std::vector<float> &vfA,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a brief documentation here. Specifically, what is a unibuffer?

}

template <typename Dimensions>
void make_matset(const std::string &type,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a brief description of what this function does. It looks like it makes some (arbitrary) choices about the material distribution

{ }
}

void mixed3d(conduit::Node &mesh)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please document this one too

Comment on lines +15 to +19
// Uncomment to generate baselines
//#define AXOM_TESTING_GENERATE_BASELINES

// Uncomment to save visualization files for debugging (when making baselines)
//#define AXOM_TESTING_SAVE_VISUALIZATION
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: Since this file takes CLI arguments, would it make sense for these to be command line arguments that are off by default rather than compiler guards?

using MaterialID = int;
using MaterialIDArray = axom::Array<MaterialID>;
using MaterialIDView = axom::ArrayView<MaterialID>;
using MaterialVF = float;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be a good idea for a follow-up PR

n_tmpInput[n_matset.path()].set_external(n_matset);
conduit::relay::io::blueprint::save_mesh(n_tmpInput,
"debug_equiz_input",
"hdf5");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: "hdf5" is only available when AXOM_USE_HDF5 is defined

#endif

/*!
* \brief Perform material interface reconstruction on a single domain.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Possible copy/paste] Is this the right doxygen comment for this function?

Comment on lines +6 to +11
/*!
* \file MeshTester.hpp
*
* \brief Contains the specification for the MeshTester class.
*
*/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we create a ticket to reimplement some of these test cases using Klee in a follow-up PR?
The ones we can't reproduce w/ Klee, (e.g. b/c they give explicit values) should probably be moved to axom_data

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants