EB-C10 tutorial, problem with tagging

[jagmohas]

Hi I am trying the laminar pipe flow tutorial
https://amrex-combustion.github.io/PeleC/ebverification/C10/README.html

with the example.inp provided. I would like to refine the region near the wall. based on velocity gradient. To test the tagging, I increased the max_level to 2 but did not add any tagging criterion. However, I noticed that PeleC refined the mesh everywhere (added one level). Is this expected?

Second, when I add the following tagging criterion
tagging.refinement_indicators = magvel
tagging.magvel.max_level = 4
tagging.magvel.value_greater = 1.2e3
tagging.magvel.field_name = magvel
amr.max_level = 4 # maximum level number allowed

It does not add any amr level? I am not sure what I am doing wrong. Could you please advise? Here is the screenshot of the velocity field, magvel is definitely is larger than the value I am specifying in the tagging, but still it does not work.

[baperry2]

By default, PeleC refines the entire embedded boundary surface up to amr.max_level. In this case, if you set amr.max_level = 2, that means essentially the whole grid is refined to the second level as a result of the block-structured nature of how refinement is done with AMReX. You should see something like this in the simulation output at the end of initialization just before the plot file gets dumped:

INITIAL GRIDS
Level 0 24 grids 12288 cells 100 % of domain
smallest grid: 8 x 8 x 8 biggest grid: 8 x 8 x 8
Level 1 48 grids 98304 cells 100 % of domain
smallest grid: 16 x 16 x 8 biggest grid: 16 x 16 x 8
Level 2 36 grids 442368 cells 56.25 % of domain
smallest grid: 32 x 16 x 16 biggest grid: 32 x 32 x 16

If you do not wish to refine the embedded boundary set tagging.max_eb_refine_lev = 0 in your input file (or set this flag to the maximum desired level for the eb if you want some refinement at the boundary but less than amr.max_level). Note however that a limitation in PeleC is that the entire embedded boundary surface must be at the same level of refinement. If you set tagging.max_eb_refine_lev no cells adjacent to the eb surface will be refined above this level, even if they should be based on other tagging criteria. In this case, setting tagging.max_eb_refine_lev = 0 prevents refinement anywhere in the domain, because the resolution on the base grid is insufficient to allow for refinement without refining the eb surface.

I'm not sure why the tagging criterion based on magvel that you're using isn't working. I just tested it and it worked for me. If you still have issues, can you post the full input file you are using?

[jagmohas]

Thank you @baperry2 for explaining this. I am not sure why I am getting a different behavior. By default (tagging.max_eb_refine_lev = 0) I get only two levels even though I set max_level to 4. However, as you explained I should be getting 4 levels. Second, the tagging does not work for me. Please see my input file below. Many thanks for your help.

------------------ INPUTS TO MAIN PROGRAM -------------------

max_step = 10

stop_time = 0.01728159250000112 # 5 flowthroughs (1 flowthrough = L / umax)

PROBLEM SIZE & GEOMETRY

geometry.is_periodic = 0 0 0
geometry.coord_sys = 0 # 0 => cart
geometry.prob_lo = 0 -2.0 -2.0
geometry.prob_hi = 12.0 2.0 2.0
#amr.n_cell = 48 16 16
amr.n_cell = 24 8 8

>>>>>>>>>>>>> BC KEYWORDS <<<<<<<<<<<<<<<<<<<<<<

Interior, UserBC, Symmetry, SlipWall, NoSlipWall

>>>>>>>>>>>>> BC KEYWORDS <<<<<<<<<<<<<<<<<<<<<<

pelec.lo_bc = "Hard" "FOExtrap" "FOExtrap"
pelec.hi_bc = "Hard" "FOExtrap" "FOExtrap"

Problem setup

pelec.eb_boundary_T = 300.
pelec.eb_isothermal = 1

WHICH PHYSICS

pelec.do_hydro = 1
pelec.do_mol = 1
pelec.do_react = 0
pelec.allow_negative_energy = 0
pelec.diffuse_temp = 1
pelec.diffuse_vel = 1
pelec.diffuse_spec = 0
pelec.diffuse_enth = 0

TIME STEP CONTROL

pelec.dt_cutoff = 5.e-20 # level 0 timestep below which we halt
pelec.cfl = 0.3 # cfl number for hyperbolic system
pelec.init_shrink = 0.8 # scale back initial timestep
pelec.change_max = 1.05 # maximum increase in dt over successive steps

DIAGNOSTICS & VERBOSITY

pelec.sum_interval = 1 # timesteps between computing mass
pelec.v = 1 # verbosity in PeleC cpp files
amr.v = 1 # verbosity in Amr.cpp
#amr.grid_log = grdlog # name of grid logging file

REFINEMENT / REGRIDDING

amr.max_level = 4 # maximum level number allowed
amr.ref_ratio = 2 2 2 2 # refinement ratio
amr.regrid_int = 2 # how often to regrid
amr.blocking_factor = 8 # block factor in grid generation
amr.max_grid_size = 16

CHECKPOINT FILES

amr.checkpoint_files_output = 0
amr.check_file = chk # root name of checkpoint file
amr.check_int = -1 # number of timesteps between checkpoints

PLOTFILES

amr.plot_files_output = 1
amr.plot_file = plt
amr.plot_int = 100
amr.derive_plot_vars=ALL

tagging.refinement_indicators = magvel
tagging.magvel.max_level = 4
tagging.magvel.value_greater = 1.2e3
tagging.magvel.field_name = magvel
tagging.max_eb_refine_lev = 0

eb2.geom_type = "cylinder"
eb2.cylinder_direction = 0
eb2.cylinder_center = 0.0 0.0 0.0
eb2.cylinder_radius = 1.0
eb2.cylinder_height = 1000.0
eb2.cylinder_has_fluid_inside = 1
ebd.boundary_grad_stencil_type = 0

[baperry2]

To clarify something, the default value for tagging.max_eb_refine_lev is 10, which effectively means the EB is refined to amr.max_level setting.

When you set tagging.max_eb_refine_lev = 0, the refinement of cells adjacent to the EB is turned off. In the case you are running, you should observe that setting this flag to 0 prevents all refinement (because all cells on the base level are close to the EB), and the simulation runs on the base grid only.

If you do not set this flag, you should see refinement up to amr.max_level. When I run with the input file you posted, but setting tagging.max_eb_refine_lev = 4, I see four levels of refinement, but the Levels 1 and 2 cover the full domain:

INITIAL GRIDS
Level 0 3 grids 1536 cells 100 % of domain
smallest grid: 8 x 8 x 8 biggest grid: 8 x 8 x 8
Level 1 12 grids 12288 cells 100 % of domain
smallest grid: 16 x 8 x 8 biggest grid: 16 x 8 x 8
Level 2 24 grids 98304 cells 100 % of domain
smallest grid: 16 x 16 x 16 biggest grid: 16 x 16 x 16
Level 3 108 grids 442368 cells 56.25 % of domain
smallest grid: 16 x 16 x 16 biggest grid: 16 x 16 x 16
Level 4 600 grids 2457600 cells 39.0625 % of domain
smallest grid: 16 x 16 x 16 biggest grid: 16 x 16 x 16

[jagmohas]

Thanks @baperry2 for the explanation. It makes sense. Now, I can get it to work. This is what I had to do.

1. Changed the default value of tagging.max_eb_refine_lev = 1, so that EB refinement works on upto level 1.
2. Added tagging based on magvel to get refinement in the flow core. The figure below shows the contours of vfrac and AMR levels at the center plane.

If I set tagging.max_eb_refine_lev = 0, I expected the magvel based refinement to be still effective, but it does not work. This is what I don't understand. Also, what shall I do to get the refinement only near the wall? I tried the following, but it did not work

tagging.refinement_indicators = magvel
tagging.magvel.max_level = 4
tagging.magvel.field_name = magvel
tagging.magvel.adjacent_difference_greater = 1e2

[baperry2]

As indicated above, setting tagging.max_eb_refine_lev = 0 supersedes other tagging criteria adjacent to the EB. This sets the maximum refinement level for cells adjacent to the EB, and the EB surface will not be refined above this level regardless of other tagging criteria. Specifying it as 0 means no refinement beyond the base level (level 0) can occur at the eb surface.

If you want refinement near the wall, simply comment out the line with tagging.max_eb_refine_lev in your input file, or set it to the same value as amr.max_level. This will result in the default behavior where the wall is automatically refined to the maximum level in the simulation. Here is what that looks like, changing amr.max_level to 4 in example.inp for EB-C10, and leaving everything else untouched:

Note that in addition to the tagging parameters we've been discussing, the way the grids are generated is dependent on several AMReX parameters that can be specified in your input file with the prefix amr. (see descriptions of what these parameters do here: https://amrex-codes.github.io/amrex/docs_html/GridCreation.html#sec-grid-creation). In particular, amr.n_error_buf, amr.blocking_factor, amr.grid_eff will have some effect on how much of the domain gets refined - I'd recommend that you do some experimentation with these as well.