Subapp (when executed on MULTIAPP_FIXED_POINT_BEGIN) solution update wrongly if fixed point iterates more than 1

[BoZeng1997]

Check these boxes if you have followed the posting rules.

• Q&A General is the most appropriate section for my question
• I have consulted the posting Guidelines on the Discussions front page
• I have searched the Discussions forum and my question has not been asked before
• I have searched the MOOSE website and the documentation does not answer my question
• I have formatted my post following the posting guidelines (screenshots as a last resort, triple back quotes around pasted text)

Question

Issue description

In a multiapp system, when the subapp's execute_on=MULTIAPP_FIXED_POINT_BEGIN and the number of fixed point iteration exceeds 1, the subapp solution will not be updated. From the console output information we can see that the subapp executes all steps as usual, but the solution is not updated. I have reproduced this issue on both conda environment and HPC installation.

To reproduce the issue

Moose's version that is giving this issue to me

MOOSE Version:           git commit 33ae80ec85 on 2025-02-10
LibMesh Version:         
PETSc Version:           3.22.1
SLEPc Version:           3.22.1

Input deck

A simple problem that reproduce the issue will be posted in the following comment section

Expected behaviors

Normal result

This will give you a normal results where the temperature field solved in the subapp is successfully updated and transferred to the main app

itr=2 # this can be any number
 _execute_on=TIMESTEP_BEGIN

with example csv results

time,T10,Tmax,Tmed,Tmin
0,0,0,0,0
0.1,300.23059129783,322.8302654104,311.41513330067,300.00000119095
0.2,300.95738602135,345.60741454915,322.80371374336,300.00001293757
0.3,302.26890341423,368.25099586961,334.1255347396,300.00007360958
0.4,304.12307975971,390.65392143627,345.32710706579,300.00029269532
0.5,306.43815208062,412.70393011112,356.35242279925,300.00091548738
0.6,309.13393345952,434.30175925264,367.15208112657,300.00240300051
0.7,312.14264025838,455.37068834537,377.68810087517,300.00551340497
0.8,315.40946226991,475.8584980048,387.93493398591,300.01136996701
0.9,318.89055377774,495.73498299563,397.87824348904,300.02150398245
1,322.55078324441,514.98769972472,407.51278345666,300.03786718861

Abnormal result

With this

 itr=2
_execute_on=MULTIAPP_FIXED_POINT_BEGIN

in subapp the temperature field is solved as the console output reports normal nonlinear residual behavior

temp0: Time Step 9, time = 0.9, dt = 0.1
temp0:  0 Nonlinear |R| = 4.203058e+01
temp0:  1 Nonlinear |R| = 3.352155e-13
temp0:  Solve Converged!

but the results is not updated. An example temperature reported by the subapp csv is

time,T10,Tmax,Tmed,Tmin
0,0,0,0,0
0.1,0,0,0,0
0.2,0,0,0,0
0.3,0,0,0,0
0.4,0,0,0,0
0.5,0,0,0,0
0.6,0,0,0,0
0.7,0,0,0,0
0.8,0,0,0,0
0.9,0,0,0,0
1,0,0,0,0

However, the main app result move forward by 1 step (only)

time,T10,Tmax,Tmed,Tmin
0,0,0,0,0
0.1,300.23059129783,322.8302654104,311.41513330067,300.00000119095
0.2,300.23059129783,322.8302654104,311.41513330067,300.00000119095
0.3,300.23059129783,322.8302654104,311.41513330067,300.00000119095
0.4,300.23059129783,322.8302654104,311.41513330067,300.00000119095
0.5,300.23059129783,322.8302654104,311.41513330067,300.00000119095
0.6,300.23059129783,322.8302654104,311.41513330067,300.00000119095
0.7,300.23059129783,322.8302654104,311.41513330067,300.00000119095
0.8,300.23059129783,322.8302654104,311.41513330067,300.00000119095
0.9,300.23059129783,322.8302654104,311.41513330067,300.00000119095
1,300.23059129783,322.8302654104,311.41513330067,300.00000119095

When the fixed point scheme iterates more than 1, somehow the subapp loses its step 1 solution and go back to step0 but mainapp still receive the step 1 solution. Then the next step when subapp is called, it is still computing step 0+1.

Supplementary result: restrict the max_itr=1

With

 itr=1
_execute_on=MULTIAPP_FIXED_POINT_BEGIN

the result will go back to normal behavior.

More info

A moose version that is not giving this issue

MOOSE Version:           git commit af2039e2a4 on 2024-01-28
LibMesh Version:         
PETSc Version:           3.20.3
SLEPc Version:           3.20.1

[BoZeng1997]

bug_main.i

# unit m, kg, s

A=8e8
T_h=300
dt=0.1

##########
# Please uncomment one section to see different behaviors
############

# This set of para will give normal behavior
# itr=2
# _execute_on=TIMESTEP_BEGIN

# This set of para will give normal behavior
# itr=1
# _execute_on=MULTIAPP_FIXED_POINT_BEGIN

# subapp solution is not updated
itr=2
_execute_on=MULTIAPP_FIXED_POINT_BEGIN

[Problem]
  type = FEProblem
  verbose_multiapps = true
  solve = False
  kernel_coverage_check = false
  material_coverage_check = false
[]

[MultiApps]
  [temp]
    type = TransientMultiApp
    input_files = bug_sub.i
    cli_args = 'T_h=${T_h};A=${A};dt=${dt};_execute_on=${_execute_on};itr=${itr}'
    execute_on = ${_execute_on}
  []
[]

[Transfers]
  [from_temp]
    type = MultiAppCopyTransfer
    from_multi_app = temp
    variable = 'T_K '
    source_variable = 'T_K '
    execute_on = SAME_AS_MULTIAPP
  []
[]

[Mesh]
  [ori]
    type = GeneratedMeshGenerator
    dim = 2
    nx = 15
    xmax = 15e-3
    ny = 15
    ymax = 15e-3
  []
[]

[AuxVariables]
  [T_K]
    [InitialCondition]
      type = ConstantIC
      value = ${T_h}
    []
  []
[]


[Debug]
  show_var_residual_norms = TRUE
  # show_execution_order = MULTIAPP_FIXED_POINT_BEGIN
[]


[Postprocessors]
  [T10]
    type = PointValue
    point = '10e-3 0 0'
    variable = T_K
  []
  [Tmin]
    type = NodalExtremeValue
    variable = T_K
    value_type = min
  []
  [Tmax]
    type = NodalExtremeValue
    variable = T_K
    value_type = max
  []
  [Tmed]
    type = ParsedPostprocessor
    expression = '0.5*(Tmax+Tmin)'
    pp_names = 'Tmax Tmin'
  []
[]

[Executioner]
  type = Transient
  solve_type = NEWTON
  petsc_options_iname = '-pc_type -pc_factor_mat_solver_package'
  petsc_options_value = 'lu      superlu_dist    '
  automatic_scaling = true
  # auto_advance=true
  line_search = bt
  compute_scaling_once = true # true #is recommended

  nl_rel_tol = 1e-10
  nl_abs_tol = 1e-8
  nl_max_its = 200

  l_max_its = 10000
  dtmin = 1e-9
  start_time = 0
  end_time = 1
  dt = ${dt}

  fixed_point_max_its = ${itr}
  fixed_point_min_its = ${itr}
  accept_on_max_fixed_point_iteration = true
  verbose=true
[]

[Outputs]
  [nemesis]
    type = Exodus
  []
  [csv]
    type = CSV
  []
  file_base = '${_execute_on}_itr${itr}/${_execute_on}_itr${itr}'
  print_linear_residuals = false
[]

[BoZeng1997]

bug_sub.i

# unit m, kg, s


specific_heat = 880 # J/(kg K)
thermal_conductivity = 25.0 # W/(mK)
density = 3980 #kg/m^3


[Problem]
  type = FEProblem
  # solve = False
[]

[Mesh]
  [ori]
    type = GeneratedMeshGenerator
    dim = 2
    nx = 15
    xmax = 15e-3
    ny = 15
    ymax = 15e-3
  []
  [inner]
    type = ParsedSubdomainMeshGenerator
    input = ori
    block_id=1
    block_name = 'inner'
    combinatorial_geometry = 'sqrt((x)^2+y^2)<=7.25e-3'
  []
[]

[Variables]
  [T_K]
    [InitialCondition]
      type = ConstantIC
      value = ${T_h}
    []
  []
[]


[Kernels]
  [timeder]
    type = ADHeatConductionTimeDerivative
    variable = 'T_K'
    density_name = density
    specific_heat = specific_heat
    use_displaced_mesh = false
  []
  [diff]
    type = ADHeatConduction
    variable = 'T_K'
    thermal_conductivity = thermal_conductivity
    use_displaced_mesh = false
  []
  [heatsource]
    type = ADMatHeatSource
    variable = 'T_K'
    material_property = heating
    block = inner
    use_displaced_mesh = false
  []
[]


[Materials]
  [heating]
    type = ADParsedMaterial
    property_name = heating
    constant_names = 'A'
    constant_expressions = '${A}'
    expression = 'A'
    block = 'inner'
    use_displaced_mesh = false
  []
  [bulk]
    type = ADGenericConstantMaterial
    prop_names = 'density thermal_conductivity specific_heat'
    prop_values = '${density} ${thermal_conductivity} ${specific_heat}'
  []
[]

[Postprocessors]
  [T10]
    type = PointValue
    point = '10e-3 0 0'
    variable = T_K
  []
  [Tmin]
    type = NodalExtremeValue
    variable = T_K
    value_type = min
  []
  [Tmax]
    type = NodalExtremeValue
    variable = T_K
    value_type = max
  []
  [Tmed]
    type = ParsedPostprocessor
    expression = '0.5*(Tmax+Tmin)'
    pp_names = 'Tmax Tmin'
  []
[]


[Executioner]
  type = Transient
  solve_type = NEWTON
    # petsc_options_iname = '-pc_type -pc_hypre_type '
    # petsc_options_value = 'hypre boomeramg '
  petsc_options_iname = '-pc_type -pc_factor_mat_solver_package'
  petsc_options_value = 'lu      superlu_dist    '
  automatic_scaling = true

  line_search = bt
  compute_scaling_once = true # true #is recommended

  nl_rel_tol = 1e-10
  nl_abs_tol = 1e-8
  nl_max_its = 20
  # l_abs_tol = 1e-9
  l_tol = 1e-5
  l_max_its = 10000
  dtmin = 1e-9
  start_time = 0
  dt = '${dt}'
  # auto_advance=true
[]


[Outputs]
  [nemesis]
    type = Exodus
  []
  [csv]
    type = CSV
  []
  file_base = '${_execute_on}_itr${itr}/${_execute_on}_itr${itr}_temp'
  print_linear_residuals = false
[]

[GiudGiud]

Thanks for reporting. I ll confirm the issue then we'll open an issue