Tests wanted / List of untested functionality

[jtkrogel]

Many tests have been added, but there still remain many features that need to have tests added. The list below is meant to capture all features that do not yet have a test, or known breakages that do not yet have a corresponding test so that fixes can be pursued and validated.

Real space statistical integration tests needed

Real space parallelization tests needed:

• Ensemble runs (minimal tests added in PR: Add minimal ensemble test #4091 , see also Tests needed for ensemble runs #1103 )
• GPU spline data sharing (PR: , see also Example and Test of GPU spline data sharing required #1229 )

Real space estimator tests needed:

• Chiesa kinetic energy correction (PR: , see also Test for Chiesa correction needed #1003 )
• MPC potential energy correction (PR: Add MPC test to diamondC_1x1x1_pp testset. #1470, see also Test for MPC needed #1002 )
• momentum distribution (PR: , see also Test of n(k) needed #627 )
• pair correlation function (PR: , see also Tests of gofr estimator needed #396 )
• structure factor--SkEstimator (PR: Add sofk estimator test and manual entry about how estimator works #764 , see also Test for SkEstimator needed #1000 )
• structure factor--StaticStructureFactor estimator, spin resolved (PR: , see also Test for StaticStructureFactor needed #999 )
• structure factor--SkAllEstimator, finite size corrections (PR: , see also Test for SkAllEstimator needed #1001 )
• energy density matrix (PR: , see also Test for energy density matrix estimator needed #981, Energy density matrix is broken #983 )
• traces (PR: , see also Test for traces needed #1063 )
• forward walking (PR: , see also Test for ForwardWalking needed #1004 )
• core polarization potential (PR: , see also Test for core polarization potential needed #1005 )
• force estimators in open BC's (PR: , see also Add Force Estimator Tests for Open and Periodic Boundary Conditions #332 )
• force estimators in periodic BC's (PR: , see also Add Force Estimator Tests for Open and Periodic Boundary Conditions #332 )
• RMC with basically any/all non-energy estimators
• CSVMC with basically any/all non-energy estimators

Real space wavefunction tests needed:

• excited states in Slater determinant (PR: Add excited state statistical integration tests #1702, up/down dets still needed, see also Tests for excitations and ground state modifications needed #740 )
• matrix tiling of B-spline orbitals (PR: , see also Integration test: matrix tiling of B-spline orbitals #575 )
• quasi-2D boundary conditions (?) (PR: , see also Integration test: quasi-2D #578 )
• electron-ion backflow (PR: , see also Test for electron-Ion backflow transformation needed #1226 )

Real space method tests needed:

• pade2 Jastrow optimization (PR: , see also Jastrow pade2 optimization is misbehaving #412)
• orbital optimization with Bspline orbitals (PR: , see also Orbital optimization wavefunction with einspline is missing kinetic energy  #637, Einspline orbital optimization not friendly with following runtime parameters #638)
• CSVMC with openmp enabled (PR: , see also CSVMC doesn't work with OpenMP #362)
• DMC restart test (PR: , see also Restart tests needed on DMC #798 )
• DMC size consistent T-moves with splines (PR: Refine Tmove v3 option algorithm #1288, see also Test of size consistent Tmoves with splines #630 )

Tests that might be useful for bugfixes:

• DMC test demonstrating stochastic reconfiguration problem (PR: , see also DMC stochastic reconfiguration is incorrect #18 )
• RMC test demonstrating BlockWeight problem (PR: , see also Fix BlockWeight in RMC #180)
• orbital optimization demonstrating incorrect energy with LCAO (PR: , see also Einspline orbital optimization not friendly with following runtime parameters #638)

Real space deterministic integration tests needed

(plan out first set of tests to add here, see also #704 )

• Chiesa kinetic energy correction
• exited states for periodic system
• 2D boundary condition

Real space unit tests needed

(need to list)

Auxiliary field integration tests needed

Auxiliary field estimator tests needed:

• Back propagated options (path restoration on/off, internal averaging...)

Auxiliary field unit tests needed:

• afqmctools python unit tests need to be turned on.
• Dense Hamiltonian unit tests

Auxiliary field method tests needed:

• Workflow tests need to be turned on.
• Multi-node tests needed for distributed propagation / memory.

See #1062 for an overarching view of the testing problem and links to its other aspects.

[jtkrogel]

@rcclay are there other RMC features that aren't covered yet?

@mmorale3 should this be filled out for AFQMC now, or following the mainline merge of your updated AFQMC?

[rcclay]

@jtkrogel Not that I can think of off the top of my head.

[Hyeondeok-Shin]

I'm now working with the real space wavefunction tests (including deterministic test), but please let me know if someone is already working with a certain test in order to avoid conflict.

[jtkrogel]

@Hyeondeok-Shin You mean via the wavefunction tester? I expect pass/fail criteria for the finite differences will require some tinkering. If this is the case, perhaps make a new issue for further discussion on that topic.

Please can you add the deterministic integration tests you will be working on next to the header comment under the appropriate section?

[Hyeondeok-Shin]

@jtkrogel I see. I'll be adding the deterministic integration tests list I'm (will be) working on.

[prckent]

Please steer clear of the "wavefunction tester" for new tests. It is another layer of indirection and uncertainty to investigate when something goes wrong.

A test of traces would be preferred by far, be simpler, and achieve the same coverage.

[prckent]

Renamed for clarity.

[walshmm]

Was wondering how up to date this issue is. Wanted to look at where I could start writing tests, though I dont want to step on anyone's toes as I understand some parts of the code are being refactored.

[prckent]

It is fairly up to date. Since this indexing issue was created we have added use of codecov, which also is a good guide to where tests might be missing. Of course the reported codecov coverage https://app.codecov.io/gh/QMCPACK/qmcpack only sees the tests run in CI, not our complete set of tests, so the coverage is somewhat better.

Many of these require either wholly new input files to use the functionality or they require a bit of infrastructure. e.g. #1103 is important to test since it is a production feature, but new cmake macros would be needed to copy multiple input files, run "qmcpack listofinputs.txt", and then analyze the results. The individual runs can be repeats of runs where we have existing tests.