Issue with Configuration Basis Output Using csf_solver

[NastaMauger]

Hello,

I would like to perform a configuration basis CASCI calculation with PySCF on methane's ground state and hence use your csf_solver. Below is the relevant part of my script:

from mrh.my_pyscf.fci import csf_solver
norb = 9 
nelec = 10
casci = mcscf.CASCI(rhf, norb, nelec)                                                                                                         
casci.verbose = 9 
casci.fcisolver = csf_solver(mol, smult=1)
casci.run()

However, I am not seeing any differences in the output compared to the native PySCF output, even though I set the verbosity level quite high. and my output shows:

******** <class 'mrh.my_pyscf.fci.csf_symm.FCISolver'> ********
max. cycles = 100
conv_tol = 1e-10
davidson only = False
linear dependence = 1e-14
level shift = 0.001
max iter space = 12

It seems like I might be missing something. Specifically, I would like to print the number of configurations before the CI-matrix diagonalization, similar to how GMESS does it, e.g.:
THE WAVEFUNCTION CONTAINS 2210880 WALKS (CSF-S).

This would help me compare and ensure that my PySCF+MRH input (and hence output) is correct.

Thank you!

[NastaMauger]

Also, when I increased the basis set (going from sto-3g to 6-31g) I got this

.local/lib/python3.10/site-packages/mrh/my_pyscf/fci/csf.py:102: RuntimeWarning: overflow encountered in scalar multiply
  mem_floats = nfloats * np.dtype (float).itemsize / 1e6

[JangidBhavnesh]

Hello,

I would like to perform a configuration basis CASCI calculation with PySCF on methane's ground state and hence use your csf_solver. Below is the relevant part of my script:

from mrh.my_pyscf.fci import csf_solver
norb = 9 
nelec = 10
casci = mcscf.CASCI(rhf, norb, nelec)                                                                                                         
casci.verbose = 9 
casci.fcisolver = csf_solver(mol, smult=1)
casci.run()

However, I am not seeing any differences in the output compared to the native PySCF output, even though I set the verbosity level quite high. and my output shows:

******** <class 'mrh.my_pyscf.fci.csf_symm.FCISolver'> ********
max. cycles = 100
conv_tol = 1e-10
davidson only = False
linear dependence = 1e-14
level shift = 0.001
max iter space = 12

It seems like I might be missing something. Specifically, I would like to print the number of configurations before the CI-matrix diagonalization, similar to how GMESS does it, e.g.: THE WAVEFUNCTION CONTAINS 2210880 WALKS (CSF-S).

This would help me compare and ensure that my PySCF+MRH input (and hence output) is correct.

Thank you!

You can get the no of csf, and no of determinant like this
Also, I don't think so, we can print this information just by increase verbosity.

mc= mcscf.CASCI(rhf, norb, nelec)
mc.fcisolver = csf_solver(mol, smult=1)
mc.kernel()

#CSF = mc.fcisolver.transformer.ncsf
#Det = mc.fcisolver.transformer.ndet

[JangidBhavnesh]

Also, when I increased the basis set (going from sto-3g to 6-31g) I got this

.local/lib/python3.10/site-packages/mrh/my_pyscf/fci/csf.py:102: RuntimeWarning: overflow encountered in scalar multiply
  mem_floats = nfloats * np.dtype (float).itemsize / 1e6

Can you set the mol.max_memory ?
Like this
mol.max_memory = 4000 # In MB, or depends on your memory availability.

[MatthewRHermes]

You can't really check the number of CSF's "before diagonalization" (i.e., before the kernel call) because PySCF's FCI solvers are designed to take the number of orbitals and electrons as kernel arguments, not as object attributes, so the necessary information isn't available before running the kernel. However I just did some stuff in the dev branch that might help you out (see https://github.com/MatthewRHermes/mrh/blob/dev/examples/csf/csf_fci.py).

As for the integer overflow, this might be related to some numpy 2.0 promotion changes in the pipeline that I haven't seen yet. I modified the relevant code to hopefully make it more robust to this. Hopefully you were not overflowing 64-bit integers because that would imply at least a 74 million terabyte array, although I imagine that if that happened then numpy would complain. As you might have gathered, the CSF solver is memory-inefficient (#48) and for low-spin wave functions it becomes unusable around (16e,16o).

[NastaMauger]

Hello,

Thank you a lot for your replies.

I have forked your master and dev branches, and here are my findings:

• Increasing the memory through mol.max_memory to 25 GB does not solve the issue I have, but this makes sense since the same system, basis set, and type of calculation led to 262 GB of memory usage with GAMESS. Fortunately, it does not cause PySCF to crash.

• Using your master branch and these inputs with my system (case 1):

from mrh.my_pyscf.fci import csf_solver

norb = 17 
nelec = 10
casci = mcscf.CASCI(rhf, norb, nelec)                                                                                                         
casci.verbose = 9 
casci.fcisolver = csf_solver(mol, smult=1)
casci.run()
ncsf = casci.fcisolver.transformer.ncsf
print(f'Number of CSF is: {ncsf}')

gives the same number of CSF compared to GAMESS using GUGA and symmetry features (NCSF = 2210880) and also energy.

• Now, using your dev branch and adapting your example (https://github.com/MatthewRHermes/mrh/blob/dev/examples/csf/csf_fci.py) with my system/basis set, the output is (case 2):

Singlet configuration:
***** CSFTransformer configuration *****
norb = 17
neleca, nelecb = 5, 5
smult = 1
orbsym = None
wfnsym = None
ndeta, ndetb = 6188, 6188
ncsf = 8836464

Notice the difference in ncsf. My understanding is that the counting of configurations is not done in the same space, correct? One seems to be in the RI space, and the other in the configuration space. Can you please confirm if this is the case?

• It might be due to the fact that I am using the dev branch, but this version is extremely slow compared to the master branch. While it takes ~15 minutes for the kernel execution with case 1, the dev branch takes more than 40 minutes (and still hasn't finished) (case 2).

• The energy obtained from Case 2 is incorrect, while Case 1 is accurate.

• I was also wondering if this is compatible with the sCI of PySCF. I tried to adapt my script to call selected_ci_spin0_symm.SelectedCI(rhf) with csf_solver but have not succeeded. Looking at the code, it seems this is not implemented (neither in the dev nor the master branch), is that correct? Is there a branch where it might be available ?

Could you please help clarify these points or suggest any solutions to improve performance and compatibility? I really appreciate your time and assistance!

Thank you once again for your support.

[MatthewRHermes]

Computing the number of single CSFs by hand gives me 8,836,464, as in your second example. In your first example, were you using point group symmetry? If you used symmetry for the first case but not for the second then it would make sense that the results would differ and that the second case would take much longer. Otherwise I'm not sure where 2,210,880 comes from.

I'm not aware of any interface PySCF's selected CI solver at the moment.

[NastaMauger]

Hello,

You’re absolutely right, I had the symmetry on. Now it makes more sense. I solve my issue
Thank you for your time and answers!

Best