Post-SCF with custom matrices

[obackhouse]

I would like to use the functionality of the psi4 API, but I would like my wavefunction to be consistent with that of an external quantum chemistry code. I have access to all of the necessary one- and two-electron properties from the external code as numpy arrays, including the orbital coefficients and energies. How can I achieve this using the python API?

I have tried to use data=Wavefunction.to_file() to form a dictionary of the wavefunction, and then replace the necessary quantities, followed by wfn=Wavefunction.from_file(data). This however results in a segfault when I attempt to use psi4.energy('MP2', ref_wfn=wfn). I have also thought about replacing the arrays using Wavefunction.set_array(), but can’t work out how to replace the epsilon_a and epsilon_b. Both of these methods would also overlook the two-electron integrals.

Is this something which I can achieve easily? In general, for post-SCF methods such as MP and CC methods, does psi4 obtain the matrices from the Wavefunction object attributes or is this over-simplifying things?

[hokru]

wfn=Wavefunction.from_file(data) takes an existing wavefunction object. If you want to read a local file that contains a wavefunction object you want .from_file('file_name') (written by .to_file('file_name')

The set_array() [new name set_array_variable()] function is ( as far as I know) not meant to change the C++ matrixes, but to attach additional arbitrary arrays to the wavefunction object for python post-processing (similar to these scalars: http://psicode.org/psi4manual/master/glossary_psivariables.html#apdx-psivariables-alpha)

You want to check out the psi4.core.Matrix and psi4.core.Vector functions. For matrices (like Ca()) you can use copy() and for vectors (like epsilon_a()) you perhaps can use set()
http://psicode.org/psi4manual/master/api/psi4.core.Vector.html?highlight=core%20vector

The answer to your last questions may depend on the actual method (and module) used.
Some recompute the 2-e ints in special format, some need DF-integrals on disk, etc.

[obackhouse]

Thanks for your response - I am using scf_type=pk and have <50 basis functions for these calculations, so I am hoping that these integrals are standard four-index quantities and are in-core? I’m planning to use MP2, CCSD, CC2, CC3 and ADC(2) - do you know if these methods get the ERIs and whether I can overload that with my precomputed ones?

[hokru]

Only conventional, ie. not density fitted, variants, right?

The PK integrals are not used. Instead most post-scf methods use IWL integrals. This basically means another integral generation after the SCF is done. You should see it in the output (“MINTS: Wrapper to libmints.”).

You want to use your own 2e-integrals?

[jmisiewicz]

I found out to my surprise last week that you can use some Matrix methods like add to change the C++ matrices.

I’ll warn you about ADC(2) in particular. The developers know there’s some bug in the code, but none of us know ADC(2) well enough to find it.

[hokru]

ADC(2) will be available through the external adcc module. (or you can use adcc directly as well).

[obackhouse]

Convential yes, no density fitting. I would like to use my own integrals, which I have as a 4d numpy array. Essentially, my method using my own code uses a reference from another quantum chemistry package (namely pyscf), however I would like to use some of the methods in psi4 (which are not in pyscf) to compare to my own method. Obviously to do this I need to make sure the post-SCF methods of psi4 are using the same reference as my own code, and the SCF solvers don’t necessarily converge to the same solution.

In theory I would only need the Fock matrix and ERIs (both in MO basis) to be the same, but I’m not sure how these are obtained in psi4 for these methods so my plan was to just replace all 1e and 2e matrices with my own arrays.

Thanks for the note on ADC(2) - that might not be the main problem as pyscf is starting to support that, but I would certainly like to use the CC2 and CC3 (and EOM thereof) functionality of psi4.

[hokru]

It seems you encountered a bug when doing you tests. (https://github.com/psi4/psi4/issues/1851)
You want to avoid any method in qc_module occ for now. (http://psicode.org/psi4manual/master/notes_c.html#alternate-implementations)

For MP2 you can use:

set scf_type pk
set mp2_type conv
set qc_module fnocc

There should be no segmentation fault then.

edit: Not sure about the integrals. You could turn the post-scf ERI generation off at the python level I think and substitute with your own. That would be simple enough. But format and order etc when writing the needed files is maybe difficult.

[obackhouse]

Ah, I see. I don’t know the psi4 API well enough to realize this was a bug and not my error. Unfortunately that fnocc doesn’t seem to support my needs:

psi4.driver.p4util.exceptions.ManagedMethodError: select_mp2: Method 'mp2' with MP2_TYPE 'CONV' and REFERENCE 'UHF' not directable to QC_MODULE 'FNOCC'

Thanks for your help - I’ll keep an eye on the bug report for now.

[hokru]

We don’t have too many UHF implementations that is true.

Perhaps meanwhile, psi4numpy can help you in some capacity.
It should be decently fast if you turn on optimizations for the einsum operations.
In any case it showcases the psi4 API (https://github.com/psi4/psi4numpy/tree/master/Tutorials)
https://github.com/psi4/psi4numpy/blob/master/Moller-Plesset/UMP2_Spin_Adapted.py

[obackhouse]

Yes, I’ve used that resource a lot to start to familiarize myself with the API (also contributed to it recently ). However, MP2 is just a test case, and really I need access to the more involved methods which I don’t really want to take the time out to implement myself.