Trouble with optimize function, cbs extrapolation and CFOUR

Dear Developers,

I am using the version 1.3 of PSI4.

I am trying to use the optimize function to carry out a cbs extrapolation of the geometry by using the CFOUR interface to employ its analytic derivatives at both the SCF and CCSD(T) levels of theory.
The version of CFOUR I am using is 1.0.

For testing purposes, I use this input file

 molecule  {
C       -0.0000005354           -0.5581834557            0.0000010991
F        1.1062856823            0.2390086133            0.0000002644
F       -1.1062865090            0.2390099455           -0.0000024344
H       -0.0000009557           -1.1553525022           -0.9121077097
H        0.0000023178           -1.1553551009            0.9121087807
}

memory 18 GB

set {
 freeze_core true
 cfour_SCF_CONV=8
 cfour_LINEQ_CONV=8
 cfour_FROZEN_CORE=1
 cfour_CC_PROGRAM=ECC
 cfour_ABCDTYPE=AOBASIS
 cfour_CC_MAXCYC=500
 cfour_SCF_MAXCYC=500
 cfour_SCF_DAMPING=500
 rms_force_g_convergence 8
 geom_maxiter 300
 }

optimize(cbs, scf_wfn='c4-scf', scf_basis ='cc-pv[tq]z', corl_wfn='c4-ccsd(t)', corl_basis='cc-pv[tq]z') "

But after a while the calculations stopped with the following errors

Traceback (most recent call last):
  File "/home/me/psi4conda/bin/psi4", line 287, in <module>
    exec(content)
  File "<string>", line 39, in <module>
  File "/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/driver.py", line 1052, in optimize
    G, wfn = gradient(lowername, return_wfn=True, molecule=moleculeclone, **kwargs)
  File "/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/driver.py", line 640, in gradient
    return name(gradient, kwargs.pop('label', 'custom function'), ptype='gradient', **kwargs)
  File "/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/driver_cbs.py", line 1582, in cbs
    response = func(molecule=molecule, **kwargs)
  File "/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/driver.py", line 691, in gradient
    wfn = procedures['gradient'][lowername](lowername, molecule=molecule, **kwargs)
  File "/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/procrouting/interface_cfour.py", line 375, in run_cfour
    if core.variable('CFOUR ERROR CODE'):
  File "/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/p4util/python_helpers.py", line 583, in _core_variable
    raise KeyError("psi4.core.variable: Requested variable " + key + " was not set!\n")

KeyError: 'psi4.core.variable: Requested variable CFOUR ERROR CODE was not set!\n'


Printing out the relevant lines from the Psithon --> Python processed input file:
    core.set_global_option("CFOUR_CC_MAXCYC", 500)
    core.set_global_option("CFOUR_SCF_MAXCYC", 500)
    core.set_global_option("CFOUR_SCF_DAMPING", 500)
    core.set_global_option("RMS_FORCE_G_CONVERGENCE", 8)
    core.set_global_option("GEOM_MAXITER", 300)
--> optimize(cbs, scf_wfn='c4-scf', scf_basis ='cc-pv[tq]z', corl_wfn='c4-ccsd(t)', corl_basis='cc-pv[tq]z')

    Psi4 stopped on: Tuesday, 09 April 2019 06:12PM
    Psi4 wall time for execution: 0:00:35.81

*** Psi4 encountered an error. Buy a developer more coffee!
*** Resources and help at github.com/psi4/psi4.

So, I switched back to PSI4 version 1.2.1; using the same input file this time I got this error:

Traceback (most recent call last):

File “/home/me/psi4conda/bin/psi4”, line 269, in
exec(content)
File “”, line 41, in
File “/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/driver.py”, line 1111, in optimize
G, wfn = gradient(lowername, return_wfn=True, molecule=moleculeclone, **kwargs)
File “/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/driver.py”, line 573, in gradient
return name(gradient, kwargs.pop(‘label’, ‘custom function’), ptype=‘gradient’, **kwargs)
File “/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/driver_cbs.py”, line 1509, in cbs
response = func(molecule=molecule, **kwargs)
File “/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/driver.py”, line 638, in gradient
wfn = procedures[‘gradient’][lowername](lowername, molecule=molecule, **kwargs)
File “/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/procrouting/interface_cfour.py”, line 254, in run_cfour
psivar, c4grad, c4mol = qcdb.cfour.harvest(qcdbmolecule, c4out, **c4files)
File “/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/qcdb/cfour.py”, line 519, in harvest
outPsivar, outMol, outGrad = harvest_output(c4out)
File “/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/qcdb/cfour.py”, line 55, in harvest_output
psivar, c4coord, c4grad = harvest_outfile_pass(outpass)
File “/home/me/psi4conda/lib//python3.6/site-packages/psi4/driver/qcdb/cfour.py”, line 102, in harvest_outfile_pass
print(‘matched nre’)

OSError: [Errno 5] Input/output error

Psi4 stopped on: Wednesday, 10 April 2019 09:15AM
Psi4 wall time for execution: 0:00:42.09

*** Psi4 encountered an error. Buy a developer more coffee!
*** Resources and help at github.com/psi4/psi4.

Please note that I tested the CFOUR binary and it seems to work properly.

Is there anything wrong in my input file?

Many thanks for your help.
Best regards,
Andrea

At what point in the 1.3 output did the program stop? Had any gradients already finished?

Hi,
yes, in both cases (i.e. with 1.3 and 1.2.1 versions) it seems that at least the calculation of one gradient was finished.
Below you can find the last lines before the error (for version 1.3)

CFOUR scratch file GRD has been read
5 0.0000000000
6.0000000000 0.0000005714 -1.0567977769 -0.0000001222
9.0000000000 2.0905777947 0.4496788579 0.0000000459
9.0000000000 -2.0905785541 0.4496780956 0.0000005174
1.0000000000 -0.0000016344 -2.1852787770 -1.7236397222
1.0000000000 0.0000091451 -2.1852940565 1.7236305591
6.0000000000 -0.0000001009 -0.0195451287 0.0000008527
9.0000000000 0.0243971225 0.0153290410 -0.0000001223
9.0000000000 -0.0243975734 0.0153296240 -0.0000006929
1.0000000000 -0.0000000981 -0.0055563712 -0.0078896362
1.0000000000 0.0000006499 -0.0055571650 0.0078895986

CFOUR scratch file DIPOL has been read
0.0000012966 -0.8677827262 0.0000001870

//>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>//
// Cfour c4-scf Gradient Results //
//<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<//

Variable Map:

“CURRENT DIPOLE X” => 0.000001565430
“CURRENT DIPOLE Y” => -2.205683665325
“CURRENT DIPOLE Z” => 0.000007109580
“CURRENT ENERGY” => -237.994122055436
“CURRENT REFERENCE ENERGY” => -237.994122055436
“HF TOTAL ENERGY” => -237.994122055436
“NUCLEAR REPULSION ENERGY” => 76.924690979800
“SCF TOTAL ENERGY” => -237.994122055436

New Matrix (Symmetry 0)

Irrep: 1 Size: 5 x 3

             1                   2                   3

1    -0.00000011623287    -0.01954512869726     0.00000091148783
2     0.02439713452465     0.01532902186189    -0.00000013589729
3    -0.02439756137407     0.01532964313586    -0.00000077151864
4    -0.00000009194554    -0.00555639493040    -0.00788961948760
5     0.00000063502783    -0.00555714127009     0.00788961531570

//>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>//
// Cfour c4-scf Gradient Results //
//<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<//

Variable Map:

“CURRENT DIPOLE X” => 0.000001565430
“CURRENT DIPOLE Y” => -2.205683665325
“CURRENT DIPOLE Z” => 0.000007109580
“CURRENT ENERGY” => -237.994122055436
“CURRENT REFERENCE ENERGY” => -237.994122055436
“HF TOTAL ENERGY” => -237.994122055436
“NUCLEAR REPULSION ENERGY” => 76.924690979800
“SCF TOTAL ENERGY” => -237.994122055436

New Matrix (Symmetry 0)

Irrep: 1 Size: 5 x 3

             1                   2                   3

1    -0.00000011623287    -0.01954512869726     0.00000091148783
2     0.02439713452465     0.01532902186189    -0.00000013589729
3    -0.02439756137407     0.01532964313586    -0.00000077151864
4    -0.00000009194554    -0.00555639493040    -0.00788961948760
5     0.00000063502783    -0.00555714127009     0.00788961531570

Thank you for your help.
Cheers,
Andrea

It looks like one of the other developers already fixed this issue after 1.3 was released. We’re currently hashing out what commits we want to put in the ofifical v1.3.1 release. When we’ve decided, I’ll let you know if that fix made the list.

While you can always compile the current version of Psi from source, using an official release would be simpler.

Thank you very much!

Anyway, also the previous stable version (1.2.1) seems to have the same problems, at least when coupled to CFOUR.

I will follow your suggestion and wait for the fix in the next release.

Cheers,
Andrea

Based on the error message you posted, the error in 1.2.1 is different. In any event, we won’t be fixing 1.2.1.

The fix made it into 1.3.1, which should be coming soon. I’ll post here when it’s available via conda.

And, 1.3.1 is now available on conda. See here for download instructions.

This should fix your problem. Let us know, either way.

Hi,

Dear Jonathan,

it seems that there is still some problems.

Below you may find the last lines (I used the same input file as before).

Cheers,

Andrea

  CFOUR scratch file DIPOL has been read

        0.0000013359       -0.8730154964        0.0000001470

Traceback (most recent call last):

  File "/home/cs/psi4conda/bin/psi4", line 287, in &lt;module&gt;

    exec(content)

  File "&lt;string&gt;", line 39, in &lt;module&gt;

  File "/home/cs/psi4conda/lib//python3.6/site-packages/psi4/driver/driver.py", line 1052, in optimize

    G, wfn = gradient(lowername, return_wfn=True, molecule=moleculeclone, **kwargs)

  File "/home/cs/psi4conda/lib//python3.6/site-packages/psi4/driver/driver.py", line 640, in gradient

    return name(gradient, kwargs.pop('label', 'custom function'), ptype='gradient', **kwargs)

  File "/home/cs/psi4conda/lib//python3.6/site-packages/psi4/driver/driver_cbs.py", line 1583, in cbs

    response = func(molecule=molecule, **kwargs)

  File "/home/cs/psi4conda/lib//python3.6/site-packages/psi4/driver/driver.py", line 691, in gradient

    wfn = procedures['gradient'][lowername](lowername, molecule=molecule, **kwargs)

  File "/home/cs/psi4conda/lib//python3.6/site-packages/psi4/driver/procrouting/interface_cfour.py", line 252, in run_cfour

    psivar, c4grad, c4mol = qcdb.cfour.harvest(qcdbmolecule, c4out, **c4files)

  File "/home/cs/psi4conda/lib//python3.6/site-packages/psi4/driver/qcdb/cfour.py", line 569, in harvest

    outPsivar, outMol, outGrad = harvest_output(c4out)

  File "/home/cs/psi4conda/lib//python3.6/site-packages/psi4/driver/qcdb/cfour.py", line 57, in harvest_output

    psivar, c4coord, c4grad = harvest_outfile_pass(outpass)

  File "/home/cs/psi4conda/lib//python3.6/site-packages/psi4/driver/qcdb/cfour.py", line 104, in harvest_outfile_pass

    print('matched nre')

OSError: [Errno 5] Input/output error

Printing out the relevant lines from the Psithon --&gt; Python processed input file:

    core.set_global_option("CFOUR_CC_MAXCYC", 500)

    core.set_global_option("CFOUR_SCF_MAXCYC", 500)

    core.set_global_option("CFOUR_SCF_DAMPING", 500)

    core.set_global_option("RMS_FORCE_G_CONVERGENCE", 8)

    core.set_global_option("GEOM_MAXITER", 300)

--&gt; optimize(cbs, scf_wfn='c4-scf', scf_basis ='cc-pv[tq]z', corl_wfn='c4-ccsd(t)', corl_basis='cc-pv[tq]z')

    Psi4 stopped on: Tuesday, 16 April 2019 04:23PM

    Psi4 wall time for execution: 0:07:12.95

*** Psi4 encountered an error. Buy a developer more coffee!

*** Resources and help at [github.com/psi4/psi4](http://github.com/psi4/psi4).

Thanks for the report. I’ll give this a closer look as soon as I can, but that may not be very soon. I’m currently investigating another very high priority bug.

If you don’t hear from me again in a week, mention me here.

reproduced locally. thanks for report

Investigation on this one has passed to Lori. Let us know if there are other cases where you see this. This bug is a bit tricky to investigate because it takes a lot of computer time to get to the bug.

Hi,

thanks a lot!
By the way, this input file should run faster

molecule {
C -0.0000005354 -0.5581834557 0.0000010991
F 1.1062856823 0.2390086133 0.0000002644
F -1.1062865090 0.2390099455 -0.0000024344
H -0.0000009557 -1.1553525022 -0.9121077097
H 0.0000023178 -1.1553551009 0.9121087807
}

memory 8 GB

set {
freeze_core true
cfour_SCF_CONV=8
cfour_LINEQ_CONV=8
cfour_FROZEN_CORE=1
cfour_SCF_MAXCYC=500
cfour_SCF_DAMPING=500
rms_force_g_convergence 8
geom_maxiter 300
}

optimize(cbs, scf_wfn=‘c4-scf’, scf_basis =‘cc-pv[dtq]z’, corl_wfn=‘c4-mp2’, corl_basis=‘cc-pv[dt]z’)

After a while, using version 1.3.1 it gave me what seems to be the same error

–executable xjoda finished with status 0

CFOUR scratch file GRD has been read
5 0.0000000000
6.0000000000 0.0000005714 -1.0567977769 -0.0000001222
9.0000000000 2.0905777947 0.4496788579 0.0000000459
9.0000000000 -2.0905785541 0.4496780956 0.0000005174
1.0000000000 -0.0000016344 -2.1852787770 -1.7236397222
1.0000000000 0.0000091451 -2.1852940565 1.7236305591
6.0000000000 -0.0000001009 -0.0195451287 0.0000008527
9.0000000000 0.0243971225 0.0153290410 -0.0000001223
9.0000000000 -0.0243975734 0.0153296240 -0.0000006929
1.0000000000 -0.0000000981 -0.0055563712 -0.0078896362
1.0000000000 0.0000006499 -0.0055571650 0.0078895986

CFOUR scratch file DIPOL has been read
0.0000012966 -0.8677827262 0.0000001870

Traceback (most recent call last):
File “/home/mine/psi4conda/bin/psi4”, line 287, in
exec(content)
File “”, line 36, in
File “/home/mine/psi4conda/lib//python3.6/site-packages/psi4/driver/driver.py”, line 1052, in optimize
G, wfn = gradient(lowername, return_wfn=True, molecule=moleculeclone, **kwargs)
File “/home/mine/psi4conda/lib//python3.6/site-packages/psi4/driver/driver.py”, line 640, in gradient
return name(gradient, kwargs.pop(‘label’, ‘custom function’), ptype=‘gradient’, **kwargs)
File “/home/mine/psi4conda/lib//python3.6/site-packages/psi4/driver/driver_cbs.py”, line 1583, in cbs
response = func(molecule=molecule, **kwargs)
File “/home/mine/psi4conda/lib//python3.6/site-packages/psi4/driver/driver.py”, line 691, in gradient
wfn = procedures[‘gradient’][lowername](lowername, molecule=molecule, **kwargs)
File “/home/mine/psi4conda/lib//python3.6/site-packages/psi4/driver/procrouting/interface_cfour.py”, line 252, in run_cfour
psivar, c4grad, c4mol = qcdb.cfour.harvest(qcdbmolecule, c4out, **c4files)
File “/home/mine/psi4conda/lib//python3.6/site-packages/psi4/driver/qcdb/cfour.py”, line 569, in harvest
outPsivar, outMol, outGrad = harvest_output(c4out)
File “/home/mine/psi4conda/lib//python3.6/site-packages/psi4/driver/qcdb/cfour.py”, line 57, in harvest_output
psivar, c4coord, c4grad = harvest_outfile_pass(outpass)
File “/home/mine/psi4conda/lib//python3.6/site-packages/psi4/driver/qcdb/cfour.py”, line 104, in harvest_outfile_pass
print(‘matched nre’)

OSError: [Errno 5] Input/output error

Printing out the relevant lines from the Psithon → Python processed input file:
core.set_global_option(“CFOUR_FROZEN_CORE”, 1)
core.set_global_option(“CFOUR_SCF_MAXCYC”, 500)
core.set_global_option(“CFOUR_SCF_DAMPING”, 500)
core.set_global_option(“RMS_FORCE_G_CONVERGENCE”, 8)
core.set_global_option(“GEOM_MAXITER”, 300)
→ optimize(cbs, scf_wfn=‘c4-scf’, scf_basis =‘cc-pv[dtq]z’, corl_wfn=‘c4-mp2’, corl_basis=‘cc-pv[dt]z’)

Psi4 stopped on: Saturday, 27 April 2019 10:54AM
Psi4 wall time for execution: 0:00:47.64

*** Psi4 encountered an error. Buy a developer more coffee!
*** Resources and help at GitHub - psi4/psi4: Open-Source Quantum Chemistry – an electronic structure package in C++ driven by Python.