Threading in python API causing SCF issues

Dear Forum,

I’m using the following input, with and without psi4.core.set_num_threads(8):

import psi4
from psi4.driver import qcdb
import numpy as np
psi4.core.set_num_threads(8)
geom=psi4.geometry("""0 1
C 1.4066424120209144 0.6335710909590454 -0.2495916639798697
C 0.7552834624848742 -0.6938282650379655 0.025317291551563384
C -0.7273256284968848 -0.6603564302925777 -0.3166017395592235
C -1.4354902144618316 0.3788242188085613 0.4973540319018165
H 0.7470456513609004 1.3151951118514082 -0.8394130558071629
H 2.3031734013121734 0.466691521758824 -0.8827549803090603
H 1.6601760740823526 1.1031086278739168 0.7326669096833057
H 0.9188372419337766 -1.0445071927991643 1.064439404370322
H 1.1897079830937536 -1.4981424290322438 -0.6367354838816501
H -0.8539718746187484 -0.5012574521331737 -1.3893709039636322
H -1.1394156514501503 -1.6654904718222394 -0.013080101389100762
H -0.8609588627728423 1.3396694065668024 0.5559806030785837
H -2.4049477449213805 0.6490573196443288 0.01403750732183584
H -1.6217001443690686 0.05685985612102404 1.5358214326126451""")
psi4.set_options({"GEOM_MAXITER": 200,"MAXITER": 200,"SCF_TYPE": "DF","BASIS": "STO-3G"})
psi4.energy('scf')
psi4.optimize('scf')

When I do not use psi4.core.set_num_threads(8), the SCF converges easily:

                       Total Energy        Delta E     RMS |[F,P]|

@DF-RHF iter SAD:  -151.51622009089326   -1.51516e+02   0.00000e+00 
@DF-RHF iter   1:  -155.45798558481556   -3.94177e+00   3.45164e-03 DIIS
@DF-RHF iter   2:  -155.46543988803791   -7.45430e-03   6.85542e-04 DIIS
@DF-RHF iter   3:  -155.46581335443705   -3.73466e-04   1.56658e-04 DIIS
@DF-RHF iter   4:  -155.46583024921415   -1.68948e-05   8.07666e-06 DIIS
@DF-RHF iter   5:  -155.46583030049607   -5.12819e-08   1.12901e-06 DIIS
@DF-RHF iter   6:  -155.46583030127013   -7.74065e-10   1.50348e-07 DIIS
@DF-RHF iter   7:  -155.46583030128869   -1.85594e-11   1.99679e-08 DIIS
@DF-RHF iter   8:  -155.46583030128940   -7.10543e-13   2.55774e-09 DIIS
Energy and wave function converged.

When I do use it, the SCF bounces around and does not converge:

@DF-RHF iter 192:  -155.48406057872126   -4.45005e-02   1.13015e-01 DIIS
@DF-RHF iter 193:  -155.16910805383216    3.14953e-01   2.46794e-02 DIIS
@DF-RHF iter 194:  -151.76815976087232    3.40095e+00   2.52451e-02 DIIS
@DF-RHF iter 195:  -155.44121257885720   -3.67305e+00   1.29083e-03 DIIS
@DF-RHF iter 196:  -155.44164465020165   -4.32071e-04   1.01146e-03 DIIS
@DF-RHF iter 197:  -195.67774154007719   -4.02361e+01   8.32327e-02 DIIS
@DF-RHF iter 198:  -162.40305527814721    3.32747e+01   1.57399e-02 DIIS
@DF-RHF iter 199:  -155.44224545508837    6.96081e+00   5.21518e-04 DIIS
@DF-RHF iter 200:  -204.61239348724249   -4.91701e+01   9.91108e-02 DIIS

PsiException: Could not converge SCF iterations in 200 iterations.

I’m using slurm and my submission script look like this:

#!/bin/bash
#SBATCH --exclude=g001,g002,g003,g004,g005,g006,g007,g008,g009,g010,g011,g012,g013,g014,g015,g016,g017,g018,g019
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=8
#SBATCH --threads-per-core=1
#SBATCH --mem-per-cpu=8G
#SBATCH --time=1:00:00
export PATH=/home/woodg07/workspace/impy/impy/psi4/bin:$PATH
ml medsci anaconda3
export PSI_SCRATCH=/home/woodg07/workspace/impy/impy/psi4_scratch
export PYTHONPATH=/home/woodg07/workspace/impy/impy/psi4/lib:$PYTHONPATH
python input  > output

This is some version related stuff:

Python 3.7.8 | packaged by conda-forge | (default, Jul 31 2020, 02:25:08)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import psi4
>>> print(psi4.__version__)
1.4rc2.dev102

Any help here would be greatly appreciated.

Step one is to update to the official 1.4 release. There are a couple bug fixes between that release candidate and the official release that may be important.

Let us know if that fixes the issue or not.

Thank you for your reply. I did a conda install psi4 -c psi4

and ran the job with the same result. i.e. the SCF just bounces around when threads are greater than 1

FYI I also tried using the standalone executable: psi4 -n 8 input output where input looks like:

molecule mymol { 
0 1
C -1.5621252343860528 0.058309806105708156 -0.4305893049404049
C -0.5290160580363157 0.3561645471276945 0.6226157041457774
C 0.7635306724253246 -0.38662645009382257 0.3961463434909243
C 1.3245761208603932 0.009834274764699641 -0.9307681463637978
H -2.587666534494592 0.13813120798596143 0.01500778841887107
H -1.4503908618385477 -0.9604493426515381 -0.8650854651343098
H -1.5331626404635899 0.8059984778963396 -1.267764523051844
H -0.34400754081843 1.4435878141334213 0.6121184071742032
H -0.9581937816010481 0.11055523252248325 1.6240826690680716
H 0.6264986088156281 -1.4734675690127002 0.524490736246237
H 1.4748238129116862 -0.047283192328704676 1.1796649957795247
H 2.0346635042258323 -0.7310722450132222 -1.3499762200756777
H 0.5363059292042349 0.22246119128419187 -1.6889010342214017
H 1.8854162362627984 0.958691113691261 -0.7955338461727224
}
set { 
GEOM_MAXITER 200 
MAXITER 200 
SCF_TYPE DF 
BASIS STO-3G 
 }  
optimize('scf')

and got the same behavior, I also tried changing SCF_TYP to direct and also used a DFT functional in the optimization with the same result. Not really sure how to proceed…are there any other details about the installation I can provide?