Fatal Error: SCF::DF: Disk based algorithm requires 2 (A|B) fitting metrics and an (A|mn) chunk on core. (Memory issue when calculating RESP charge)

Xiaobo · March 18, 2022, 12:30am

Dear PSI4 Developer and User,

I am trying to use PSI4 to calculate RESP charges (GitHub - cdsgroup/resp: A. Alenaizan's restricted electrostatic potential (RESP) plugin to Psi4) for MXene (Ti3C2(OH)2) materials. However, the program produced errors as shown below, which is related to PSI4:

RuntimeError: 
Fatal Error: SCF::DF: Disk based algorithm requires 2 (A|B) fitting metrics and an (A|mn) chunk on core.
         This is 2Q^2 + QNP doubles, where Q is the auxiliary basis size, N is the
         primary basis size, and P is the maximum number of functions in a primary shell.
         For this problem, that is 579894560 bytes before taxes,828420800 bytes after taxes. 

Error occurred in file: /scratch/psilocaluser/conda-builds/psi4-multiout_1638121835567/work/psi4/src/psi4/libfock/DiskDFJK.cc on line: 727
The most recent 5 function calls were:

psi::DiskDFJK::preiterations()

The errors are related to PSI4 package. The input file is shown below.

import resp
import numpy as np
import psi4

mol = psi4.geometry("""
                Ti2 5.894640000000001 4.6752 4.88655
                Ti2 4.273479000000001 1.8975999765561422 4.88655
                Ti2 2.6523220000000003 4.6752 4.88655
                Ti2 1.0311610000000009 1.8975999765561422 4.88655
                C 5.894640000000001 0.9717333177040944 6.09059592
                C 4.273479000000002 3.7493332942602366 6.09059592
                C 2.6523220000000003 0.9717333177040944 6.09059592
                C 1.0311610000000009 3.7493332942602366 6.09059592
                C 1.0311610000000004 0.04586665885204755 3.6825040800000024
                C 5.894640000000002 2.8234666354081903 3.6825040800000024
                C 4.2734830000000015 0.04586665885204755 3.6825040800000024
                C 2.6523220000000007 2.8234666354081903 3.6825040800000024
                Ti1 1.031161 0.04586665885204755 7.37478126
                Ti1 5.894640000000002 2.823466635408189 7.37478126
                Ti1 4.273483000000001 0.04586665885204755 7.37478126
                Ti1 2.6523220000000003 2.823466635408189 7.37478126
                Ti1 5.894640000000001 0.9717333177040951 2.3983187400000006
                Ti1 4.273479000000001 3.7493332942602375 2.3983187400000006
                Ti1 2.6523220000000007 0.9717333177040951 2.3983187400000006
                Ti1 1.0311610000000018 3.7493332942602375 2.3983187400000006
                O 5.894640000000001 4.6752 8.368592162039999
                O 4.273479000000001 1.8975999765561422 8.368592162039999
                O 2.6523220000000003 4.6752 8.368592162039999
                O 1.0311610000000009 1.8975999765561422 8.368592162039999
                O 5.894640000000001 4.6752 1.4045078379599998
                O 4.273479000000001 1.8975999765561422 1.4045078379599998
                O 2.6523220000000003 4.6752 1.4045078379599998
                O 1.0311610000000009 1.8975999765561422 1.4045078379599998
                H 5.894640000000001 4.6752 9.36859216204
                H 2.6523220000000003 4.6752 0.40450783796000034
                H 4.273479000000001 1.8975999765561422 0.40450783796000006
                H 4.273479000000001 1.8975999765561422 9.36859216204
                H 1.0311610000000018 1.8975999765561422 9.36859216204
                H 5.894640000000001 4.6752 0.4045078379599998
                H 2.6523220000000007 4.6752 9.36859216204
                H 1.0311610000000009 1.8975999765561422 0.40450783796000006
                """
                )
mol.update_geometry()
# Call for first stage fit

options = {'VDW_SCALE_FACTORS'  : [1.4, 1.6, 1.8, 2.0],
           'VDW_POINT_DENSITY'  : 1.0,
           'RESP_A'             : 0.0005,
           'RESP_B'             : 0.1,
           }
options['VDW_RADII'] = {'Ti': 1.96}
options['reference'] ='rhf'
options['scf_type'] ='direct'
charges1 = resp.resp([mol], options)

print('Electrostatic Potential Charges')
print(charges1[0])
print('Restrained Electrostatic Potential Charges')
print(charges1[1])

I actually set up the scratch folder in .bashrc and .bash_profile. If type in echo $PSI_SCRATCH, it print /global/cscratch1/sd/linx6, the scratch folder has at least 10 TB. (I am running the code on NERSC)

Could you help me out?

Thank you so much in advance!

jmisiewicz · March 18, 2022, 1:25am

This error is unrelated to RESP but occurs during the SCF step. You have not allocated enough memory for the SCF to complete. Read this section of the docs: Psithon: Structuring an Input File

Xiaobo · March 18, 2022, 4:56am

Thank you so much for your response! That works now. But when I increase the system size (add more atoms), it reaches to integral step, but still failed as shown below:

 => Loading Basis Set <=

    Name: (6-31G* AUX)
    Role: JKFIT
    Keyword: DF_BASIS_SCF
    atoms 1-16, 49-80 entry TI         line  1474 file /global/project/projectdirs/m1046/Xiaobo/installed_software/intel/oneapi/intelpython/latest/envs/resp/share/psi4/basis/def2-universal-jkfit.gbs 
    atoms 17-48       entry C          line   121 file /global/project/projectdirs/m1046/Xiaobo/installed_software/intel/oneapi/intelpython/latest/envs/resp/share/psi4/basis/cc-pvdz-jkfit.gbs 
    atoms 81-112      entry O          line   221 file /global/project/projectdirs/m1046/Xiaobo/installed_software/intel/oneapi/intelpython/latest/envs/resp/share/psi4/basis/cc-pvdz-jkfit.gbs 
    atoms 113-144     entry H          line    51 file /global/project/projectdirs/m1046/Xiaobo/installed_software/intel/oneapi/intelpython/latest/envs/resp/share/psi4/basis/cc-pvdz-jkfit.gbs 

==> Integral Setup <==

  DFHelper Memory: AOs need 720.343 GiB; user supplied 1396.984 GiB. Using in-core AOs.

Traceback (most recent call last):
  File "/global/project/projectdirs/m1046/Xiaobo/installed_software/resp/examples/try.py", line 106, in <module>
    charges1 = resp.resp([mol], options)
  File "/global/project/projectdirs/m1046/Xiaobo/installed_software/resp/resp/driver.py", line 156, in resp
    psi4.prop(options['METHOD_ESP'], properties=['GRID_ESP'])
  File "/global/project/projectdirs/m1046/Xiaobo/installed_software/intel/oneapi/intelpython/latest/envs/resp/lib/python3.9/site-packages/psi4/driver/driver.py", line 907, in properties
    wfn = procedures['properties'][lowername](lowername, **kwargs)
  File "/global/project/projectdirs/m1046/Xiaobo/installed_software/intel/oneapi/intelpython/latest/envs/resp/lib/python3.9/site-packages/psi4/driver/procrouting/proc.py", line 2993, in run_scf_property
    scf_wfn = run_scf(name, scf_do_properties=False, do_timer=False, **kwargs)
  File "/global/project/projectdirs/m1046/Xiaobo/installed_software/intel/oneapi/intelpython/latest/envs/resp/lib/python3.9/site-packages/psi4/driver/procrouting/proc.py", line 2390, in run_scf
    scf_wfn = scf_helper(name, post_scf=False, **kwargs)
  File "/global/project/projectdirs/m1046/Xiaobo/installed_software/intel/oneapi/intelpython/latest/envs/resp/lib/python3.9/site-packages/psi4/driver/procrouting/proc.py", line 1607, in scf_helper
    e_scf = scf_wfn.compute_energy()
  File "/global/project/projectdirs/m1046/Xiaobo/installed_software/intel/oneapi/intelpython/latest/envs/resp/lib/python3.9/site-packages/psi4/driver/procrouting/scf_proc/scf_iterator.py", line 83, in scf_compute_energy
    self.initialize()
  File "/global/project/projectdirs/m1046/Xiaobo/installed_software/intel/oneapi/intelpython/latest/envs/resp/lib/python3.9/site-packages/psi4/driver/procrouting/scf_proc/scf_iterator.py", line 193, in scf_initialize
    self.initialize_jk(self.memory_jk_, jk=jk)
  File "/global/project/projectdirs/m1046/Xiaobo/installed_software/intel/oneapi/intelpython/latest/envs/resp/lib/python3.9/site-packages/psi4/driver/procrouting/scf_proc/scf_iterator.py", line 128, in initialize_jk
    jk.initialize()
MemoryError: std::bad_alloc

As you can see, it is finally killed. (I set psi4.set_memory('2000 GB'))
Do you know why this happens? Thank you so much for your help!. I attached the geometry input as shown below:

Geometry (in Angstrom), charge = 0, multiplicity = 1:

       Center              X                  Y                   Z               Mass       
    ------------   -----------------  -----------------  -----------------  -----------------
         TI           5.674064824248     4.991329330058    -0.000000000000    47.947941980000
         TI           4.052903824248    -3.341470693386    -0.000000000000    47.947941980000
         TI           2.431742824248    -0.563870716830    -0.000000000000    47.947941980000
         TI           0.810581824248     2.213729259726    -0.000000000000    47.947941980000
         TI          -4.052903175752     4.991329330058    -0.000000000000    47.947941980000
         TI          -5.674064175752    -3.341470693386    -0.000000000000    47.947941980000
         TI           5.674064824248    -0.563870716830    -0.000000000000    47.947941980000
         TI           4.052903824248     2.213729259726    -0.000000000000    47.947941980000
         TI          -0.810581175752     4.991329330058    -0.000000000000    47.947941980000
         TI          -2.431742175752    -3.341470693386    -0.000000000000    47.947941980000
         TI          -4.052903175752    -0.563870716830    -0.000000000000    47.947941980000
         TI          -5.674064175752     2.213729259726    -0.000000000000    47.947941980000
         TI           2.431740824248     4.991329330058    -0.000000000000    47.947941980000
         TI           0.810579824248    -3.341470693386    -0.000000000000    47.947941980000
         TI          -0.810581175752    -0.563870716830    -0.000000000000    47.947941980000
         TI          -2.431742175752     2.213729259726    -0.000000000000    47.947941980000
         C            5.674064824248    -4.267337352238     1.204045920000    12.000000000000
         C            4.052903824248    -1.489737375682     1.204045920000    12.000000000000
         C            2.431742824248     1.287862600874     1.204045920000    12.000000000000
         C            0.810581824248     4.065462577430     1.204045920000    12.000000000000
         C           -4.052903175752    -4.267337352238     1.204045920000    12.000000000000
         C           -5.674064175752    -1.489737375682     1.204045920000    12.000000000000
         C            5.674064824248     1.287862600874     1.204045920000    12.000000000000
         C            4.052903824248     4.065462577430     1.204045920000    12.000000000000
         C           -0.810581175752    -4.267337352238     1.204045920000    12.000000000000
         C           -2.431742175752    -1.489737375682     1.204045920000    12.000000000000
         C           -4.052903175752     1.287862600874     1.204045920000    12.000000000000
         C           -5.674064175752     4.065462577430     1.204045920000    12.000000000000
         C            2.431740824248    -4.267337352238     1.204045920000    12.000000000000
         C            0.810579824248    -1.489737375682     1.204045920000    12.000000000000
         C           -0.810581175752     1.287862600874     1.204045920000    12.000000000000
         C           -2.431742175752     4.065462577430     1.204045920000    12.000000000000
         C           -5.674064175752    -5.193204011090    -1.204045920000    12.000000000000
         C            5.674064824248    -2.415604034534    -1.204045920000    12.000000000000
         C            4.052903824248     0.361995942022    -1.204045920000    12.000000000000
         C            2.431742824248     3.139595918578    -1.204045920000    12.000000000000
         C           -2.431742175752    -5.193204011090    -1.204045920000    12.000000000000
         C           -4.052903175752    -2.415604034534    -1.204045920000    12.000000000000
         C           -5.674064175752     0.361995942022    -1.204045920000    12.000000000000
         C            5.674064824248     3.139595918578    -1.204045920000    12.000000000000
         C            0.810579824248    -5.193204011090    -1.204045920000    12.000000000000
         C           -0.810581175752    -2.415604034534    -1.204045920000    12.000000000000
         C           -2.431742175752     0.361995942022    -1.204045920000    12.000000000000
         C           -4.052903175752     3.139595918578    -1.204045920000    12.000000000000
         C            4.052901824248    -5.193204011090    -1.204045920000    12.000000000000
         C            2.431740824248    -2.415604034534    -1.204045920000    12.000000000000
         C            0.810579824248     0.361995942022    -1.204045920000    12.000000000000
         C           -0.810581175752     3.139595918578    -1.204045920000    12.000000000000
         TI          -5.674064175752    -5.193204011090     2.488231260000    47.947941980000
         TI           5.674064824248    -2.415604034534     2.488231260000    47.947941980000
         TI           4.052903824248     0.361995942022     2.488231260000    47.947941980000
         TI           2.431742824248     3.139595918578     2.488231260000    47.947941980000
         TI          -2.431742175752    -5.193204011090     2.488231260000    47.947941980000
         TI          -4.052903175752    -2.415604034534     2.488231260000    47.947941980000
         TI          -5.674064175752     0.361995942022     2.488231260000    47.947941980000
         TI           5.674064824248     3.139595918578     2.488231260000    47.947941980000
         TI           0.810579824248    -5.193204011090     2.488231260000    47.947941980000
         TI          -0.810581175752    -2.415604034534     2.488231260000    47.947941980000
         TI          -2.431742175752     0.361995942022     2.488231260000    47.947941980000
         TI          -4.052903175752     3.139595918578     2.488231260000    47.947941980000
         TI           4.052901824248    -5.193204011090     2.488231260000    47.947941980000
         TI           2.431740824248    -2.415604034534     2.488231260000    47.947941980000
         TI           0.810579824248     0.361995942022     2.488231260000    47.947941980000
         TI          -0.810581175752     3.139595918578     2.488231260000    47.947941980000
         TI           5.674064824248    -4.267337352238    -2.488231260000    47.947941980000
         TI           4.052903824248    -1.489737375682    -2.488231260000    47.947941980000
         TI           2.431742824248     1.287862600874    -2.488231260000    47.947941980000
         TI           0.810581824248     4.065462577430    -2.488231260000    47.947941980000
         TI          -4.052903175752    -4.267337352238    -2.488231260000    47.947941980000
         TI          -5.674064175752    -1.489737375682    -2.488231260000    47.947941980000
         TI           5.674064824248     1.287862600874    -2.488231260000    47.947941980000
         TI           4.052903824248     4.065462577430    -2.488231260000    47.947941980000
         TI          -0.810581175752    -4.267337352238    -2.488231260000    47.947941980000
         TI          -2.431742175752    -1.489737375682    -2.488231260000    47.947941980000
         TI          -4.052903175752     1.287862600874    -2.488231260000    47.947941980000
         TI          -5.674064175752     4.065462577430    -2.488231260000    47.947941980000
         TI           2.431740824248    -4.267337352238    -2.488231260000    47.947941980000
         TI           0.810579824248    -1.489737375682    -2.488231260000    47.947941980000
         TI          -0.810581175752     1.287862600874    -2.488231260000    47.947941980000
         TI          -2.431742175752     4.065462577430    -2.488231260000    47.947941980000
         O            5.674064824248     4.991329330058     3.482042162040    15.994914619570
         O            4.052903824248    -3.341470693386     3.482042162040    15.994914619570
         O            2.431742824248    -0.563870716830     3.482042162040    15.994914619570
         O            0.810581824248     2.213729259726     3.482042162040    15.994914619570
         O           -4.052903175752     4.991329330058     3.482042162040    15.994914619570
         O           -5.674064175752    -3.341470693386     3.482042162040    15.994914619570
         O            5.674064824248    -0.563870716830     3.482042162040    15.994914619570
         O            4.052903824248     2.213729259726     3.482042162040    15.994914619570
         O           -0.810581175752     4.991329330058     3.482042162040    15.994914619570
         O           -2.431742175752    -3.341470693386     3.482042162040    15.994914619570
         O           -4.052903175752    -0.563870716830     3.482042162040    15.994914619570
         O           -5.674064175752     2.213729259726     3.482042162040    15.994914619570
         O            2.431740824248     4.991329330058     3.482042162040    15.994914619570
         O            0.810579824248    -3.341470693386     3.482042162040    15.994914619570
         O           -0.810581175752    -0.563870716830     3.482042162040    15.994914619570
         O           -2.431742175752     2.213729259726     3.482042162040    15.994914619570
         O            5.674064824248     4.991329330058    -3.482042162040    15.994914619570
         O            4.052903824248    -3.341470693386    -3.482042162040    15.994914619570
         O            2.431742824248    -0.563870716830    -3.482042162040    15.994914619570
         O            0.810581824248     2.213729259726    -3.482042162040    15.994914619570
         O           -4.052903175752     4.991329330058    -3.482042162040    15.994914619570
         O           -5.674064175752    -3.341470693386    -3.482042162040    15.994914619570
         O            5.674064824248    -0.563870716830    -3.482042162040    15.994914619570
         O            4.052903824248     2.213729259726    -3.482042162040    15.994914619570
         O           -0.810581175752     4.991329330058    -3.482042162040    15.994914619570
         O           -2.431742175752    -3.341470693386    -3.482042162040    15.994914619570
         O           -4.052903175752    -0.563870716830    -3.482042162040    15.994914619570
         O           -5.674064175752     2.213729259726    -3.482042162040    15.994914619570
         O            2.431740824248     4.991329330058    -3.482042162040    15.994914619570
         O            0.810579824248    -3.341470693386    -3.482042162040    15.994914619570
         O           -0.810581175752    -0.563870716830    -3.482042162040    15.994914619570
         O           -2.431742175752     2.213729259726    -3.482042162040    15.994914619570
         H            0.810581824248     2.213729259726     4.482042162040     1.007825032230
         H            4.052903824248     2.213729259726    -4.482042162040     1.007825032230
         H           -5.674064175752     2.213729259726    -4.482042162040     1.007825032230
         H            2.431740824248     4.991329330058    -4.482042162040     1.007825032230
         H           -2.431742175752    -3.341470693386    -4.482042162040     1.007825032230
         H           -2.431742175752     2.213729259726     4.482042162040     1.007825032230
         H           -4.052903175752     4.991329330058    -4.482042162040     1.007825032230
         H            4.052903824248    -3.341470693386    -4.482042162040     1.007825032230
         H           -5.674064175752    -3.341470693386     4.482042162040     1.007825032230
         H           -2.431742175752    -3.341470693386     4.482042162040     1.007825032230
         H            4.052903824248     2.213729259726     4.482042162040     1.007825032230
         H            0.810579824248    -3.341470693386     4.482042162040     1.007825032230
         H            0.810579824248    -3.341470693386    -4.482042162040     1.007825032230
         H            2.431740824248     4.991329330058     4.482042162040     1.007825032230
         H            4.052903824248    -3.341470693386     4.482042162040     1.007825032230
         H            5.674064824248    -0.563870716830    -4.482042162040     1.007825032230
         H            0.810581824248     2.213729259726    -4.482042162040     1.007825032230
         H           -0.810581175752     4.991329330058     4.482042162040     1.007825032230
         H            2.431742824248    -0.563870716830     4.482042162040     1.007825032230
         H           -4.052903175752     4.991329330058     4.482042162040     1.007825032230
         H           -5.674064175752     2.213729259726     4.482042162040     1.007825032230
         H            5.674064824248    -0.563870716830     4.482042162040     1.007825032230
         H           -2.431742175752     2.213729259726    -4.482042162040     1.007825032230
         H            2.431742824248    -0.563870716830    -4.482042162040     1.007825032230
         H            5.674064824248     4.991329330058    -4.482042162040     1.007825032230
         H           -4.052903175752    -0.563870716830    -4.482042162040     1.007825032230
         H           -0.810581175752    -0.563870716830     4.482042162040     1.007825032230
         H           -0.810581175752    -0.563870716830    -4.482042162040     1.007825032230
         H           -0.810581175752     4.991329330058    -4.482042162040     1.007825032230
         H           -4.052903175752    -0.563870716830     4.482042162040     1.007825032230
         H           -5.674064175752    -3.341470693386    -4.482042162040     1.007825032230
         H            5.674064824248     4.991329330058     4.482042162040     1.007825032230

zachglick · March 18, 2022, 12:18pm

Hi!

I think your second issue is that you’ve told psi4 to use more memory than is physically available. Does your machine have 2000 GB of memory? The biggest machines that I work with have ~256 GB memory. Note that memory refers to the amount of RAM available. This is not the same as the size of your scratch space.

I would try rerunning the calculation with a smaller value specified in the get_memory call.

Best,
Zach Glick