Single test failure from full ctest suite (scf-coverage)

build
1

I recently installed psi4 on a desktop ubuntu machine, and got through the compilation/build process with no alarming error messages or showstopping warnings. When I ran the test suite, my build only failed one test: the scf-coverage test. Based on my inspection of the log file (copied below) it appears to have failed it by getting the wrong energy in the tenths place.

I’ll expose my unfamiliarity with quantum chemistry software in general here, but to me this test looks like several SCF calculations with different small systems. It seems hard for the test to have gone wrong if all the other tests went right, because of this (presumably many other tests use an SCF solver…). Is there something else that could be wrong here? If my build is somehow flawed, what steps should I take to correct the flaw?

Below is a record of the command I ran and the logfile it produced. I didn’t think copying the block of text that said my build passed every other test but this one would be useful, but I would be happy to supply whatever additional information is deemed necessary. Thanks for your help!

$ ctest -I 352,352 -j 6
Test project /home/louis/psi4/objdir
Start 352: scf-coverage
1/1 Test #352: scf-coverage …***Failed 0.80 sec

0% tests passed, 1 tests failed out of 1

Label Time Summary:
psi = 0.80 sec (1 test)
scf = 0.80 sec (1 test)

Total Test time (real) = 1.28 sec

The following tests FAILED:
352 - scf-coverage (Failed)
Errors while running CTest
$ cat Testing/Temporary/LastTest.log
Start testing: Sep 25 15:43 EDT

352/367 Testing: scf-coverage
352/367 Test: scf-coverage
Command: “/usr/bin/python2.7” “/home/louis/psi4/tests/runtest.py” “/home/louis/psi4/tests/scf-coverage/input.dat” “/home/louis/psi4/objdir/testresults.log” “false” “/home/louis/psi4” “false” “/home/louis/psi4/objdir/tests/scf-coverage/output.dat” “/home/louis/psi4/objdir/stage/usr/local/psi4/bin/psi4” “/home/louis/psi4/objdir/stage/usr/local/psi4/share/psi4” “/home/louis/psi4/objdir/stage/usr/local/psi4/lib/”
Directory: /home/louis/psi4/objdir/tests/scf-coverage
“scf-coverage” start time: Sep 25 15:43 EDT
Output:

Lithium anion stability...........................................PASSED
Water RHF.........................................................PASSED
Water Perturb Y RHF: computed value (-81.2791727) does not match (-81.3546428) to 6 digits.

Traceback (most recent call last):
File “/home/louis/psi4/objdir/stage/usr/local/psi4/bin/psi4”, line 260, in
exec(content)
File “”, line 49, in
File “/home/louis/psi4/objdir/stage/usr/local/psi4/lib//psi4/driver/p4util/util.py”, line 226, in compare_values
raise TestComparisonError(message)

TestComparisonError: Water Perturb Y RHF: computed value (-81.2791727) does not match (-81.3546428) to 6 digits.
Exit Status: infile ( 1 ); autotest ( None ); sowreap ( None ); overall ( 1 )

Test time = 0.80 sec

Test Failed.
“scf-coverage” end time: Sep 25 15:43 EDT
“scf-coverage” time elapsed: 00:00:00

End testing: Sep 25 15:43 EDT

psi = 0.80 sec

scf = 0.80 sec

$

2

Hello, yes we recently broke this test on the current master. Please try the following test:

molecule water {
O  0.000000000000  0.000000000000 -0.075791843589
H  0.000000000000 -0.866811828967  0.601435779270
H  0.000000000000  0.866811828967  0.601435779270
symmetry c1
no_reorient
no_com
}

set scf_type df
set damping_percentage 0
set stability_analysis none
scf_e = energy("SCF")
compare_values(-75.98977754532046, scf_e, 6, "Water RHF") #TEST

set perturb_h true
set perturb_with dipole_y
set perturb_magnitude 1
scf_e = energy("SCF")
compare_values(-81.35464281519849, scf_e, 6, "Water Perturb Y RHF") #TEST

Passing the full test suite outside of a full release (1.0, 1.1, …) is not a requirement as we sometimes cause ancillary/minor issues with our changes. It is recommended to use our binaries or a full release for most computations.

3

Thanks for your response.

I’m concerned that I don’t understand the ctest system well enough. When I tried to replace test 352 with the input file you inserted above, it failed immediately. I assume that’s because the ‘.ref’ file in the scf-coverage dir was not changed, but I don’t really know. Could you be more explicit with what you meant by ‘please try the following test’?

As a final point, I wasn’t trying to use a bleeding edge version of the package. I would happily roll back to 1.1 (which was pretty darn recent anyway). Because I only missed one test I was hoping I could figure out what’s going on with it and move forward. But even if rolling back and recompiling everything is what I really should do, I’d appreciate you explaining how to run an alternative test using your harness. I’m just trying to learn more about the package and how tools like cmake and ctest work in general.

4

Ah, so the test folders are laid out as psi4/tests/test_name which will contain a input.dat and CMakeLists.txt. The input.dat its the file that will be run and the CMakeLists.txt basically contains a bit of metadata that tells CMake what categories the tests falls under and which files to add.

What I was suggesting above was to run the pasted text by dropping it to a file and running the Psi4 program. If you are running from a “staged” version, that is before you run make install you can find the Psi4 binary in build_dir/stage/.../psi4/bin/psi4 where the ... represents the install directory you gave to CMake and build_dir is your chosen build directory (usually CMake is run with -Bobjdir). The default install directory will: objdir/stage/usr/local/psi4/bin/psi4.

I think its a bug in the tests more than a bug in the code. So the above will get around any tests wonkiness and attempt to test the code directly.

5

So after I added a basis statement to that code, it worked. The basis I used was cc-pVDZ (which is what was in the benchmark my build didn’t pass).

Thanks for helping me figure it out.

6

Great, glad everything is working and we have a PR in to fix that test in the future.

Hope you like Psi!

7

The scf-coverage fix @dgasmith mentioned is now in. If you’d like to know more about ctest, some of the FAQs around here may help. I notice now that some of the formatting isn’t right, so note that psi4 --test is two dashes, not the endash it appears to be.