Build crash in versioner.py

Kirk · November 15, 2016, 10:37pm

Hi, I’m doing a fresh build of Psi4 from a new git clone (from today). The program seems to build ok all the way to the end before it fails with (using gcc 6.2, python 2.6.6):

– Build files have been written to: /home/kipeters/psi4-new/objdirb/psi4_core-prefix/src/psi4_core-build
[ 95%] Performing build step for ‘psi4_core’
Scanning dependencies of target consolidate_psidatadir
[ 0%] Symlink external resources into PSIDATADIR
[ 0%] Built target consolidate_psidatadir
Scanning dependencies of target update_version
[ 0%] Generating version info
Traceback (most recent call last):
File “versioner.py”, line 327, in
ans = reconcile_and_compute_version_output(quiet=args.formatonly)
File “versioner.py”, line 220, in reconcile_and_compute_version_output
trial_version_long_devel))
ValueError: zero length field name in format
make[5]: *** [CMakeFiles/update_version] Error 1
make[4]: *** [CMakeFiles/update_version.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [psi4_core-prefix/src/psi4_core-stamp/psi4_core-build] Error 2
make[1]: *** [CMakeFiles/psi4_core.dir/all] Error 2

amjames · November 15, 2016, 11:00pm

@Kirk someone might be able to correct me if I am wrong, but I believe we require python 2.7. It is concerning that using python 2.6.6 this was not caught by cmake at the start of the build.

Kirk · November 15, 2016, 11:04pm

I wondered about this, but could have sworn that during the cmake step it said that only v2.6 was required

Kirk · November 16, 2016, 1:10am

Yep, upgrading to Python 2.7.8 did the trick. Now I’m hitting a -fPIC error, so I’m rebuilding with BUILD_FPIC=ON

Kirk · November 16, 2016, 3:49am

Is there a easy way to get psi4-core to link with the static library version of python? My build crashes with:

[ 99%] Linking CXX shared library core.so
/share/apps/bin/ld: /share/apps/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct’ can not be used when making a shared object; recompile with -fPIC
/share/apps/lib/libpython2.7.a: could not read symbols: Bad value
collect2: error: ld returned 1 exit status
make[5]: *** [src/core.so] Error 1
make[4]: *** [src/CMakeFiles/core.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [psi4_core-prefix/src/psi4_core-stamp/psi4_core-build] Error 2
make[1]: *** [CMakeFiles/psi4_core.dir/all] Error 2

Kirk · November 16, 2016, 7:42am

Sorry, my python2.7 build was all messed up (well,
subtly messed up). Everything builds fine now.

loriab · November 19, 2016, 5:31pm

If the libpython linking keeps giving you trouble, psi4 doesn’t actually need to be linked to the python library anymore. It needs python headers to compile against and the interpreter to run but not library dependency. This hasn’t percolated into the main branch yet, but I can help on this if you need it.

Kirk · November 19, 2016, 5:48pm

While I get everything to build and compile seemingly without error, I immediately get a segmentation fault when psi4 is invoked. Calling python with the -v option seems to show all the initialization working correctly. Psi4 was built with gcc 6.2, so I’m thinking of building 4.9.4 to see if that has any effect.

loriab · November 19, 2016, 6:31pm

Hmm, one of our travis-ci continuous integration builds is a 6.2 and others are 4.9, so the 6.2 should be ok. I’m not quite sure what’s going on. Maybe if you send me your ldd -v objdir/stage/.../lib/psi4/core.so, I can see if anything looks suspicious.

Kirk · November 19, 2016, 7:16pm

    linux-vdso.so.1 =>  (0x00007ffe473f6000)
    libpython2.7.so.1.0 => /opt/python/lib/libpython2.7.so.1.0 (0x00002b3f9278a000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00002b3f92b76000)
    libmkl_intel_lp64.so => /share/apps/intel/mkl/lib/intel64/libmkl_intel_lp64.so (0x00002b3f92d7a000)
    libmkl_intel_thread.so => /share/apps/intel/mkl/lib/intel64/libmkl_intel_thread.so (0x00002b3f9349e000)
    libmkl_core.so => /share/apps/intel/mkl/lib/intel64/libmkl_core.so (0x00002b3f94538000)
    libguide.so => /share/apps/intel/Compiler/11.1/064/lib/intel64/libguide.so (0x00002b3f95a66000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b3f95c0c000)
    libmkl_gf_lp64.so => /share/apps/intel/mkl/lib/intel64/libmkl_gf_lp64.so (0x00002b3f95e29000)
    libmkl_gnu_thread.so => /share/apps/intel/mkl/lib/intel64/libmkl_gnu_thread.so (0x00002b3f9654b000)
    libm.so.6 => /lib64/libm.so.6 (0x00002b3f970a8000)
    libifport.so.5 => /share/apps/intel/Compiler/11.1/064/lib/intel64/libifport.so.5 (0x00002b3f9732c000)
    libifcoremt.so.5 => /share/apps/intel/Compiler/11.1/064/lib/intel64/libifcoremt.so.5 (0x00002b3f97465000)
    libimf.so => /share/apps/intel/Compiler/11.1/064/lib/intel64/libimf.so (0x00002b3f9770a000)
    libsvml.so => /share/apps/intel/Compiler/11.1/064/lib/intel64/libsvml.so (0x00002b3f97a9c000)
    libiomp5.so => /share/apps/intel/Compiler/11.1/064/lib/intel64/libiomp5.so (0x00002b3f97cb2000)
    libirc.so => /share/apps/intel/Compiler/11.1/064/lib/intel64/libirc.so (0x00002b3f97e64000)
    libstdc++.so.6 => /share/apps/gcc6.2/lib64/libstdc++.so.6 (0x00002b3f97fa1000)
    libgomp.so.1 => /share/apps/gcc6.2/lib64/libgomp.so.1 (0x00002b3f98322000)
    libgcc_s.so.1 => /share/apps/gcc6.2/lib64/libgcc_s.so.1 (0x00002b3f98550000)
    libc.so.6 => /lib64/libc.so.6 (0x00002b3f98766000)
    libutil.so.1 => /lib64/libutil.so.1 (0x00002b3f98afa000)
    /lib64/ld-linux-x86-64.so.2 (0x00000030e8200000)
    libintlc.so.5 => /share/apps/intel/Compiler/11.1/064/lib/intel64/libintlc.so.5 (0x00002b3f98cfe000)
    librt.so.1 => /lib64/librt.so.1 (0x00002b3f98e3b000)

    Version information:
    ./core.so:
            libgomp.so.1 (GOMP_4.0) => /share/apps/gcc6.2/lib64/libgomp.so.1
            libgcc_s.so.1 (GCC_3.0) => /share/apps/gcc6.2/lib64/libgcc_s.so.1
            libiomp5.so (VERSION) => /share/apps/intel/Compiler/11.1/064/lib/intel64/libiomp5.so
            libpthread.so.0 (GLIBC_2.2.5) => /lib64/libpthread.so.0
            libguide.so (VERSION) => /share/apps/intel/Compiler/11.1/064/lib/intel64/libguide.so
            libm.so.6 (GLIBC_2.2.5) => /lib64/libm.so.6
            libc.so.6 (GLIBC_2.3.2) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.3) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
            libstdc++.so.6 (GLIBCXX_3.4.14) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libstdc++.so.6 (GLIBCXX_3.4.18) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libstdc++.so.6 (CXXABI_1.3.1) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libstdc++.so.6 (CXXABI_1.3.8) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libstdc++.so.6 (CXXABI_1.3.5) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libstdc++.so.6 (GLIBCXX_3.4.22) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libstdc++.so.6 (GLIBCXX_3.4.11) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libstdc++.so.6 (GLIBCXX_3.4.15) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libstdc++.so.6 (CXXABI_1.3.3) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libstdc++.so.6 (GLIBCXX_3.4.9) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libstdc++.so.6 (CXXABI_1.3) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libstdc++.so.6 (GLIBCXX_3.4.20) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libstdc++.so.6 (GLIBCXX_3.4) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libstdc++.so.6 (GLIBCXX_3.4.21) => /share/apps/gcc6.2/lib64/libstdc++.so.6
            libdl.so.2 (GLIBC_2.2.5) => /lib64/libdl.so.2
    /opt/python/lib/libpython2.7.so.1.0:
            libdl.so.2 (GLIBC_2.2.5) => /lib64/libdl.so.2
            libutil.so.1 (GLIBC_2.2.5) => /lib64/libutil.so.1
            libpthread.so.0 (GLIBC_2.2.5) => /lib64/libpthread.so.0
            libm.so.6 (GLIBC_2.2.5) => /lib64/libm.so.6
            libc.so.6 (GLIBC_2.3) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /lib64/libdl.so.2:
            ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
            libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /share/apps/intel/mkl/lib/intel64/libmkl_intel_lp64.so:
            libdl.so.2 (GLIBC_2.2.5) => /lib64/libdl.so.2
    /share/apps/intel/mkl/lib/intel64/libmkl_intel_thread.so:
            libdl.so.2 (GLIBC_2.2.5) => /lib64/libdl.so.2
    /share/apps/intel/mkl/lib/intel64/libmkl_core.so:
            libdl.so.2 (GLIBC_2.2.5) => /lib64/libdl.so.2
    /share/apps/intel/Compiler/11.1/064/lib/intel64/libguide.so:
            libdl.so.2 (GLIBC_2.2.5) => /lib64/libdl.so.2
            libc.so.6 (GLIBC_2.3.2) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
            libpthread.so.0 (GLIBC_2.2.5) => /lib64/libpthread.so.0
    /lib64/libpthread.so.0:
            ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
            ld-linux-x86-64.so.2 (GLIBC_2.2.5) => /lib64/ld-linux-x86-64.so.2
            ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
            libc.so.6 (GLIBC_2.3.2) => /lib64/libc.so.6
            libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /share/apps/intel/mkl/lib/intel64/libmkl_gf_lp64.so:
            libdl.so.2 (GLIBC_2.2.5) => /lib64/libdl.so.2
    /share/apps/intel/mkl/lib/intel64/libmkl_gnu_thread.so:
            libdl.so.2 (GLIBC_2.2.5) => /lib64/libdl.so.2
    /lib64/libm.so.6:
            libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /share/apps/intel/Compiler/11.1/064/lib/intel64/libifport.so.5:
            libc.so.6 (GLIBC_2.3) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.3.2) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /share/apps/intel/Compiler/11.1/064/lib/intel64/libifcoremt.so.5:
            libc.so.6 (GLIBC_2.3) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /share/apps/intel/Compiler/11.1/064/lib/intel64/libimf.so:
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /share/apps/intel/Compiler/11.1/064/lib/intel64/libiomp5.so:
            libdl.so.2 (GLIBC_2.2.5) => /lib64/libdl.so.2
            libc.so.6 (GLIBC_2.3.2) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
            libpthread.so.0 (GLIBC_2.2.5) => /lib64/libpthread.so.0
    /share/apps/intel/Compiler/11.1/064/lib/intel64/libirc.so:
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /share/apps/gcc6.2/lib64/libstdc++.so.6:
            ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
            libm.so.6 (GLIBC_2.2.5) => /lib64/libm.so.6
            libgcc_s.so.1 (GCC_4.2.0) => /share/apps/gcc6.2/lib64/libgcc_s.so.1
            libgcc_s.so.1 (GCC_3.3) => /share/apps/gcc6.2/lib64/libgcc_s.so.1
            libgcc_s.so.1 (GCC_3.0) => /share/apps/gcc6.2/lib64/libgcc_s.so.1
            libc.so.6 (GLIBC_2.3) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.3.2) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /share/apps/gcc6.2/lib64/libgomp.so.1:
            librt.so.1 (GLIBC_2.2.5) => /lib64/librt.so.1
            libpthread.so.0 (GLIBC_2.3.4) => /lib64/libpthread.so.0
            libpthread.so.0 (GLIBC_2.2.5) => /lib64/libpthread.so.0
            libc.so.6 (GLIBC_2.3) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.6) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /share/apps/gcc6.2/lib64/libgcc_s.so.1:
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /lib64/libc.so.6:
            ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
            ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
    /lib64/libutil.so.1:
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /share/apps/intel/Compiler/11.1/064/lib/intel64/libintlc.so.5:
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
    /lib64/librt.so.1:
            libpthread.so.0 (GLIBC_2.2.5) => /lib64/libpthread.so.0
            libpthread.so.0 (GLIBC_PRIVATE) => /lib64/libpthread.so.0
            libc.so.6 (GLIBC_2.3.2) => /lib64/libc.so.6
            libc.so.6 (GLIBC_2.2.5) => /lib64/libc.so.6
            libc.so.6 (GLIBC_PRIVATE) => /lib64/libc.so.6

Kirk · November 19, 2016, 7:22pm

I just now noticed that for some reason it is picking up the wrong fortran compiler libraries.

Kirk · November 19, 2016, 8:38pm

Progress - I now get some output but now it crashes with:

Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so

LD_LIBRARY_PATH is correct, but this seems like a python configuration problem (at least that’s what Google seems to imply). Is there something I can add to the psi4 python script?

loriab · November 20, 2016, 8:20pm

It’s not clear why, but LD_PRELOADing the libmkl_rt.so library (and sometimes also libiomp5.so) fixes that avx & def library message.

Kirk · November 20, 2016, 9:36pm

Where do I stick this LD_PRELOAD statement?

thanks!

Kirk · November 20, 2016, 10:08pm

Got it. Didn’t realize offhand that this was just an environment variable. I set it to preload libmkl_rt.so and now all is good. Thanks for the help!