I am trying to replace a block of computation intensive code in PSI4 with a machine learning model, in order to speed it up (just a proof-of-concept project). I’m looking at LibXCFunctional::compute_functional() now. I’d like to use Tensorflow to invoke my machine learning model here. I have compiled and obtained a shared library libtensorflow_cc.so containing tensorflow C++ APIs. I was trying to link this library to PSI4, but I was totally lost… I am not very familiar with cmake but I assume that I should add a target_link_libraries() some where. Does anyone know where exactly?
I’d suggest to have a look here: http://psicode.org/psi4manual/master/prog_newcode.html
especially at the pluging section. It also uses cmake, but it is much easier to understand than the giant the whole psi4 setup has become.
Doesn’t tensor flow have a python API? That could be easier.
Thank you for the suggestion and the helpful link. I took a look at the docs but I’m afraid that adding a plugin can’t achieve my goals. I am trying to replace a code block in the main psi4 code (e.g. the exchange correlation calculations) and I don’t see how I can do that with a plugin. However, the structure of plugin templates are helpful to look at anyways.
Now it seems that I have successfully compiled psi4 with tensorflow by making the following change to psi4/psi4/src/psi4/libfunctional/CMakeLists.txt
+ find_package(TensorflowCC REQUIRED)
+ target_link_libraries(functional PUBLIC TensorflowCC::Shared Libxc::xc)
- target_link_libraries(functional PUBLIC Libxc::xc)
It compiled without an error. However when I run ctest -L smoke all tests failed and gave this error message: Traceback (most recent call last): File "/psi4/objdir/stage/lib/psi4/__init__.py", line 55, in <module> from . import core ImportError: libtensorflow_cc.so: cannot open shared object file: No such file or directory
I suspect that there is an additional step needed to make the python driver aware of the tensorflow library.
Run ldd -v <objdir>/lib/psi4/core.cython<much_stuff>.so and make sure libtensorflow_cc.so is in the dependency list for “core.so”. As an immediate fix for your error above, find exactly where the TF lib is on your system, e.g., /path/to/lib/libtensorflow_cc.so and export LD_LIBRARY_PATH=/path/to/lib before invoking psi4. Probably ldd shows “cannot find library” before that export but finds the lib after the export. LD_LIBRARY_PATH is never the right long-term solution, but it’s excellent for the diagnostic stage. If you’re on a Mac, ldd–>otool -L and LD_–>DYLD_.
It could be nice to get TF plumbed into the build sys as an optional dependency. Could you make a pull request with your current changes? I can probably advise where to fill in from there.
CMake is powerful and useful. But because building software in general is full of details, CMake is full of details and isn’t very user friendly. What you should know about psi4’s cmake is that we use a “superbuild” system, where there’s two passes of cmake run. The first looks for dependencies and either finds them or builds them. The second pass always builds psi4 itself. So for every (compiled & linked) dependency, there’ll be two find_package(addon) calls: one in external/upstream to find-or-build and one in psi4/CM to detect-and-use. This lets us either build everything from scratch self-contained or pre-build most stuff and do minimal compiles for fast dev.
Regarding a pull request, I’m afraid that my changes are not a good reference. I used a docker container that have tensorflow pre-installed and usable by cmake. My changes are also very hacky and probably not a good way to do it.