Always always always double-check your active space and your geometry. Always. I ran this computation myself, and it looks like the ordering of orbital energies in vanilla SCF is:
Doubly Occupied:
1Ag -2.661020 1B1u -2.548529 2Ag -0.209733
Virtual:
1B3u 0.005156 1B2u 0.005156 2B1u 0.011595
3Ag 0.049491 1B2g 0.059896 1B3g 0.059896
3B1u 0.063416 2B3u 0.078978 2B2u 0.078978
4Ag 0.123702 5Ag 0.160921 4B1u 0.180307
2B2g 0.182642 2B3g 0.182642 1B1g 0.326548
6Ag 0.326548 5B1u 0.348420 3B3u 0.402386
3B2u 0.402386 7Ag 0.434997 3B2g 0.441744
3B3g 0.441744 1Au 0.490063 6B1u 0.490063
7B1u 0.547246 4B3u 0.626574 4B2u 0.626574
8Ag 0.658857 4B2g 1.041967 4B3g 1.041967
8B1u 1.157239 9Ag 1.775201 9B1u 2.239306
10B1u 6.233364 5B3u 6.241537 5B2u 6.241537
5B2g 6.422021 5B3g 6.422021 10Ag 6.605620
11Ag 15.192822 11B1u 15.320472
Psi itself reports your active space of choice as:
------------------------------------------------------------------------------
Space Total Ag B1g B2g B3g Au B1u B2u B3u
------------------------------------------------------------------------------
Frozen DOCC 0 0 0 0 0 0 0 0 0
Restricted DOCC 0 0 0 0 0 0 0 0 0
Active 16 5 0 0 0 0 5 3 3
Restricted UOCC 28 6 1 5 5 1 6 2 2
Frozen UOCC 0 0 0 0 0 0 0 0 0
------------------------------------------------------------------------------
With no core orbitals, the key information is in the active section. Cross-referencing that against the Hartree-Fock orbitals, you’re skipping over two B2g/B3g pairs, a B1g/Ag pair, and another Ag to get a high-lying B1u, B3u, and B2u. I would need to work through the correlation tables to be sure, but I have a very strong suspicion that this does not make good chemical sense.
When I tried a CAS(6, 15) with the active space [5, 0, 1, 1, 0, 4, 2, 2], it still didn’t converge, but I got a lower energy (-14.78393), which tells me I’m on the right track. I tried removing the highest-energy orbital, the 4th B1u, and the computation still failed to converge, but with a much higher energy.
However, this is a strange geometry. A 1.25 separation for the lithium dimer is tiny. NIST tells me 2.7 angstroms is more realistic for the ground state. Did you mean to have the atoms that close together? This is causing problems even in a conventional SCF - there’s a UHF solution about 0.2 hartrees lower. It’s horrifically spin contaminated, of course, but that shouldn’t be there. My hunch is that it’s making your CASSCF harder than it should be.