Two items here,
Garry is 200% correct on getting a square cut on the ends of the speg seal. Must not be twisted either.
The second item is to have a bit of preload on the end of the spegg seal so it has a little pressure keeping the end against the stop.
I use silicone to seal everything except tapping in new seals. Be sure to tap then test the shift shaft to get the proper friction & pressure preload against the lubed new fat O-ring & shaft. I put the shift shaft in then add the assembly to the case. This way I have less chance of taking a chunk out of the shift O-ring.
Want a hard case to seal try a 53 Firestone 10hp! it’s a bear to get right & not leak. Tiny o-ring for shift shaft seal