Agree, the problem is the vertical "wall" at the back, and i see 2 ways to "fix" it, either cut the vertical wall (the cut will look a bit ugly but is not going to be visible externally), or move the shelf up
If i where you i would start by moving the shelf up, you just need to drill some holes at the sides
After that... calculate distances again and see if is needed to cut the verticall wall also