Old battery's probably swelled up and that's why it's getting stuck. Don't be shy about removing the entire top cover and front of the frame, if needed.
The battery on my UPS died (APC BE550R), so right now it only functions as a 4 plug surge protector, since the battery plug are useless. I was looking at replacement batteries, but I might end up ...