These can be very stubborn and I find the best method is to use a large pry bar in behind the drive shaft and give it a tweek. Sometimes a chisel in behind the shaft and a whack with a hammer will pop it out. Good luck.
