The only time you have to worry about your lower unit is if water got into the drive shaft. Every time I drain my lower unit oil, it come out very clean with not water at all. This means there's no way freezing temps could hurt it, its just oil inside like there should be (but I change it anyway to winterize it).
You engine block is a whole nother story! If it has a raw water pickup like most do, then it DEFINITELY has water in it, and must be drained and/or flushed with antifreeze.
Freeze plugs do not always do what the're supposed to do. Sometimes the block cracks regardless of the plugs.
Good luck, it may already be cracked!