Beating the drum is hard on the wheel bearing, and potentially hard on the axle shaft and C-clips.
That said...it's done all the time. Ideally, you'd hit it from the back-side of the drum more than the front or the shoe-contact side, to minimize problems with the bearings and maximize the force separating the drum from the axle shaft, which it's probably rusted-to.
I used to see a lot of problems with drums carved and gouged leaving a huge ridge at the open-end. The ridge would catch on the side of the brake shoes, preventing removal. I don't see a lot of that any more, but it can still happen. Then there's little choice but to retract the adjuster so the shoes clear the ridge, or--in extreme cases--grinding the heads off of the brake nails so the shoes can pop off of the backing plates when the drum comes off.