When I find antlers with the skull the rest of the bones have likely been dragged away by coyotes. As I mentioned in the gun (Bowling for Columbine) thread, coyotes can do a lot of damage to a small/dying/dead animal. Occasionally other parts of the skeleton might be lying around, but not very often.
Why, Jaykeeley, did you want a penis bone or something?