I would sample replace/augment personally... If you're trying to go for a completely natural sound, sample replacing the toms, if done right, will not interfere with that goal.
If you want to be able to say "all the drums are 100% natural" then you'll need to do like Revson and drumattacks said methinks. It may be worth a shot, but depending on how much bleed there is, you may still run into problems.