Good point.
I think that when I declined her offer I would say something like "I really would love to, but I just feel that it might be inappropriate. I'm not sure how it would be viewed by others here at the office, or my girlfriend for that matter."
Now, in all likelihood no one at the office would ever know; but having suggested it raises the possibility, and thus makes her response more meaningful. For instance, if she says: "No one here needs to know," then I know what she has in mind. If she says, "Who cares if they know, we're just getting a drink," it's less likely (although still possible, especially if she's a tactful person). No matter what is said, I doubt that this deline could be interpreted as offensive. If she's intelligent, she'll understand my hesitance. If she's offended, then it's likely I wouldn't want to be involved with her anyway.