On a "good" day, it takes about an hour to get from Penrith to Sydney. On average you'd be looking at 90 minutes minimum.... and then you've gotta add in anywhere up to another 90 minutes coming from the mountains.
I live in Parramatta which is pretty much smack-bang in the middle of Sydney. One day I got into the CBD in 21 minutes by car (every green light) but I had to drive in during peak hour once and it took 2 and a half fucking hours!