generally. a university consists of multiple colleges, but many of the big universities started as a college and eventually added more colleges but retained the name. so "Boston College" is actually a university (within it are the College of Arts and Sciences, Carroll School of Management, and others) and it's also not in Boston (it moved to Chestnut Hill as it expanded).
so when you're "going to college" you are probably going to a college within a university. "going to the university" makes me think of a state university more than a private one, but that's not necessarily true, I guess.