If you guys are hearing THAT much difference (like in the video), it means that there is something wrong somewhere.
The first thing to check is the impulse response used, it the file is not getting resampled according to the new sampling rate, the sound will change drastically.
Same goes for the VSTs, if they don't handle sampling rate change "on the fly" correctly, all the filters are going to have a frequency shift, causing a completely different frequency response.
In my experience, between 44.1KHz and 96KHz, there is a sound difference, but is not night and day and of course, the frequency response must remain the same.
If you change the sample rate on the fly and hear a huge difference, try saving the project and reload it with the new sampling rate, sometimes it can be an initialization issue.