IR impulses latency...

Some impulses have a little silence in the begining, which causes latency. You can cut it if there is such. Another thing to bear in mind is some plugins such as SiR have latency no matter what your interface and computer is.