Soundcard digital sync jumps between external/internal

[Spectivum]

I have outboard gear sending SPDIF coaxial signal to the soundcard. It shows external clock sync, but every now and then it jumps back to internal clock for a split second, causing a switching noise.
This happens with all outboard gear and all cables I have.
Has anyone seen this issue before?