Yeah.. Avant-garde metal has been turned into a fairly loose term.. which can allow for quite a lot of bands to be classed as it.
If you were to literally use it in its strict sense.. Avant-garde is a potentially fleeting thing. Once something is done more than once is can hardly be regarded as truly avant-garde. It is art that pushses things forward, so doing something that has been done before is not exactly furthering artistic boundaries. That's if you're a pedantic sod about it.
But yes, it is generally used in this context as a term that is interchangable with 'experimental' and has parrallels with 'progressive' music. I remember reading an article about the difference between avant-garde and progressive music - both having quite a lot of similar characteristics, but progressive music generally focussing more on unconventional structures, rhythms, progressions and time signatures.. Avant-garde focussing more on unconventional textures, timbre, tonality and instrumentation. This isn't the be all, end all opinion though.
And an article i read on wikipedia ages ago used this simple comparison to try and show a 'progressive' musician and an 'avant-garde' musician - jon petrucci is the progressive musician, mike patton is the avant-garde musician.. Which kind of supports the idea above.
But yes.. All sorts of bands could be regarded as 'avant-garde metal' in the loose usage - opeth included. So yes, Deathspell Omega are 'avant-garde metal'.. And actually one of the few bands that is fairly deserving of the title seeing as their last release really has pushed the boundaries of their music. Maudlin Of The Well/Kayo Dot are another great example .. Although blue lambency downward is a pretty tame album.
I personally would avoid getting yourself worried about it.. Rather just keep looking for avant-garde/experimental music.