MY FH is in the military and while I know the "etiquette" of the poppy, I disregard part of it, as always pin mine with a Canadian flag.
I think if you're wearing it over your heart and you haven't "defaced" (and I do not consider use of another pin defacing... I mean, seriously?) it or destroyed it, it shouldn't matter.
But that's just me