The XMT#2 with what pad? What buffer were you using? Did you follow any instructions?
I have never been a fan of the yellow cutting pad. To me you don't get any benefit over the orange cutting Pads which will not marr like the yellow ones will.
You should be able to clean up any swirls or marring that you might have put in with the #4/yellow pad combo using the XMT #2 and a white polishing pad.
My guess is you probably didn't buff long enough with the #4 to break it down.
Welcome to DC by the way.