If the X5 M rides anything like the X3/4M I wouldn't want it, incredibly harsh ride in the X3 M, even caused my dog to throw up in the back when we had an overnight test drive
the car simply sees a bump and transfers it into the cabin and its occupants, even if its an M car its inexcusable in an SUV in my opinion, not to mention there are other SUV's that handle well or even better than the X3/4/5/6M and still are comfortable (Porsche Macan/Cayenne, new GLE63).
I imagine having an X7 which rides buttery smooth on air suspension will make you notice every little bump in the X5 M, I'd recommend a Cayenne, X5 M50 or GLE63 for a better ride but still fun and fast SUV.