It's been done
Have seen several successful installations putting the Garmin 430 "teardrop" antenna under the cowling. When I asked some of the guys at SteinAir about the heat affecting the antenna, they said it would not be an issue.
Some I've seen have been mounted to a bracket attached to the firewall, and some have been attached via U-bolts or Adel clamps to the engine mount. Just make sure you have it high enough so that the firewall can't obstruct the signal.
Not sure about the XM antenna, but someone else may chime in.