I dislike extreme heat more.
If it's cold, you can bundle up in loads of layers and get warm.
If it's insanely hot you're just kinda stuck with it.
I know the above statement is repeated quite often, but it is not scientifically true.
There are extreme cold weather issues which cannot be resolved with simple "layering". It can get so cold that unless you find shelter, you will freeze to death or your toes, nose, ears and any body parts exposed to the cold can get frost bite. Many a people who were "bundled" with layers still died and still got frost bite.
As far as heat goes. Even if it is "insanely" hot, you are not stuck with it. One would simply need shade and a lot of water but can survive 115F+. While not something enjoyable, as long as water is available and you are in the shade, you can survive "insane" temps.
Both hot & cold extremes are not fun to deal with. The old "layering" while cold is true but reaches a point where it will no longer work in extreme cold. The human body will require shelter or one would face death.
In my life I have been exposed to both, 120F+ and -35F below zero. I will take the 120F any day over the -35F below zero. The -35F below zero is something that imprinted on my mind and body for the rest of my life. Unless I think about, I forgot about the 120F, it was uncomfortable but manageable. The -35F below zero hits you hard and you never forget what it feels like when it was that cold.