Something rotten in Denmark. I run 4 ohms daily on an 8 fin 400, push the speakers to bottoming out on the cone backswing, that's my signal to turn it down. Only grab a fan if the cranking is long term stupid. Pic below of what I am bottoming out. Zero audible distortion from barely heard to all the speakers can take. Since you change speaks and it goes away it is either the amp or the speakers imo. I get mine plenty hot, never a hint of distortion.
Could be a safe operating area (SOA) issue. Exact issue I encountered. On all my WOPLs I've disabled that protection so I can drive 2 ohm speakers. No problem since. Most don't recommend that balls to the wall approach. Fuse protection is still there. Ask Joe about it.
I ran 4 ohm for quite a while on my 400WOPL, only had distortion if I ran it hard enough to heat it up pretty hot. 8 fin no fan, took about 8-10 minutes of hard cranking and then it would start to lose clarity.