Clipping is very similar to playing back a signal that has gone through hard knee compression with a fast attack and high ratio, if this was the case, then you would damage your speakers playing some 'modern' music.
It does increase the amount of average power in the signal.
Speakers can be damaged either by thermal issues or mechanical issues, IE: running the coil into the backplate, tearing the spider of it's landing, etc.
To re-iterate, clipping itself does not damage speakers. Klipsch uses a pair of zener diodes to protect the K77 horn tweeter in their AA networks for the Klipschorn and the Lascala.
At 2.5 watts at the output of the crossover to the tweeter (4.5Vrms), they produce 6.7% distortion, but below 2 watts, they are around 1.7%, at 1.5 watts they contribute a third of a percent distortion. They were added to keep people from damaging tweeters from over powering.
The K-77 with the newer copper beryllium lead out wires can handle about 5 watts RMS long term before failure. The same tweeter can handle over 100 watt input at 20khz with a very short burst of one second on, one second off.
If you looked at the tweeter output of the zener diode equipped crossover on a scope, you would see it flattening off the tops of the waveforms, when the voltage got high enough for them to start conducting. Below this, they are essentially 'out of the circuit'.