It's a bit more complex.

Speed/quickness/rapidity has effects on many levels. A small advantage may have great effect on the ability of the enemy to shoot back (the famous first shot kill of MBT combat), while the same small advantage would be useless on another level (few seconds are meaningless in brigade staffs).

Overall, I'd say that the speed difference counts, and it has diminishing returns (adding xy speed advantage provides a lesser benefit than the previous xy speed advantage).