WSUS Looping Issue

Be careful when declining updates in WSUS.
I came across an issue where a recent "Cumulative Security Update for Internet Explorer 7" update was approved and being installed successfully on the clients. After a reboot, an earlier "Cumulative Security Update for Internet Explorer 7" would also get installed. After another reboot, the first one would be installed again, etc.

The cause of this was because I had declined some updates in between the ones that were looping. i.e. with updates A, B, C, D, I had previously approved update D, then afterwards declined B and C because I wanted to install the more recent A.
The solution is to either decline the oldest approved update (or any update that is looping and is older than the current one), or approve each update in between.
NB: even after the fix, the looping will continue until the client talks to the server again, i.e. at 3am or if you force it.