Quartz updates the latency earlier than the WoW in-game latency. Just mouseover the in-game WoW latency every 2 sec and you'll see that it doesn't always update on time.

Plus, if your addons are communicating with others in a raid, there's a lot of information being sync between parties. Examples of usage is Omen, KTM, any Damage Meter, SCT, any Castbar addon. With so much data travelling to and fro, you may get lag in graphics updating (dunno how else to describe it). If you replace your addons with less syncing or turn off the syncing option, you'll see that your Quartz red bar becoming smaller.