It sounds like you're disconnecting during the initial data load when the caches are being refreshed, which happens the first time you log in.

If your broadband latency is fine, you probably need to ensure you have more swap space on your hard disk - i.e. check that the drive WoW is on isn't at a critically low space available.

Removing the contents of the Cache, WTF and WDB only makes it worse since the game has to repopulate the item and creature cache again, so I wouldn't do it very often.

Just try running with and without addons and see whether it makes a difference.