Cross post from Unifi thread

The slow link from South East Asia to US
I have tested this problem affect Malaysia and Singapore as a whole. If you have CDN in Malaysia / Singapore, but Origin server in the US, transfer will be very slow and almost certainly timeout.

This investigation started because I have a lot of problem working on Fedora, a flavor of Linux distro.

A little background:
There has been a lot of fights between Tier 1 and Tier 2 providers in EU and Asia lately, all of them involving Cogent. In the beginning of the year, Cogent de-peer NTT in EU, causing all EU traffic to peer in the US, increasing latency even if you are in EU. Example, two different AS in Amsterdam must peer in US, even when they are physically in the same city.
Last month, Cogent de-peer TATA in Asia, cutting off many single-homed customers from each other.
Submarine cable breakage towards EU and Japan also adds to the problem.

Example:
The website is fronted by Cloudflare or AWS CloudFront, and pings are fine. But when you actually starts transferring bulk data, speed quickly drop to less than 10KB/s and depending on how much data is being transferred, you will get a timeout. This happens regardless of the location of CDN either in KL, Johor or Singapore.

In Fedora's case, they are fronted by AWS CloudFront but their Origin is physically in the US.

Observation:
Anecdotes suggest the link will mostly be fine between 4am to 11am Malaysia Time. As you can see, I don't have time to wait for the stars to align.

My solution:
Since I have built my own anti-censorship "VPN", I use it to solve my problem. In AWS, all inter-AZ or inter-region traffic will be routed via what is known as "cold potato routing".
So I spin up one EC2 instance in Oregon and CloudFront it. I then connect to CloudFront KL, which will cold potato route me to my EC2 in Oregon.

The end result:
Hot potato routing Oregon -> TM


Hot potato routing TM -> Oregon


Cold potato routing EC2 Oregon -> CloudFront KL


The ping! (Cold potato routing)


The ping! (Hot potato routing)


Quantifiable improvement:
Packet loss reduce from 30% to 0%. Infinite improvement!
Latency reduce by 82ms, a 33% improvement.
Jitter reduce by 99.8%. Gamer rejoice!

This post has been edited by kwss: Jun 17 2024, 09:48 PM

Short answer: As long as you plug in your network cable and Ookla speedtest (with telco server) give you the speed, you cannot touch them.

That's the agreement they have with MCMC. That's also what is specified in their T&C with you.
No telco will admit they have a congestion issue, forever and ever.
If the technician visit your place, understand your story, he still must buat bodoh and not tell you it is due to congestion.

What you do next is up to you:
Downgrade your package if your usage is only during evening time, where congestion happens. Or cancel your fiber altogether if mobile data serve your needs.

Changing telco is another option, provided you know they are not as shitty.

Use VPN which does not traverse the congested path. This option involves a lot of trial and error. Basically start with any VPN with peering at MyIX and test them first. Try those that offer money back guarantee first in case they don't work.

You can build your own VPN on path that is known to never congest, like PNI towards Google Cloud, Microsoft Azure or Amazon Web Services. You will be billed by data transfer if you use this method so use this selectively for important task. Don't route all your traffic.

Shop for VPS that has direct peering in any of the local data center and try them instead of the big cloud. The downside is that you need one month commitment. Where big cloud like Amazon has spot instances and per second billing granularity.

Layan YouTube or any Meta services which won't congest because they are served directly inside TM. Same goes for all OTT offering from TM.