Hi all,

I'm experiencing some anomalies while streaming videos off YouTube on Unifi. For certain 'political' videos -- I've observed that the HTTP connection for the videoplayback stream to YT's local CDNs are being disrupted as follows :

1. Client video player makes a connection to the YT CDN
2. HTTP GET request is sent

There's a few different behaviors after this .. :

3a. HTTP 200 OK is received however it arrives 90 seconds later (should be instant) :


3b. HTTP 200 OK is received instantly, first 1-4KB of video stream traffic is sent (allowing the YT player to show the first frame of the video with a timestamp of 0:00).. then no traffic is received for 90 seconds once again :


There's a duplicate TCP ACK when the stream returns, did my ACK at packet #271 ever reach the CDN in the first place??

Further testing :

1. Using an unencrypted SOCKS proxy on a remote server + non standard TCP port results in the same behavior with packet loss between the client and SOCKS proxy server

2. Using an encrypted SSH tunnel to the same remote server results in absolutely no issues with viewing the videos

Sample videos :
http://www.youtube.com/watch?v=hHTz22bTBRw
http://www.youtube.com/watch?v=uVWxB4AWOxc

UPDATE :

I performed a simultaneous packet capture on both my client + remote server while encapsulating the HTTP connection via plaintext SOCKS. All the video payload packets were dropped en route back to my SOCKS client :



Dafuq?

UPDATE 2 :

Confirming all plaintext HTTP connections on Unifi (and maybe Celcom + Maxis) are being man-in-the-middle'd and dropped if they contain blacklisted data.

UPDATE 3 :

Other sources confirming this .. (thanks wkkay):

https://plus.google.com/1013966581485225280...sts/ak6opfbDxwa

UPDATE 4 :
What we know  :

i. The DPI isn't only being used to selectively block YouTube videos, however unencrypted Facebook pages belonging to certain parties are also being blocked. You can get around this by appending 'https://' to the Facebook URLs rather than trying to use 'http://'.

ii. The DPI is based on TCP segment analysis. Basically, every single TCP packet has its payload analyzed for certain request URI strings that have been blacklisted. Obfuscation attacks such as packet fragmentation (splitting a large TCP payload containing a single HTTP request into smaller TCP segments) as well as packet padding (appending large amount of junk data to the HTTP request URI in order to force the 'HTTP/1.1\r\n' trailer into a separate TCP segment) will also work however you need specialized HTTP proxy software or iptables rules (on Linux) to do this.

iii. Once a blacklisted payload is detected within a packet, the header information for the TCP stream (SRC/DST port + SRC/DST IP address) is added to some kind of blacklist for 90 seconds. This causes all traffic for that particular TCP stream to be dropped for 90 seconds (hence the 90 second gaps in my packet capture samples above). This is also why some of you have noticed that if you wait long enough (well, 90 seconds in my tests).. the videos/sites that are blocked will eventually continue to load. Due to the persistent nature of TCP, once the 90 second blacklist window passes.. your TCP stream will continue and the payload data for whatever you're requesting will reach your computer.
Mitigation techniques :

i. Use 'https://' wherever possible (especially on Facebook). Users in the thread have recommended HTTPS Everywhere which is a Firefox/Chrome addon to do this automatically for most major websites.

* While YouTube supports HTTPS for their main website, their player does not support it so even if you were to use HTTPS on YT.. the videos won't load.

ii. For accessing blocked YouTube videos, you can use some of the various YouTube proxy sites such as ProxFree.

iii. Get a VPN/SSH tunnel service if you're worried about having your HTTP requests intercepted.
UPDATE 5 :
Response from MCMC
Hey, here's a simple test you can do with less than 2 commands on a Linux box + Wireshark :




So a HTTP GET request for /DAPMalaysia results in the query taking 109 seconds to respond along with 8 TCP retransmissions (I'm basically getting 0 TCP responses from the server for 109 seconds). Let's see what happens when we request for the exact same URL however we append 1500 bytes of junk URI padding to the end :


.. which results in ..



Oh? What do you know, no issues at all. Apparently appending an extra 1500 bytes of junk data to every HTTP request in a 'congested' network results in less network congestion. Who would have guessed /s

---

My final comments on this issue ..

I'm pretty apolitical when it comes to the Internet and networking. The only reason I have to keep testing what some may call PR-friendly URLs is because it seems that the only time we have 'congestion' is when accessing such content.. and the 'congestion' goes away the moment you obfuscate the requests enough. With the resources that the MCMC has available to debug these kind of issues, I'm honestly surprised they haven't figured this out already.

The tests we've done here show at the very least there is some kind of HTTP request inspection happening and traffic is being dropped once certain strings have been identified. As Internet users and/or caretakers, we should be against any form of Internet censorship. I leave you with these two articles hosted on the MCMC/SKMM website :

http://www.skmm.gov.my/Media/Press-Clippin...sur-fitnah.aspx
http://www.skmm.gov.my/Media/Press-Clippin...edia-In-Ge.aspx