well previously i had the same issue and it was TM fault, their conenction from endpoint was dirty and this causes the ping or latency issue .. fibre optic cable is very senstive to any dirt or dust particle etc.

after that no issue, but now i no longer use d-link dir 615, i am using tp-link for unifi