Confirmed with my friend regarding this. He had the same issue as me then he changed his router and the issue is gone for him. I personally have not bought a new router yet so I can't confirm

I'm an IT technical person but not into network. This is my suspicion (I don't have any other way to explain this):

To put it as bluntly short as humanly possible, the router provided by TM is so lousy (TM tech: specs are below minimum for fiber optics)

Long version: the router fails to cope with a more congested/stressed fiber optical signal which is clearly evident during peak hours that your connection gets 'flooded'/over buffered/bufferfloat to the point where your connection stability degrades. In other words, your router is bottle-neck(king) what is otherwise a really fast and stable connection. A better router that has a much better specification are able to handle this amount of stress and therefore would result in a more stable connection overall

Further reading: https://en.wikipedia.org/wiki/Bufferbloat