Those are not dirt.. It is dried semen that wasn't washed off when you ejaculate or had wet dreams.

You need to pull down the skin and wash it off every time you bath to keep it clean.