Usually I would say, get the mirrorless first (perhaps borrow one), try it out and then make a verdict. People who only own mirrorless, almost all will say it's worth it but usually just shoot with kit lens and never touch dslr in their life.

However I can say this, dslr is heavy and bulky. The only time you will take it out is if you plan ahead where and what you want to take. Mirrorless on the other hand with pancake lens suitable for street photography, you can dump it in your small bag and the moment a scene pops-out, just take it out and shoot. Like what [PF] T.J. said, IQ may not be up to par if you are a pro DSLR shooter, but with smaller camera, you will find yourself taking photos more frequently. But IQ is rather subjective until you try it out rather than depending on word of mouth.

Actually another advantage of mirrorless is the evf though depends on preference. Evf can see real time changes as you adjust the dial. With ovf, you shoot just to realise you forgot to dial up iso. Also mirrorless is still slimer with body only even if they look like dslr. So if switch to pancake lens can take to anywhere and still versatile. Only attach big lens for certain occasion.