yup, not damage but battery drain. once the battery is used (from new) even once, it will start draining whether u start your car or not. after a month or so without starting the car to charge it, it will be low on battery and u might not be able to use it to start your car, so if possible, start your car at least once a week to let it warm up to charge the battery, it's better to let it charge for 15 minutes or more because starting the car will use a fair amount of power.

if u use maintenance free battery u will need to charge for even longer, because it won't hold as much power as wet battery and it charges more slowly than wet battery.