No harm, unless you are talking more than couple of years. Just store your watch in a cool dry place away from direct sun light, hot and damp place.

U could just shake the watch once a week if u are not wearing the whole week, to lubricate the movement itself.