I remember reading an article before which gives an example that bricks are your protein (the building block) and the calories are your cement. No matter how much protein you eat all will goes down the drain if you do not have enough calories to support it. So yeah don't worry about your 6pack if you have not achieved your desired muscle strength.