Babies don't 'start from scratch'