Prince William and Kate's move to their "forever" home has disrupted the lives of families who were living close to the couples' new house.