This wasn't just a random change — it was influenced by the French word "pomme d'orenge," or "apple of the orange tree," which sounded similar to the evolving pronunciation of the fruit's name.
and later through the French word for the citrus fruit, pomme d’orenge. It wasn't used as a descriptor for color in English until the 1500s, many centuries after the cultivation of oranges began ...