From the Tesco website Whole Milk has 4.7% carbs, Skimmed Milk has 5.0% carbs so for a 40ml addition to tea or coffee that makes a difference of 0.12g of carbs.
And how many cups a day? If you're on a very LC diet, then it can make a difference to the more nutritious food choices you can eat, so trimming back where you can is helpful. 'Look after the pennies and the pounds will look after themselves"