We have switched entirely to using unsweetened almond milk. The one we get is Almond Breeze unsweetened which is 0.2 carbs per 100ml - dramatically less than cows milk. We use almond milk for lattes, and for hot chocolate using dark chocolate, and my husband uses it in his tea - I can't taste the almonds in it in drinks, and even drunk on its own I only get a very faint flavour. Cows milk is 4.7g/100ml for whole, 4.8g/100ml for semi-skimmed, and 5.0g/100ml for skimmed. I add cream to coffee otherwise, with double cream around 1.6g/per 100ml, and single cream 2.2g/100ml. Reference:
https://milk.co.uk/nutritional-composition-of-dairy/
You could also try testing your blood glucose before a cup of tea and then 2 hours later to see what effect it has - I found that milk really makes my BG go up a lot, particularly skimmed milk. It is possible that the tea with milk does not have a huge effect on you and changing other things may have a bigger effect on making your BG lower.