As a point of interest here, coffee has no carbs in itself but can raise sugar levels. Read a good medical study on this from, I think, W.H.O
Strong Coffee increrases resistance to insulin. This means that whereas our BGs would have been coming down at our usual rate after eating, the coffee drunk between eating and +2 hours slows down that rate of BG decline so our +2 hour reading is higher than it would have been. My take is therefore that the coffee hasn't INCREASED our BG, but it's stopped it going down so much.
Interestingly, although green tea has caffeine in it, it REDUCES our resistance to insulin and thus has the reverse effect.
These effects aren't great, and i think the green tea effect is less than the coffee one (unfortunately!)
By the way, agree the milk shouldn't have a great effect, but if you're low carbing, that extra 2 (say) grams of carbs could be a relevant %age of the carbs you had in the previous meal.......so could have some effect I guess.