If you think it's weird, then, I can't have any dairy because it plays havoc with my system. If I have butter cream, or milk, I vomit, and tastes vile to my palette, cakes or pasties or anything with milk in, including cheese, sends my blood glucose levels up.
But I'm not lactose intolerant!
Here's the weird thing, yoghurt, doesn't do anything to my bloods, and I can eat it any time! Mmmmm
Could it be to do with the cheese being made from the fat in the milk? If you think about it, the carbs in milk get less as the fat content increases ie skimmed milk is higher in carbs than cream.