It all boils down to controlled experiments and measurements. You adjust one variable (the one specific food) while keeping the the others the same and note the effect on your BG level. For example at breakfast have milk with weetabix/porridge/ or whatever, and see how breakfast other than milk affects your BG. You find the lowest influence on your BG and then eat just that but then try varying the milk type - skimmed/semi-skimmed/full fat. This of course assumes that you are able to test your pre and post meal BG levels and can be bothered taking time and effort to get really scientific cos it takes time and effort.
However, if you want to save a lot of time and effort and reduce the impact of foods, regardless of "brands" on your BG the best and simplest strategy is to look on the BACKS of food packages (ignore the Traffic Lights) for low carbohydrate foods. All carby foods will push up your BG level - bread, rice, potatoes etc. Meats, fish and veg will have little effect.
Us ' human beans' are all different and how we react to different foods does vary considerably. I for example get a high BG from most breakfast cereals though many others don't. There's no 'one size fits all,' but as a general rule of thumb, cutting carbs is good for you.
Dave