Well the short answer is yes it will affect your reading- but more importantly as
@KK123 asks- what information are you hoping to get out of your reading? A lot of my readings are just to keep an eye on my general levels so can be quite random. If I am testing a particular food/meal I am more careful about the time. I think you want to work out why you are testing- having a milky coffee after a meal you know won't raise your levels tells you about its affect. The problem is that so many things can affect your levels.
I am still a fan of testing but work out why and that will have an influence on when and how often you test.