Difficult to answer without knowing the full background, but two possibilities spring to mind.
First, if you're recently dx'd, you're maybe still honeymooning, so there's a chance you're correctly calculating for the bolus, but then your beta cells think, oh, some food, and squirt out some insulin on top, which takes you down quicker.
Second, I tend to pre-bolus about twenty minutes before a meal to give the insulin time to get into the system, but I've noticed that if its a low GI meal, I'll drop in the first couple of hours more than I'd like, reckon it's because the insulin is kicking in before the high fibre food has broken down to glucose, so now I tend to shorten the timing of the pre-bolus depending on the GI of the food. If you're going low but it eventually balances out, it kind of suggests the food is playing catch up with the insulin (even though it's normally the other way around!). So, maybe think about how far in advance you're pre-bolusing.
It might also be, as you say, that you need less units. If you take fewer, don't drop in the two hours, and are fine after five hours, then that would be the case. On the other hand, if you end up too high at four or five hours after, it's maybe more the digestion/pre-bolus timing which is governing it.
Of course, it can be subtler things, which you wouldn't necessarily think about until they're pointed out. The nutritionist on my DAFNE course mentioned that the amount you chew affects things as well Sounds obvious once it's pointed out but I hadn't thought about it before. If you chew well, food starts digesting in the mouth because of the amylase in saliva (chew a starchy food long enough and it'll start tasting sweet because the amylase breaks the starch down to sugar), so once it gets to the stomach it's already part of the way into turning into glucose, so will get into your blood quicker, whereas if you wolf it down without chewing that much, the stomach is starting from scratch and will take longer to break down.
Don't know if it's been mentioned, but seeing as you're trying the Libre, there's a good book (kindle and paperback), Sugar Surfing by Stephen Ponder, dealing with cgm. There's a lot of dingbat books out there about diabetes, but this one is mentioned approvingly by the Edinburgh Centre for Endocrinology and Diabetes, in the leaflet linked below, and they know their stuff. Ooh, I've just noticed on the leaflet that Gary Scheiner, the guy who wrote the much recommended Think like a a pancreas book, has also written one on cgm, will have to have a look at that.
http://www.edinburghdiabetes.com/s/CGM-guide.pdf