My late afternoon numbers have always been roughly in line with my per-lunch levels taking into consideration what I ate for lunch and breakfast.
Whether yours are cause for concern depends upon why they are now higher. If it's only those that are higher, not any of your your pre-breakfast, post breakfast, pre-lunch, post-lunch, pre-dinner, post-dinner readings, then you can rule out most possible reasons for concern such as decreasing Insulin production from your Pancreas.
Have you changed your lunch/
Have you changed your medication or the timing of taking your medication?