The medical advice is that you as a Diabetic should remain below 6.5%. Your GP is incorrect in describing your level as non-Diabetic ? You are just a well controlled Diabetic, that's all.
A normal non-diabetic HbA1C is 3.5-5.5%. In diabetes about 6.5% is good.
As you are going lower that all seems good. As to what level you get down to, that has to be a personal thing.....one you are comfortable with and can achieve without too much hardship.
If you want to go for the 5% mark.....then go for it !
Ken