Don't be terrified. I think if you are older than 35 you are considered a high risk pregnancy anyway. I think it's rubbish. I had my first with almost 39 after 3 rounds of IVF. He's 17 months now and we're planning going abroad to cycle again in a few months. I'm 40 now.
Would you have the possibility to get a libre? It makes controlling your levels SO much easier and was (and still is) a godsend for me. To lower your HbA1C you can only check, check and check. And believe me I think I wouldn't manage as easily as I do now without the libre.
Sorry, don't have much advice other than ignore what your hospital has said, try and get your Hba1C down safely and get trying for another as soon as you are ready. And big well done for breastfeeding that long (I only managed 3.5 months and was most upset). Good luck! xx