In my opinion this is one of those cases where you just have to listen to your Doctor, he/she has done all the tests and arrived at a considered diagnosis. I am inclined to agree with what you have been told, a slightly higher than normal fasting BG is not, on it's own, any indication of diabetes.
The body has some wonderful methods of pushing your BGs up in the morning, whether or not you are diabetic.
The HbA1c test is the standard used by most Doctors now, although it does not show any highs or lows it does give a good indication of your average BGs over a given period.
So, on balance, you are probably not diabetic.
Now I will qualify my next few sentences, I dislike the term "pre-diabetic" or similar, in my book either you are or you are not, but from what you have said I concede that you may just be pre-diabetic, or to put it another way, have a pre-disposition for higher than normal BGs.
So watch your diet, keep the weight off and excercise. That may be all you need to do to mantain a perfectly normal life.
Ask your Doctor for a periodic HbA1c just to be on the safe side.
You could also ask about a Gucose Tolerance Test, cheap and easy to do.
H