Many Type 2s have insulin resistance; this means that the muscle cells are resistant to the action of insulin, which is trying to store excess blood glucose in them as glycogen, for energy. This means that the blood glucose remains high, so the pancreas pumps out even more insulin, in an effort to get rid of the glucose - which is then stored in the fat cells as fat!
As you eat less carbohydrate, so there is less blood glucose and the pancreas doesn't have to work so hard. The insulin resistance slowly decreases, and the glucose is used more efficiently. It's a combination of 2 things - less carbs = less blood glucose = lower production of insulin; the insulin resistance decreases and the "rested" pancreas begins to work more efficiently.
This takes time; it took me a full year to get down from 6s and even 7s in the beginning, to where I am now, which is hardly ever out of non-diabetic levels.
If you've been running high blood glucose levels for a long time without knowing it, the pancreas may well have been damaged and it will take longr to get to lower levels. Some people never do, without drugs.
The above is by no means a scientific answer, and I may well be shot down in flames for it; but it's basically accurate, if simplistic, and it works for me!