You have a store of fatty acids which you can think of as going either way, ie be converted to glucose for fuel or laid down as fat for storage. If you don't eat any foods which are a source of glucose, your body will obtain the glucose from your existing fatty acids, to keep you going. Typically it does this when your BG levels fall to around 4.0 and it is part of the body's normal regulatory process, the limits it likes to operate in, body temperature, hydration levels and so on.
If you eat a normal meal, with carbs, you are topping your BG levels up yourself but, if you don't eat a carbohydrate rich food, your body takes it from your existing stores. People who diet can often get raised BG levels to begin with as their body is forced to look to its own stores for energy.