I know when I get sick or don't feel well, I lose my appetite, which is fine with me. Depending on your activity level, you don't always need as much food. And I can always stand to lose a few pounds, so it can be a plus for me.
However, things shouldn't be ignored necessarily either. In my case when I fractured my back and just wasn't eating, I dropped some weight fast and the doctors freaked out and wanted to start all sorts of tests. I declined because I knew it was because I wasn't eating because of my back and very slowly, months later my appetite slowly came back and I was fine. But I knew the distinct reason.
If there isn't a reason and you are worried, sometimes it's best to pursue it. The squeaky wheel type thing? So if you keep going to the doctor and complaining and he doesn't do anything, you might pursue seeing a different doctor for a second opinion, different doctors have different experience from different patients and might be able to suggest an answer.
It doesn't matter if you feel you are wasting their time, that is what they are there for.