I would test the sweet potato out carefully.
It actually has more carbs than normal potato. However some people can eat it without a significant blood glucose rise, while others (me) get silly high readings.
If you are eating low carb bread, then you might be better having lower carb options to replace your potato. Swede, turnip, celeriac, cauliflower in the form of cauli rice, cauli cheese (LC version), broccoli riced or cheesed like the cauli, oomi noodles, konjac noodles, Lo-Dough... there are quite a few options.