If you are losing weight that you don't wish to lose, and you want to avoid starch/carbohydrate then you must increase fat and/or protein.
It is possible that at your weight, and the amount you have lost that you are losing muscle mass - though your swimming and walking may help preserve it...