The benefits of exercise outweigh any risks you have of causing yourself long term damage by running high temporarily - whether these are social benefits, feeling good (endorphins), weight loss, muscle retention for greater strength and mobility as you age (and therefore better quality of life), etc., etc.
Running low can be dangerous (in extremis), however, when exercising as soon as you stop your glucose needs also tend to stop so you don't drop lower and can then fix the problem. Typically the advice is to not exercise with bolus on board, though if you can stop and have snacks to hand you can ignore this (as otherwise it can make finding a time difficult - potentially just reduce bolus for the previous meal) and just make sure you stop and treat if needed. It's all a learning experience, which is interesting and hopefully fun.
Going to the gym (for weights) often/usually increases blood glucose level initially. Give it a go, take a correction if you feel you must, but beware that you may then go low (so don't overcorrect - don't try to keep control too tight.)
Practice, though things change (though you will eventually also see these trends too and know roughly what to expect). When you first start or restart exercise, you may get a rise due to hepatic glucose generation (from stress) hormones, this goes away with time as you get used to what you're doing. Also, as you become used to a given type of exercise it's likely that whatever reaction you were getting (whether running high or low) will reduce as your body adapts to the exercise.
You mention long walks, in many ways these are more difficult to deal with than the gym in that then tend to make people run low, and you're potentially quite a long way away from home/a seat/readily available food. Going to the gym has the advantage that you can sit down, there are people around and you can bring quite a lot of emergency food/drink with you in a bag to deal with any and all eventualities.
I still find long rides somewhat unnerving from the point of view of BG management when I've not done one for a while, I always overpack food, but I'd far prefer to have it (and variety just in case I don't fancy a certain thing) than not. Once I've done one it all falls back into place and I forget (most) of my concerns (I still pack too much food, but just less than the first long ride of the season!)
You mention the thought of altering doses and/or eating extra scaring you - do you not alter doses normally based on what and when you eat? If not, I can understand that this might seem a bit scary, but it's not; even if you get it a bit wrong it's not going to cause you any problems running a bit high, nor running a bit low (have a snack, etc.) Getting used to this also adds flexibility to life in general which is nice (reduces concerns about needing a certain quantity of food at a given time, etc., which was always the issue with fixed doses before the advent of MDI.)
What insulins are you on?