sugar level wont come down

It all depends if there was sufficient insulin to move the glucose on so to speak....so not necessarily....

Walking has no affect on my BG.
I need to do more rigorous exercise to see a change - I think it depends on how much exercise you typically do.

The only way I would expect my BG to come down would be with a correction insulin dose (unless I already have some active insulin).

Walking affects my BSL about 6 hours later then levels drop unless I have anticipated this by lowering my basal pump insulin starting at the end of the walking.
But if you are walking about with a low level insulin ( such as without basal insulin, and some 5 + hours after short-acting insulin) then would you expect much change in your bsl?

Walking does drop me but very very slightly. Sometimes nothing. Mind you, it used to drop me tons when I was using premix insulin. It was not good because I felt restrained. Now I am more spontaneous and free on basal bolus