We really can't advise you to ditch a prescribed medication, that's between you and your doctor, and ultimately it's your decision.
Whatever you do, I'd do more than just occasional testing.
Especially, I'd test when you have symptoms, to make sure it is in fact a hypo and not a 'false hypo'. (False hypos are caused by the body being used to higher numbers and panicking when you go lower than it's used to, it definitely does feel like a real hypo.)
That way you can at least prove you are having hypos.
Should you decide to ditch the meds, I'd test even more often to make sure you're not running high all the time.