Hi and welcome to the forums
Have a look at the NDSS website, which gives diabetics access to free/subsidised supplies
You shouldn't need private health insurance as long as you are covered by medicare (disclaimer do your own due diligence on this)
You get insulin/prescriptions from a GP. If you can register with one that "bulk bills" you won't pay to see them, but in practice you may have to pay a top up fee for appointments.
If you're going to be there for a couple of years you'll want ypur GP to refer you to a hospital clinic or to a private endocrinologist.
Not sure what the status is on pumps atm (I'm MDI). As I remember it (6 years ago) a pump using friend had his private health insurance pay for the pump but all the consumables were covered. I suggest you check on the NDSS website. Private health insurance paid for his pump but I think they exclude pre-existing conditions for a while , so you might need to pay for the pump if it goes out of warranty during your stay?