Aussie Broadband (ABB) consistently rates well on Whirlpool:
https://forums.whirlpool.net.au/forum/92
Are the slow-downs time-of-day related? (I.e. peak-hour evening slow-down) If so, this sounds like your RSP could be providing insufficient CVC, but if it's random or continuous, this could point to a deeper issue.
What technology is your connection (FTTN, FTTC, FTTP, HFC, etc) and what speed does your modem sync at? If you are at the end of a long, corroded copper run on FTTN, it is possible that changing RSP won't make a lot of difference, but ABB are reported to be very good at helping customers get the best out of their connection, and pursuing matters with NBN where necessary.
Have you checked out your home's network cabling and Wi-Fi, to make sure that any issues under your control have been optimised? (E.g. get rid of all old phone points other than the actual internet connection, use dual-band Wi-Fi, avoid congested Wi-Fi channels, use Ethernet cabling where practical, etc.)
[Disclaimer: No, I'm not on ABB - I use Pivit FttP, who have a monopoly over internet connections in the community where I live.]