I went through this same thing. I bought an APC uninterruptible power supply and keep the modem and router plugged into. Also routing coaxial and Ethernet cables through it so it is 100% connected to the PSU. if power went out we were still able to maintain internet and charge our devices for several hours. However this is negated if there is an outage with your ISP. So yes and no. But for my experience it has been mostly yes to having Internet during a power outage.
I went through this same thing. I bought an APC uninterruptible power supply and keep the modem and router plugged into. Also routing coaxial and Ethernet cables through it so it is 100% connected to the PSU. if power went out we were still able to maintain internet and charge our devices for several hours. However this is negated if there is an outage with your ISP. So yes and no. But for my experience it has been mostly yes to having Internet during a power outage.