No public Certificate Authority (CA) will issue a certificate for any IP address. Certificates are issued on domain names, and a basic DV certificates (like the ones that Let's Encrypt hands out) will not be issued for domains set to a private IP address such as the 192.168.x.x blocks.
Because of that, you can never have an HTTPS connection to an IP-only address, and I would expect that I would not see an HTTPS connection to a consumer-grade router in my network. If I noticed such a connection (honestly, I probably wouldn't notice the lock one way or another), then I would be very suspicious that someone had installed rogue certificates, and has an actively snooping proxy somewhere on my machine or in my network.
It is possible to set up a secure HTTPS connection, depending on your router's capabilities. If you can add a certificate to your router, you can create a self-signed certificate. This will effectively encrypt your HTTPS connection inside of a perfectly valid TLS tunnel, although your browsers will be complaining very loudly that the validity of the certificate can't be verified.
There are ways to get past these complaints by the browser, such as adding yourself as a trusted CA on your computer, or getting a certificate for a domain and using your internal DNS or your hosts files to point the domain to the private IP, as mentioned in the comments.
You can also get a valid certificate and, within your own network, override the domain's A record to point to devices in your private net block, such as by editing your hosts files or using your own DNS server.
As far as the specific cases that you're asking about:
1, your own home: There should be nothing to worry about, assuming that all other devices are secure. Rogue devices on your network will be able to do packet sniffing, so don't log in using the same password that you log into your bank with (which is good advice regardless of what you're authenticating with).
2, you're hosting a public access point: Assume the network is being actively probed at all times. In this case, I might take the time to set up TLS properly and even then, if I were truly paranoid (I.e., my access point was set up at DefCon), then each time I access the management web page, I would disable wireless, ensure nothing else is connected via any hubs or switches, then change the password before re-enabling wireless and making my configuration changes.