Like when using public wifi, what can I install/use on both my Android phone and Windows laptop to make it impossible (or very difficult for someone who is not an expert) from seeing what websites i visit?
You can route your connection through a VPN. This will encrypt the connection from your device to the VPN provider, therefore making it impossible for the wifi router owner to scan your traffic. See Understanding VPNs : How VPNs can be used for anonymization? for more info.
I would recommend using a VPN(Virtual Private Network), and of course it should be switched on immediately after connecting to the access point you want to use. A VPN would assign you a different IP address and encrypt your network connection, so that even a skilled network admin would find it nearly impossible to see your traffic. There are a number of free ones you may want to try to see how it works or you may want to read a bit more about them.