The first point to observe is that if you click one of the above addresses you shouldn't be able to open it. They all point (clearly in the case of Azure) to a link-local IP address. These are reserved addresses in the space
169.254.0.0/16 that are not forwarded by routers, and crucially are only accessible from a given instance.
The second point to note is the data that the URL contains. User data can be stored via these APIs, but it shouldn't be as it's not protected by any cryptographic methods. Identity and Access Management (IAM) data can also be stored, and that is one of their strengths. But wait! Doesn't that mean any user with access can see data that allows them to do things they really shouldn't? Well...yes. However, that data is rotated on a relatively short time frame (~1 hour by default) meaning any reveal is limited, and the data can be rotated on demand.
As for the question as to why the URL is unencrypted? Yes, there is still network communication taking place, even if it was hosted on a local network interface. However, since it's only accessible from your instance, and any attacker looking to sniff network packets requires root access (for any reasonably normal scenario at least), it can't be intercepted by anybody except root - and if a user has root access to your machine, then it doesn't matter what data is stored in the credentials! Trying to determine exactly how the metadata server is called is tricky, but it's likely that the data is simply stored by the physical host of the virtual machine, and authentication is done by the physical host simply returning information corresponding to the virtual machine that the request comes from. In that sense, the requests do leave your instance, but they never cross a network to be intercepted, unless a physical host has been compromised - at which point, again, your system has been compromised anyway. AWS or Azure don't seem to detail anything specific, but Google seem to back up this logical assumption:
When you make a request to get information from the metadata server, your request and the subsequent metadata response never leaves the physical host running the virtual machine instance.
That doesn't quite explain it all however, and the real answer is speed. Any additional handshake takes time, and if there is no practical need to having the data encrypted then why do it? Securing it here offers no additional protection, and only serves to delay the many requests to the API.
Why then is it not a big worry that IAM credentials can be gained over these URLs in an insecure model? The answer lies in two principles of least privilege and defence in depth. First, the IAM roles assigned to the instances assigned should offer very limited capability, and ideally none to write or create new data or interfaces. Second, your application can be built to prevent requests to metadata - if somehow an attacker manages it, other defence mechanisms can be used such as not allowing requests from servers other than localhost or having alarms set when requests are made with credentials outside of a certain time period.
As a useful case for why you would want to utilise a metadata server:
To be able to read or write to a database ordinarily requires credentials to be stored somewhere on the server, normally in a fixed file or memory out of public view. However, IAM roles can be assigned to an instance to allow an application to fetch these credentials 'on the fly' without storing them, as well as being able to read them after the credentials are rotated on a frequent basis, thus automating what can otherwise be quite a tedious admin task.