Information Security Stack Exchange is a question and answer site for information security professionals. It only takes a minute to sign up.
Sign up to join this community
I’ve written a little utility that, given a web site address, goes and gets some metadata from the site. My ultimate goal here is to use this inside a web site that allows users to enter a site, and then this utility goes and gets some information: title, URL, and description.
I’m looking specifically at certain tags within the HTML, and I’m encoding the return data, so I believe I’ll be safe from XSS attacks. However, I wonder if there are any other attack vectors that this leaves me open to.
What you're describing is a sort of proxy that returns basic information from an external website. I don't think it's a weird question at all, and I know of a couple of attack scenarios.
Your design practially allows SSRF (Server-Side Request Forgery) by design. This doesn't have to be a big deal, but can have serious consequences. For example, if you run your application in AWS, one can call the AWS meta-data endpoint, which in some cases can grant an attacker temporary access keys to parts of your AWS environment.
SSRF can also be used to access your internal network, the server's loopback interface, and other endpoints that your server is whitelisted for. You've got to make sure to blacklist internal addresses (I would resolve the domain and parse the IP address to match certain ranges). And you've got to block the AWS meta-data endpoint when using AWS.
Can users enter a full URL? Ensure that they can't use different schemas, like file:// or smb://, or they might read local files from your server. Also, note that if you return the URL on your website, they might use a javascript:-schema to perform XSS when the URL is clicked.
Are you using an XML parser to parse the external HTML? If so, make sure you're not vulnerable to XXE and secure your XML parser by not allowing External Entities.
Needless to say I hope you don't append your user's input into some system command, like curl -s [URL], since that would allow for Command Injection. It's so stupid that I don't like to mention it, but I've seen people do it.
Note that showing data from the external website in JavaScript-context or in HTML attributes requires different encoding than HTML encoding.
The rest depends on your application's architecture. Are you storing the data in a database? If so, make sure you're not vulnerable to SQL-Injection, etc. Good luck!
