Multi-tenancy and Heartbleed

Question

We use a well-known service provider/API management gateway that sits in front of our RESTful APIs providing key management functionality etc. etc.

The service provider was running the dodgy OpenSSL version.

None of our web services have SSL enabled, though we use API keys to throttle API access, measure usage.

Traffic obviously arrives at the service provider's web servers which acts as a proxy and talks to our web servers.

If the infrastructure of the service-provider is such that many customers share a web-server instance and at least one customer has SSL enabled, then does that expose all customers, including those not using SSL, to the threat?

Apparently, the bug will not allow memory to be read outside of the process address space, so the answer is hinged around how the proxy web server runs a process per customer.

Has anyone covered this scenario?

Hypothetical Scenario

I am a programmer working for APIs-r-Us. Instead of running a separate website and process for each of my thousands of customers, sharding and all that, I write the proxy system as a single stateless web service running in a website instance handling all requests to apisrus.net

My service handles both SSL and non-SSL traffic.

My service uses some rules based on the URL to route requests to different customer web services. API consumers talk to my service and my service talks to the customer's real service, payloads can even be converted or augmented by my service before being sent back to the API consumer.

Loaded into process memory are the various routes and credentials to access my customer's APIs, as well as the keys and hashes use to authenticate the users.

All traffic is going via a single process, so if this process memory is compromised, then all customers are compromised.

I'm assuming that OpenSSL is loaded into the address space of the website (and not the server pipeline).

Answer 1

The answer to your specific scenario is "it depends".

Specifically, it depends on the finer details of how the application's memory allocator works, the threading mechanism it uses, and the application's inner structure. There are some scenarios (eg. old-fashioned fork+exec threading with a new thread to handle each request, or a memory allocator that always wipes memory before use plus per-thread memory pools plus dedicated SSL and non-SSL threads) in which non-SSL data cannot leak into the OpenSSL-accessible memory space. However, unless you know that one of these is the case, you need to assume that anything in the webserver's memory may have been leaked by Heartbleed.

Answer 2

If I resume correctly your situation in the below drawing (sorry for ugliness) :

Then, you are not at risk, as the data between your customers -> the provider -> your servers goes unencrypted and then does not go through the OpenSSL process memory.

The vulnerable server can still be exploited to dump private keys / customer data for other users and other services than yours though, but as the Heartbleed attack can not read other processes memory (as far as we know today), you are not affected.

On a side note, as Lucas Kauffman noted, the fact that your data goes unencrypted between your customers and your web servers is probably a more serious risk than the Heartbleed vulnerability.

Answer 3

Heartbleed is a memory scraping bug. Its not just keys that can be found, but ANY data that the affected machine handled. In the case of this proxy, then, the heartbleed vulnurabilty could see the memory in another process, that happened to be your customer's HTTP request to your server. Due to the nature of this flaw, ANY sensitive information that has passed through that machine should be considered compromised.