Benefits of random responses to exceptions over generic error responses

Question

Attackers can send requests with data that the server does not expect in order to try to get responses that reveal secret data. One common example is when the server experiences an exception. A poorly configured server might include a detailed error message in the response which leaks information the attacker can take advantage of. One common way to handle this issue is to reply with generic error messages, for example in the case of HTTP with an HTTP error response 400 or 404 with little to no additional information. Another approach I recently heard mentioned in a talk was to reply to exceptions with fake generated responses. For example by including fake exception details in error responses or by replying with an OK repsonse with fake data (in a way that leaves genuine users unaffected).

It feels like this second approach could be a case of security by obscurity. If the attacker knows that responses can be fake then the fake repsonses might no longer be much different than a generic error message. The only clear benefit I can think of is possibly confusing an attacker when they first encounter it until they figure out the scheme. The people who presented the approach seemed to think it was much better than generic error messages, however, and they work in security so I trust them more than myself. They did sadly not elaborate further in their talk and I fail to understand why it would be better. What could other possible benefits be?

Answer 1

This is definitely security by obscurity. I would strongly recommend against the approach of sending fake responses for a number of reasons.

• The effectiveness is going to be very low. Many applications already have poor error handling with incorrect or misleading responses, so attackers are well aware that they cannot blindly trust server responses.
• The approach can backfire. For preparation, attackers often perform mass scans to look for any application which behaves “interestingly” and is potentially vulnerable. For example, they may send characters which have a special meaning in SQL and then test if the server responds with an error code, indicating an SQL injecting vulnerability. If your application does respond with an interesting status code, this can catch the attacker's attention and make them try more sophisticated techniques against your application in particular.
• Wrong status codes and messages can cause a lot of bugs and confusion in perfectly legitimate use cases, even if you try to avoid this. The idea of generating fake data sounds particularly bad, because users may actually trust this data and never realize that something went wrong. People use websites in all kinds of ways, and the status code and error message can play a very important role. For example, imagine a visually impaired user who visits the website with a screen reader. In that case, you cannot assume that the user will ignore your bogus status messages based on visual cues.

The proper approach to error handling is to distinguish between end users and the developers working on the application.

• For end users, you want a correct and succinct explanation of what happened. Use the full range of HTTP status codes to indicate the issue and additionally include some short human-readable text. For client-side errors, go into detail and explain why the request invalid (missing parameters, unexpected values etc.). For server-side errors, keep the information generic, because it's not the job of your users to diagnose server issues, and internal information like file paths, database names etc. are none of their business. Just say something like “The request could not be processed due to an issue on the server. The administrator has been notified. Please try again later. For any questions, contact ...”
• For the application developers, you want as much technical information as possible, so that the problem can be diagnosed and fixed. Don't print this data on the screen. Write it into an internal log which can only be accessed by the people who need the information.