I am building a REST API to store security sensitive data for both my client and third-party partners.
Each request is validated using an HMAC of certain data which is keyed on a temporary key assigned for a short duration after the user is logged in. Of course everything runs over TLS. Every request gets logged, along with the keys to any data returned, as well as the keys to any data that would have been returned if the request were not blocked for some reason.
If the data is not present, or if the data at a location may be read by a particular user, but not updated, or if the update request is malformed, should the server return a meaningful error message, or not?
If the user does not have permissions to perform an action, a 401 or 403 response should be returned, indicating that the request failed due to lack of permissions to perform the action. Other response codes may be used in the case where the data is malformed, or the expected data types do not match (sending a string to an int field, etc.). Basically, follow good REST/HTTP practices and log everything, but don't provide any information to the user beyond a basic hint as to what went wrong.
Never give anyone any information, in case it could be used against the system in some way. Only ever return a 500 error, with no description of any kind. There is some level of disagreement as whether or not to even return a 404, or if an empty response or 500 error should be returned instead. If a third-party developer is encountering these responses, and doesn't understand why, they must contact our company, have someone look into the logs and explain how to make the requests properly.
Does anyone have "from the trenches" experience as to which methodology makes the most sense? Of particular concern is how to balance security with usability and support requests.