This is a great question! I'm also a developer on a c++ product that handles high sensitivity data, and we face this dilemma almost every day.

When a production system starts throwing alarms (especially performance-related ones, thought misbehaving software / bug ones also apply) we often need stack-trace level debugging turned on temporarily in order to diagnose the problem. These logs inevitably contain sensitive information: IP addresses or users involved in the transaction, sometimes pieces of the data, you name it. Often our customers don't feel comfortable giving us these logs for obvious reasons. It's tough.

The administrators actually running your server will know what level of sensitivity their data is, and what government or industry security policies they have to follow when handling it (for example: government, financial, and health data all have special rules). It's fair that your software has the ability to produce sensitive logs, but the admins need to be fully in control of how, when, and where they get produced and stored.

Some things you can do:

• Make sure you are very careful to correctly categorize all error messages containing (even potentially) sensitive information into the correct log level - INFO, WARNING, ERROR, DEBUG, TRACE, etc.
• Be explicit in your documentation about kinds of sensitive information is included in each log level. The more detail the better so the security-minded admins can do their jobs properly.
• Make sure DEBUG/TRACE log level is off in the software's default configuration, and that the admin has to run a gambit of warnings to turn it on (ie it's not possible to do by accident).
• You could send DEBUG/TRACE level logs to a separate log stream (you mentioned a separate "secure" log) that defaults to local storage, making it harder to "accidentally" ship it outside the secure zone, or get it mixed in with the standard logs.
• Ship your product with a redaction tool that knows how to mask all raw data, usernames, IP addresses, etc from your DEBUG/TRACE logs with unique placeholders (each unique username / IP gets masked by a unique A, B, .., AA, etc). Often the actual username / IP / data is irrelevant, but the key to diagnosing the problem is that the error only happens with data of type X, or that IP address A (and only A) always always sends its requests three times, or whatever. This way an admin can be confident that nothing sensitive is leaking if they ever ever need to move these logs out of the secure zone (for example sending them back to you for debugging).

Bottom line: I think it's fair that your software has the ability to log sensitive information for debugging purposes - and you'll thank yourself later when you have to debug a malfunctioning prod system! But make it as difficult as possible to turn on "by accident". Trust the admins to know their security requirements, and make their jobs easy.

This is a great question! I'm also a developer on a c++ product that handles high sensitivity data, and we face this dilemma almost every day.

When a production system starts throwing alarms (especially performance or configuration-related ones, thought misbehaving software / bug ones also apply) we often need stack-trace level debugging turned on temporarily in order to diagnose the problem. These logs inevitably contain sensitive information: IP addresses or users involved in the transaction, sometimes pieces of the data, you name it. Often our customers don't feel comfortable giving us these logs for obvious reasons. It's tough.

The administrators actually running your server will know what level of sensitivity their data is, and what government or industry security policies they have to follow when handling it (for example: government, financial, and health data all have special rules). It's fair that your software has the ability to produce sensitive logs, but the admins need to be fully in control of how, when, and where they get produced and stored.

Some things you can do:

• Make sure you are very careful to correctly categorize all error messages containing (even potentially) sensitive information into the correct log level - INFO, WARNING, ERROR, DEBUG, TRACE, etc.
• Be explicit in your documentation about kinds of sensitive information is included in each log level. The more detail the better so the security-minded admins can do their jobs properly.
• Make sure DEBUG/TRACE log level is off in the software's default configuration, and that the admin has to run a gambit of warnings to turn it on (ie it's not possible to do by accident).
• You could send DEBUG/TRACE level logs to a separate log stream (you mentioned a separate "secure" log) that defaults to local storage, making it harder to "accidentally" ship it outside the secure zone, or get it mixed in with the standard logs.
• Ship your product with a redaction tool that knows how to mask all raw data, usernames, IP addresses, etc from your DEBUG/TRACE logs with unique placeholders (each unique username / IP gets masked by a unique A, B, .., AA, etc). Often the actual username / IP / data is irrelevant, but the key to diagnosing the problem is that the error only happens with data of type X, or that IP address A (and only A) always always sends its requests three times, or whatever. This way an admin can be confident that nothing sensitive is leaking if they ever ever need to move these logs out of the secure zone (for example sending them back to you for debugging).

Bottom line: I think it's fair that your software has the ability to log sensitive information for debugging purposes - and you'll thank yourself later when you have to debug a malfunctioning prod system! But make it as difficult as possible to turn on "by accident". Trust the admins to know their security requirements, and make their jobs easy.