I work for a Level 1 merchant with annual onsite QSA audits, so this is a huge deal.
PCI DSS 10.5 requires the entity under test to "Secure audit trails so they cannot be altered". In our Unix and Linux environment this requires that the syslog configuration be checked to see whether a) the permissions on all files used locally to store audit logs (defined as the auth, authpriv, and security facilities in syslog) are restricted to root, and b) these facilities are also exported to a log collector for safekeeping.
In the past, with classic syslog, we have managed this with a test of the syslog.conf file to determine which files to check permissions and which collector servers are in use. The issue is with the newer versions of syslog, namely rsyslog and syslog-ng.
The configuration files for these logging components are far more flexible and human-readable than their predecessors. However, we have been going quietly mad trying to validate configurations of the new tools for the following reasons:
In rsyslog (and syslog-ng) you can route messages on just about anything you can think of… including regular expressions. Not only can you route by those criteria, you could also completely discard certain messages based on them. For example, it would be easy enough to configure a system to look like it is logging all of the security stuff if one was simply looking through the logs, but all the while it is silently discarding any log message that contained a specific user ID. Easy to configure, difficult to detect.
With syslog-ng we have tried validating "known good" configurations with a checksum (this could be made more secure with a proper HMAC such as SHA-256, but that doesn't solve the underlying problem). The issue is that valid configurations tend to proliferate and it becomes impossible to keep up.
Both tools are capable of using include files, and rsyslog in particular uses this feature extensively "out of the box", which further complicates matters because now you have to checksum/hash not only the original file but every file it includes.
One stopgap is to flag every system running rsyslog or syslog-ng and require the SA to attest that the logs are being sent to the proper places, but with tens of thousands of systems, that approach does not scale, nor does it provide good evidence for auditors.
In short, we have found no technical solution to the problem of knowing for certain where every authentication log is being stored or transmitted, and yet PCI DSS 10.5 requires it. Therefore I am asking the security community what they are doing to facilitate this.