I wanted to know if anyone had experience sensoring single flows that generate 90 kpps or upward of traffic. My conundrum is that I use tools which I would like to be able to properly see an entire flow with (Zeek, Suricata), however both of these utilities are not capable of handling larger traffic volumes if a single connection exceeds a "worker thread's" maximum processing rate. Because a single connection is able to generate this volume of traffic, I am not aware of a load balancing method and accompanying software that would produce similar resulting information from Zeek & Suricata(Understandably this matters a lot less with Zeek, as seeing the entire connection isn't strictly necessary, however having a giant connection fill the rx ring results in losing subsequent connections).
For referencing I'm currently using the AF_Packet family sockets for both Suricata in Zeek, which are both in their own fanout. I get roughly 48,000pps running suricata with 4 capture threads with my current configuration and ~26,000pps per Zeekctl cluster.
I'm aware the most sensible thing to do would be to strictly analyze the flow, and then filter it out pre-sensor, however I'm doubtful policy in my org will allow me to do this, but I've been unlucky in my search for a satisfactory tool or reconfiguration.
I am hoping someone might have experience handling this type of situation or knows of software which has a less restrictive processing per "worker/thread" rate that would produce similar results(or honestly any results that aren't massive amounts of packet loss).