Web Application Firewalls relies on negative and positive security traffic rules to protect web application from being exploit. My question is that is their any other web traffic modelling technique that can identify the unintended violation in the workflow of web application
Here are some research papers that tackle this sort of problem and might interest you:
The Ripley paper runs a replica of the client-side code on the server, to verify that the sequence of requests from the client could have arisen from a legal interaction with the system.
The WWW 2009 paper derives a finite-state automaton that captures all possible legal sequences of requests that could arise (while interacting with the web application through its intended interface), and then uses this to build a runtime monitor that can detect attacks by detecting deviations from this finite-state automaton. Several other works also take a similar approach of attempting to somehow obtain a finite-state automaton that characterizes all legal sequences of requests, and then enforcing the requirement that requests must be consistent with this finite-state automaton.