The graphs for one server that I am using are showing consistent 7.5 kb/s outgoing network traffic that started a few days ago.
How can I monitor/log/capture/analize this traffic to see what is it, where is it coming from and where does it go to?
The server is a linode (hosted at linode.com) running linux debian lenny.
Are there any command line http sniffers for linux ?
Have a look at tshark.
It's like wireshark, but then for commandline. Just install it with apt-get.
A tutorial on how to use it can be found here. You can easily filter for http with it.
To capture http traffic:
tshark -R "tcp.port == 80" -r /tmp/capture.cap
If port 80 is your http port. If you don't know the port just capture everything and you can later filter it for http. Filtering for http while capturing is not possible.
Sniffers
For commandline sniffers, run tcpdump. You can run this from the server itself or from another computer in the same network segment.
Capture packets. Ctrl-C to stop the capture (remember to stop the capture at some point ...):
tcpdump -w test.pcap -i eth0
Then to read the file:
tcpdump -r test.pcap
Wireshark will provide a nice GUI and more analysis tools, but works much like tcpdump.
Programs sending data
You will also want to run netstat or lsof to see what the server is actively connected to and what local process initiated the connection.
All tcp ports w/ program name that opened them:
netstat -pat
List all internet ports open:
lsof -i
Bro detects traffic on non-standard ports. Unlike other tools, it has notion of inbound and outbound traffic once you tell it the address space of your network:
bro -r trace.pcap local "Site::local_nets += { 1.2.3.0/24, 5.6.7.0/24 }"
See the quickstart guide for details on how to get started. Running Bro with the default arguments, i.e.,
bro -i <interface>
creates already a variety of log files in the current directory. The connection log (conn.log), contains for example entries along the lines of:
# ts uid orig_h orig_p resp_h resp_p
1311627961.8 HSH4uV8KVJg 192.168.1.100 52303 192.150.187.43 80
These are only a subset of the available columns. The log files are designed so that you can easily process them with awk and friends. E.g., to see the connection breakdown by service:
bro-cut service < conn.log | sort | uniq -c
This and other quick analyses will quickly tell you what's going on in your network.
another option is the iptraf tool - it's awesome. quick & easy.
% sudo iptraf
You can use justniffer. It's a cool http sniffer that logs network traffic in apache log format.
So you can postprocess logs with any web log analyzer, such as awstats or Piwik
There is lots of tools in the dsniff package ( http://www.monkey.org/~dugsong/dsniff/ ) see urlsnarf, etc.
