Troubleshooting network issue is not an easy task in any operating systems.In order to troubleshoot network issues, you need to capture the network packets in OS level and need a help from network team. Earlier we have discussed about snoop utility which is default packet analyzer in Solaris operating system.In Redhat Linux you have utility called “tcpdump” which is freeware and distributed under the BSD license.It runs on most of the Unix/Linux flavors. tcpdump output can be redirected to file and this can be analysed using tcpdump or other common network packet analyzer like wireshark.
TCPDUMP:
[root@mylinz ~]# tcpdump -help
tcpdump version 4.1-PRE-CVS_2009_12_11
libpcap version 1.0.0
Usage: tcpdump [-aAdDefIKlLnNOpqRStuUvxX] [ -B size ] [ -c count ]
[ -C file_size ] [ -E algo:secret ] [ -F file ] [ -G seconds ]
[ -i interface ] [ -M secret ] [ -r file ]
[ -s snaplen ] [ -T type ] [ -w file ] [ -W filecount ]
[ -y datalinktype ] [ -z command ] [ -Z user ]
[ expression ]
[root@mylinz ~]#
Options:
-S : Print absolute sequence numbers.
-e : Get the ethernet header.
-q : Show less protocol information.
-E : Decrypt IPSEC traffic by providing an encryption key.
-n : Don't resolve hostnames.
-nn : Don't resolve hostnames or port names.
-X : Show the packet's contents in both hex and ASCII.
-XX : Same as -X, but also shows the ethernet header.
-v, -vv, -vvv : Increase the amount of packet information you get back.
-c : Only get x number of packets and then stop.
-s : Define the snaplength (size) of the capture in bytes.
Use -s0 to get everything, unless you are intentionally capturing less.
Regular Expressions:
AND - and or && OR - or or || EXCEPT - not or !
Here we will see how to use tcpdump on redhat Linux.
tcpdump – Examples
1.To capture all the interfaces network traffic using tcpdump,just use “tcpdump”.
[root@mylinz ~]# tcpdump tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 19:30:36.790792 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 1825914036:1825914232, ack 225742662, win 71, length 196 19:30:36.791801 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 196:376, ack 1, win 71, length 180 19:30:36.792084 IP 192.168.10.75.57427 > mylinz.ssh: Flags [.], ack 376, win 16072, length 0 19:30:36.793203 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 376:620, ack 1, win 71, length 244 19:30:36.794220 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 620:768, ack 1, win 71, length 148 19:30:36.794474 IP 192.168.10.75.57427 > mylinz.ssh: Flags [.], ack 768, win 16425, length 0 3681 packets captured 3681 packets received by filter 0 packets dropped by kernel [root@mylinz ~]#
2.To capture the network traffic on the particular interface,
[root@mylinz ~]# tcpdump -i eth0 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 19:32:34.897001 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 1826490736:1826490932, ack 225746822, win 71, length 196 19:32:34.898342 IP 192.168.10.75.57427 > mylinz.ssh: Flags [.], ack 196, win 16363, length 0 19:32:34.898954 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 196:456, ack 1, win 71, length 260 19:32:34.899903 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 456:604, ack 1, win 71, length 148 19:32:34.900170 IP 192.168.10.75.57427 > mylinz.ssh: Flags [.], ack 604, win 16261, length 0 4683 packets captured 4683 packets received by filter 0 packets dropped by kernel [root@mylinz ~]#
3.To capture the “N” no of network packets , use “-c” option (To specify “N” value.)
The below command captured just 10 packets from interface eth0.
The below command captured just 10 packets from interface eth0.
[root@mylinz ~]# tcpdump -c 10 -i eth0 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 19:35:19.795327 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 1827237676:1827237872, ack 225754170, win 71, length 196 19:35:19.796254 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 196:376, ack 1, win 71, length 180 19:35:19.797602 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 376:524, ack 1, win 71, length 148 19:35:19.798317 IP 192.168.10.75.57427 > mylinz.ssh: Flags [.], ack 524, win 16177, length 0 19:35:19.798717 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 524:768, ack 1, win 71, length 244 19:35:19.800621 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 768:916, ack 1, win 71, length 148 19:35:19.800964 IP 192.168.10.75.57427 > mylinz.ssh: Flags [.], ack 916, win 16079, length 0 19:35:19.802653 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 916:1160, ack 1, win 71, length 244 19:35:19.804629 IP mylinz.ssh > 192.168.10.75.57427: Flags [P.], seq 1160:1308, ack 1, win 71, length 148 19:35:19.804879 IP 192.168.10.75.57427 > mylinz.ssh: Flags [.], ack 1308, win 16425, length 0 10 packets captured 10 packets received by filter 0 packets dropped by kernel [root@mylinz ~]#
4.To display ASCII output of the network packets,use “-A” option.
[root@mylinz ~]# tcpdump -A -i eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
19:46:14.964464 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 334492741:334492937, ack 1238794546, win 71, length 196
E.....@.@..C..
...
K.......EI..2P..G.......E6.h....
....q.. f..P_au.0INC3..^.8....H;..<.*1.|......9>.\ND-..DX.....X..-..c... AX.WG....n../..W.Ei..Q_[?.o....)i.......f.6~.=...1....*...Y. .1....V
@.w.....i.{.'..N...A........O
19:46:14.965784 IP 192.168.10.75.57515 > mylinz.ssh: Flags [.], ack 196, win 16192, length 0
E..(*.@...;$..
288 packets captured
326 packets received by filter
8 packets dropped by kernel
[root@mylinz ~]# ^C
5.To capture the packets in Hexadecimal and ASCII , use “-XX” option.
[root@mylinz ~]# tcpdump -XX -i eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
19:48:42.296944 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 334885025:334885221, ack 1238796798, win 71, length 196
0x0000: 0050 56c0 0008 000c 2927 0e8a 0800 4510 .PV.....)'....E.
0x0010: 00ec cf5c 4000 4006 d4f5 c0a8 0a0e c0a8 ...\@.@.........
0x0020: 0a4b 0016 e0ab 13f5 f0a1 49d6 89fe 5018 .K........I...P.
0x0030: 0047 9688 0000 bac1 5e05 9ed7 d400 51d0 .G......^.....Q.
0x0040: f886 01d4 b861 4423 3f8d 820d cc8e 74ce .....aD#?.....t.
0x0050: 8658 81e3 52e0 cf27 f3d9 d1fa af87 c9a4 .X..R..'........
0x0060: 801a 4c08 cdce d40a 36ef 05a5 8008 f605 ..L.....6.......
0x0070: 076c 4488 e29f 3939 007b d5bb 92ee ce0a .lD...99.{......
0x0080: 2492 e6df 0470 839d 19b6 33e8 ee82 d514 $....p....3.....
0x0090: 290b 9dbb b9b3 9759 d706 a4a7 2508 7440 )......Y....%.t@
0x00a0: 5593 5c0e f54f dbe5 3cf9 3729 7efe bb10 U.\..O..<.7)~...
0x00b0: db9c 15cd a9bb b8ec 0dbe d7c4 989b a6d4 ................
0x00c0: 6363 449e e580 e4b5 4728 4f2c d15f 605d ccD.....G(O,._`]
0x00d0: 5369 39ba b06b ed01 e4dc 16d8 3a1d ae8c Si9..k......:...
0x00e0: d961 4590 5385 ac90 9fb7 6b62 5b0f f755 .aE.S.....kb[..U
0x00f0: af7c 13d0 f4e6 9d33 a64d .|.....3.M
90 packets captured
390 packets received by filter
270 packets dropped by kernel
[root@mylinz ~]#
6.To capture the network traffic between two IP address.
Source IP address: Local system IP
Destination IP address:Remote system IP
[root@mylinz ~]# tcpdump src 192.168.10.14 and dst host 192.168.10.75 -c 10 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 22:14:38.655205 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 343323697:343323893, ack 1238875346, win 116, length 196 22:14:38.657136 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 196:376, ack 1, win 116, length 180 22:14:38.658088 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 376:524, ack 1, win 116, length 148 22:14:38.659090 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 524:672, ack 1, win 116, length 148 22:14:38.660085 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 672:820, ack 1, win 116, length 148 22:14:38.661108 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 820:968, ack 1, win 116, length 148 22:14:38.662085 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 968:1116, ack 1, win 116, length 148 22:14:38.663097 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 1116:1264, ack 1, win 116, length 148 22:14:38.664082 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 1264:1412, ack 1, win 116, length 148 22:14:38.665116 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 1412:1560, ack 1, win 116, length 148 10 packets captured 10 packets received by filter 0 packets dropped by kernel [root@mylinz ~]#
7.To capture the specific local port traffic using tcpdump,
[root@mylinz ~]# tcpdump src port 22 -c 5 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 22:16:35.920556 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 344247425:344247621, ack 1238882158, win 116, length 196 22:16:35.922482 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 196:376, ack 1, win 116, length 180 22:16:35.922679 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 376:524, ack 1, win 116, length 148 22:16:35.923242 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 524:672, ack 53, win 116, length 148 22:16:35.924449 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 672:820, ack 53, win 116, length 148 5 packets captured 5 packets received by filter 0 packets dropped by kernel [root@mylinz ~]#
8.To capture network traffic of destination port ,
[root@mylinz ~]# tcpdump dst port 22 -c 5 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 22:17:41.353796 IP 192.168.10.75.57515 > mylinz.ssh: Flags [.], ack 344249861, win 16099, length 0 22:17:41.555751 IP 192.168.10.75.57515 > mylinz.ssh: Flags [.], ack 149, win 16062, length 0 22:17:41.755814 IP 192.168.10.75.57515 > mylinz.ssh: Flags [.], ack 281, win 16425, length 0 22:17:41.955932 IP 192.168.10.75.57515 > mylinz.ssh: Flags [.], ack 413, win 16392, length 0 22:17:42.156828 IP 192.168.10.75.57515 > mylinz.ssh: Flags [.], ack 545, win 16359, length 0 5 packets captured 5 packets received by filter 0 packets dropped by kernel [root@mylinz ~]#
9.To filter the network traffic using destination IP address and local port number,
[root@mylinz ~]# tcpdump dst 192.168.10.75 and src port 22 -c 5 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 22:20:33.786893 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 344284213:344284409, ack 1238907950, win 116, length 196 22:20:33.788807 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 196:376, ack 1, win 116, length 180 22:20:33.789760 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 376:524, ack 1, win 116, length 148 22:20:33.790759 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 524:672, ack 1, win 116, length 148 22:20:33.791749 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 672:820, ack 1, win 116, length 148 5 packets captured 5 packets received by filter 0 packets dropped by kernel [root@mylinz ~]#
10.To filter the specific network traffic,use the below command.
[root@mylinz ~]# tcpdump net 192.168.10.0/24 -c 5 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 22:23:23.814953 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 347457749:347457945, ack 1238926462, win 116, length 196 22:23:23.816086 IP 192.168.10.75.57515 > mylinz.ssh: Flags [.], ack 196, win 16154, length 0 22:23:23.816903 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 196:456, ack 1, win 116, length 260 22:23:23.817995 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 456:604, ack 1, win 116, length 148 22:23:23.818302 IP 192.168.10.75.57515 > mylinz.ssh: Flags [.], ack 604, win 16425, length 0 5 packets captured 5 packets received by filter 0 packets dropped by kernel [root@mylinz ~]#
11.To filter the traffic using the protocal, Ex:ICMP
[root@mylinz ~]# tcpdump icmp tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 22:24:43.800087 IP 192.168.10.75 > mylinz: ICMP echo request, id 1, seq 4127, length 40 22:24:43.800126 IP mylinz > 192.168.10.75: ICMP echo reply, id 1, seq 4127, length 40 22:24:44.800979 IP 192.168.10.75 > mylinz: ICMP echo request, id 1, seq 4128, length 40 22:24:44.801018 IP mylinz > 192.168.10.75: ICMP echo reply, id 1, seq 4128, length 40 ^C 4 packets captured 4 packets received by filter 0 packets dropped by kernel [root@mylinz ~]#
12.To filter the traffic using port range,
[root@mylinz ~]# tcpdump portrange 20-24 -c 4 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes 22:26:13.002994 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 348186937:348187133, ack 1238932494, win 116, length 196 22:26:13.004058 IP 192.168.10.75.57515 > mylinz.ssh: Flags [.], ack 196, win 16425, length 0 22:26:13.004881 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 196:456, ack 1, win 116, length 260 22:26:13.005861 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 456:604, ack 1, win 116, length 148 4 packets captured 4 packets received by filter 0 packets dropped by kernel [root@mylinz ~]#
13.To capture the network packets to the file, use “-w” option.File should have extension with the name “pcap”.
[root@mylinz ~]# tcpdump portrange 20-24 -w saveme.pcap tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes ^C3 packets captured 3 packets received by filter 0 packets dropped by kernel [root@mylinz ~]# file saveme.pcap saveme.pcap: tcpdump capture file (little-endian) - version 2.4 (Ethernet, capture length 65535) [root@mylinz ~]#
14.To read the captured file,use “-r” option.
[root@mylinz ~]# tcpdump -r saveme.pcap reading from file saveme.pcap, link-type EN10MB (Ethernet) 22:31:19.306713 IP mylinz.ssh > 192.168.10.75.57515: Flags [P.], seq 348192593:348192725, ack 1238936498, win 116, length 132 22:31:19.503628 IP 192.168.10.75.57515 > mylinz.ssh: Flags [.], ack 132, win 16132, length 0 22:31:22.930852 IP 192.168.10.75.57515 > mylinz.ssh: Flags [P.], seq 1:53, ack 132, win 16132, length 52 [root@mylinz ~]#
Still there are plenty of options that you can use to filter the network traffic.Check out the advanced tcpdump options here.Hope this tcpdump tutorial has covered almost everything on L2 Level troubleshooting.
If you couldn’t find the tcpdump in Redhat Linux, install it using yum.
[root@mylinz ~]# yum list |grep tcpdump This system is not registered with RHN. RHN support will be disabled. tcpdump.x86_64 14:4.0.0-3.20090921gitdf3cb4.1.el6 [root@mylinz ~]# yum install tcpdump.x86_64 Loaded plugins: refresh-packagekit, rhnplugin This system is not registered with RHN. RHN support will be disabled. Setting up Install Process Package 14:tcpdump-4.0.0-3.20090921gitdf3cb4.1.el6.x86_64 already installed and latest version Nothing to do [root@mylinz ~]#
If you don;t have yum repository ,create one for you .
Thank you for reading this article.
bsreekanth says
pls provide cloud computing article
Feji Thomas says
Thank you very much. Very useful article for the troubleshooting.
vendhan says
Really very helpful article for the ppl who are preparing for interview.. Thanks..