Here i would like to share small piece of information about dtrace. I have used dtrace to find which process are making more system calls (i.e kernel usage). DTrace is very useful d to get the amount of memory, CPU time, filesystem and network resources used by the active processes. It can also provide much more fine-grained information, such as a log of the arguments with which a specific function is being called, or a list of the processes accessing a specific file.
There are few simple commands to find high kernel usage using dtrace.
Here we are just grepping all the system calls.
# dtrace -n 'syscall:::entry {@num[probefunc]=count();}' dtrace: description 'syscall:::entry ' matched 237 probes yield 1096 portfs 1400 getdents 1440 p_online 1536 accept 8022 modctl 8430 setcontext 8658 recv 11367 readlink 14599 lwp_park 18311 unlink 215507 statvfs 216256 open 257824 read 344775 gtime 351321 close 353795 getpid 437632 pollsys 685370 write 739093
So the above highlighted process are making more system call through those “pollsys” and “write” functions.Using below command ,we are finding what are process are calling “write” function more often.
# dtrace -n 'syscall::write:return {@num[execname]= count(); }' dtrace: description 'syscall::write:return ' matched 1 probe dtrace eauth_userpass 1 rcapd 2 nbrmms 3 sbatchd 3 expr 30 init 60 vemkd 60 csthb.agt 154 sas.e9bd41 188 syslogd 202 ls 212 sas.e9bd12 292 esd 510 adclient 560 java 880 sas 1619 jfd 30172 spdslog 30616
Using below command . i am finding what are process are calling “pollsys” function more often.
# dtrace -n 'syscall::pollsys:return {@num[execname]= count(); }' dtrace: description 'syscall::pollsys:return ' matched 1 probe vxdclid 1 esd 2 nbrmms 2 vmd 3 cdcwatch 6 sbatchd 6 acsssi 7 vxpal 7 pem 8 syslogd 8 init 9 sendmail 10 mbschd 11 vemkd 15 lim 22 motifxsassm 25 mbatchd 30 egosc 36 eauth_userpass 40 sshd 46 nrpe 151 jproxy 294 objspawn 515 top 522 sastcpd 588 sas.e9bd41 811 adclient 1118 sas.e9bd12 1301 cstd.agt 1471 sas 2762 java 13360 spdslog 20554 jfd 171801
From the above output, process name,we can identify the pid’s which are making the more system calls.The below mentioned process are making more system calls.
bash-3.00# ps -ef |grep -i jfd 0041199 26430 1 0 May 25 ? 10:31 /comm/pm/3.0/sparc-sol7-32/etc/jfd0041248 4491 1 0 May 25 ? 6:32 /comm/pm/3.0/sparc-sol7-32/etc/jfd0041199 11487 1 0 May 25 ? 835:06 /comm/ThirdParty/PlatformComputing/PM/7.1/sparc-sol10-64/etc/jfd0041248 10977 1 0 May 25 ? 437:53 /comm/ThirdParty/PlatformComputing/PM/7.1/sparc-sol10-64/etc/jfd root 11652 23517 0 13:54:23 pts/25 0:00 grep -i jfd
Note:Dtrace command will not stop unless you press control+C.You have to terminate it after 2 to 3 minutes
Thank you for reading this article.Please leave a comment if you have any doubt ,i will get back to you as soon as possible.
Leave a Reply