How to check server load


If the server load is high then do the following tricks:

1) Top
check for the server load and watch for process

2) free -m
it will display the memory status

3)if  a particular user is causing a high server load then use "ps U username": it will display the process that are running by the user .

4) Then you can take action against that user.

4) netstat -an |grep :80 |wc -l :
Show how many active connections there are to apache (httpd runs on port 80)
To check port open or not netstat -tnpl | grep httpd

5) netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort –n
netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort –n
netstat -anp |grep '80' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
netstat -anp |grep '21' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
netstat -anp |grep '3306' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort –n
netstat -utn | grep :80 | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n | tail -10


6)if there is MYSQL taking cpu usage then check for mysql status
mysqladmin -u root process list
check for the databases and suspend that user if any database is using high resources.

7)check /tmp if there is any process taking high resources : chown to root for that process and chmod 000 also if .sh process is running then chown it t0 root:root

8)if exim is taking high resources then check for exim logs : use this command :
replace :blackhole: : fail: -- /etc/valiases/*


General
Top –c d2  
top –d 2
free –m : Memory status
w : Current users
ps –aufx : show cpu usages
netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort –n
netstat -plan|grep :25|awk {'print $5'}|cut -d: -f 1|sort|uniq -c|sort -nk 1
exim -bp | exiqsumm
ps aux | head -1;ps aux --no-headers| sort -rn +3 | head :
 Use below mentioned command to get top memory consuming processes
ps aux | head -1;ps aux --no-headers | sort -rn +2 
ps aux | head -1;ps aux --no-headers | sort -rn +2





If Exim causes
exim –bpc
shows the total no of email in qmail

eximstats -nr -ne /var/log/exim_mainlog
Total mail server report

pidof exim
shows no of exim pids running

exim -bpr | grep frozen | wc -l
Shows no of frozen emails

exiqgrep -z -i | xargs exim –Mrm
exim -bp | exiqgrep -i | xargs exim -Mrm
it deletes the FROZEN mails from the server

tail -f /var/log/exim_mainlog | grep public_html
check for spamming if anybody is using php script for sending mail through public_html

tail -f /var/log/exim_mainlog | grep /tmp
Used for checking for who is spamming through the /tmp

tail -3000 /var/log/exim_mainlog |grep 'rejected RCPT' |awk '{print$4}'|awk -F\[ '{print $2} '|awk -F\] '{print $1} '|sort | uniq -c | sort -k 1 -nr | head -n 5 
It will display the IP and no of tries done bu the IP to send mail but rejected by the server.

netstat -plan|grep :25|awk {'print $5'}|cut -d: -f 1|sort|uniq -c|sort -nk 1
shows the  connections from a certain ip to the   SMTP server

exim -bp | exiqsumm | more
It shows the domain name and the no of emails sent   by that domain

If  spamming from outside domain then you can block that domain or email id on the server
pico /etc/antivirus.exim
Add the following lines:
if $header_from: contains "name@domain.com"
then
seen finish
endif

eximstats -nr -ne /var/log/exim_mainlog


Catching spammer

exim -bp | exiqsumm | more

exim -bpr | grep "<*@*>" | awk '{print $4}'|grep -v "<>" | sort | uniq -c | sort -n
That will show you the maximum no of email currently in the mail queue have from or to the email address in the mail queue with exact figure.

exim -bpr | grep "<*@*>" | awk '{print $4}'|grep -v "<>" |awk -F "@" '{ print $2}' | sort | uniq -c | sort -n
That will show you the maximum no of email currently in the mail queue have for the domain or from the domain with number.

Check if any php script is causing the mass mailing with
cd /var/spool/exim/input
egrep "X-PHP-Script" * -R
Just cat the ID that you get and you will be able to check which script is here causing problem for you.

To Remove particular email account email
exim -bpr |grep "ragnarockradio.org"|awk {'print $3'}|xargs exim -Mrm



If Mysql causes :
mysqladmin -u root processlist
mysqladmin version
watch mysqladmin proc




If Apache causes :
netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort –n
netstat -an |grep :80 |wc –l
netstat -n | grep :80 | wc -l;uptime ; netstat -n | wc –l
netstat –tupl
netstat –tupl
netstat -utn | grep :80 | awk '{print $5'} | cut -d: -f1 | sort | uniq -c | sort -n
                             

Other Commands

pidof php
shows the PIDs for php

history | netstat
shows no of connection details

lsof  -p pid
shows the details of pid

mysqladmin -u root processlist

watch mysqladmin proc

netstat -an |grep :80 |wc –l

very imp
netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

netstat -na |grep :80 |sort

Use below mentioned command to get top memory consuming processes"

ps aux | head -1;ps aux --no-headers| sort -rn +3 | head

Use below command to get top cpu consuming processes:

ps aux | head -1;ps aux --no-headers | sort -rn +2 |more

You can check if any backup is going on, run the following commands:
# ps aux | grep pkg
# ps aux | grep gzip
# ps aux | grep backup
 If any backup process is going on, kill that process.

We can trace the user responsible for high web server resource usage by the folowing command
cat /etc/httpd/logs/access_log | grep mp3
cat  /etc/httpd/logs/access_log | grep rar
cat  /etc/httpd/logs/access_log | grep wav etc


cat /etc/httpd/logs/access_log | grep 408 can be used to check for DDOS attacks on the server.

cat  /etc/httpd/logs/access_log | grep rar

Port scanning :
root@server161 [/tmp]# nmap localhost

Starting Nmap 4.11 ( http://www.insecure.org/nmap/ ) at 2007-10-04 13:22 MYT
Interesting ports on localhost (127.0.0.1):
Not shown: 1664 closed ports
PORT     STATE SERVICE
1/tcp    open  tcpmux
21/tcp   open  ftp
25/tcp   open  smtp
53/tcp   open  domain
80/tcp   open  http
110/tcp  open  pop3
111/tcp  open  rpcbind
143/tcp  open  imap
443/tcp  open  https
465/tcp  open  smtps
631/tcp  open  ipp
783/tcp  open  spamassassin
953/tcp  open  rndc
993/tcp  open  imaps
995/tcp  open  pop3s
3306/tcp open  mysql

Nmap finished: 1 IP address (1 host up) scanned in 0.125 seconds
root@server161 [/tmp]#

netstat -anp |grep :3306

Hello Team,

Please do not restart apf or iptables on following servers,

202.75.39.88
203.121.68.68
203.121.73.130

DDOS ATTACK :

Who

 top –d 2

ps -aux|grep HTTP|wc –l : It will show you no of http connections to the server

netstat -lpn|grep :80 |awk '{print $5}'|sort    : It will helpful to check the no of connections from a certain ip

Check for the ips and block them with firewalls as apf/csf/iptables

For iptables : iptables -A INPUT -s <Source IP> -j DROP

For apf : apf –d ip_address


If load due to SPAMD:

Issue
The load issues on a cpanel 11 box (later versions too) because of spamd are quite common, sometimes they may take more than 90% of the CPU. Here are some ways to resolve it.
[edit] Solution
We can use the following steps to resolve the issue in a considerable manner.

1. Edit the /etc/mail/spamassassin/local.cf file

vi /etc/mail/spamassassin/local.cf

then add the following lines to it

a) use_bayes 0 (this will disable some spamassassin functionalities)
b) lock_method flock

(Uncomment these options, if they are commented)

2) Run scripts2/autofixer/spamd_dbm_fix

a) Log into WHM and use another URL as "http://your server IP:2086//scripts2/autofixer"
   then type the script name "spamd_dbm_fix" and run

b) Or you can also do the same from server back end as follows.

# wget http://httpupdate.cpanel.net/autofixer/spamd_dbm_fix
# chmod 755 spamd_dbm_fix
# ./spamd_dbm_fix

3) Run the following lines of codes

# rm -f /home/*/.spamassassin/__db.bayes_toks.new
# rm -f /home/*/.spamassassin/*.lock

NOTE : If the load issue is related to a particular user then there is no need for running this,
       you can simply enter into the '.spamassassin/' directory for that user and delete the
       files '__db.bayes_toks.new' and *.lock.


4) Restart spam assassin (restart Exim)

5) :)Hope this will help you in such a situation.

: memory usage serverwide.
ps aux | awk '{print $4"\t"$11}' | sort | uniq -c | awk '{print $2" "$1" "$3}' | sort –nr  


[root@emerald ~]# mymem
Memory Usage
------------
Current:        326.344 MB
Allocated:      492.012 MB
Guaranteed:     512 MB
Burstable:      768 MB

====================

5.1 1 spamd
0.7 1 spamd
0.6 1 /usr/bin/spamd
0.4 1 /usr/sbin/mysqld
0.2 1 cpsrvd
0.1 4 /etc/authlib/authProg
0.1 1 eximstats
0.1 1 cpdavd
0.1 1 cpbandwd
0.1 1 cpanellogd
0.1 1 cPhulkd
0.1 1 /usr/local/cpanel/bin/leechprotect
0.0 6 /usr/libexec/courier-authlib/authdaemond
0.0 4 /usr/sbin/courierlogger
0.0 4 -bash
0.0 3 sshd:
0.0 3 /usr/sbin/exim
0.0 3 /usr/lib/courier-imap/libexec/couriertcpd
0.0 2 sort
0.0 2 awk
0.0 11 /usr/local/apache/bin/httpd
0.0 1 xinetd
0.0 1 uniq
0.0 1 syslogd
0.0 1 pure-ftpd
0.0 1 ps
0.0 1 init
0.0 1 crond
0.0 1 chkservd
0.0 1 antirelayd
0.0 1 /usr/sbin/sshd
0.0 1 /usr/sbin/saslauthd
0.0 1 /usr/sbin/pure-authd
0.0 1 /usr/sbin/portsentry
0.0 1 /usr/sbin/named
0.0 1 /etc/authlib/authProg
0.0 1 /bin/sh
%MEM 1 COMMAND
[root@emerald ~]# mymem

====================================================

kill nobody process
ps aux | grep nobody | awk '{print $2}' | xargs kill -9

To see only the memory resources occupied by each category of processes, such as Apache httpd, MySQL mysqld or Java, use the following command:

ps aux | awk '{print $4"\t"$11}' | sort | uniq -c | awk '{print $2" "$1" "$3}' | sort -nr
====================================================

Okay, so everyone knows about ps. But I'll just highlight one of my favorite options:

ps -eo pid,%cpu,vsz,args,wchan

Shows every process, their pid, % of cpu, memory size, name, and what syscall they are currently executing. Nifty.

whereis locates source/binary and manuals sections for specified files. The supplied names are first stripped of leading pathname components and any (single) trailing
 extension of the form .ext, for example, .c. Prefixes of s. resulting from use of source code control are also dealt with. whereis then attempts to locate the desired program in
 a list of standard Linux places. For example find out where is ls command?
$ whereis ls

cd /var/spool/exim/input
egrep "X-PHP-Script" * -R

For netstat

netstat -nap | grep :110 | awk '{print $5}' | cut -d":" -f1 | sort | uniq -c | sort -nr | head
netstat -nap | grep :25 | awk '{print $5}' | cut -d":" -f1 | sort | uniq -c | sort -nr | head
netstat -n | grep :80 | wc -l;uptime ; netstat -n | wc -l
netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
netstat -nap | grep :80 | awk '{print $5}' | cut -d":" -f1 | sort | uniq -c | sort -nr | head


root@server58 [~]# tail -f /var/log/slow.log
[root@server60 log]# tail -500 /var/log/mysql-slow-queries.log

 ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10


Comments

Popular Posts

Install and configure rsyslog Centralized logging server in CentOS

How to fix postfix/smtp Network is unreachable error

Could not join realm: Necessary packages are not installed: sssd-tools sssd libnss-sss libpam-sss adcli