Virtual Memory Statistics ( vmstat )

vmstat -  vmstat reports virtual memory statistics of   process, virtual memory, disk, trap, and CPU activity.

On multicpu systems , vmstat averages the number of CPUs  into  the  output. For per-process statistics .Without options, vmstat displays a one-line summary  of  the  virtual memory activity since the system was booted.

Basic synctax is vmstat  <options>   interval  count

option – let you specify the type of information needed such as paging  -p , cache   -c ,.interrupt -i  etc.

if no option is specified  information about   process , memory , paging , disk ,interrupts & cpu  is displayed  .

interval – is time period in seconds between two samples . vmstat   4  will give data at each 4 seconds interval.

count  – is the number of times the data is needed . vmstat 4   5   will give data at 4 seconds interval   5

             times.

The following command displays a summary of what the  system is doing every five seconds.

     example% vmstat 5

     procs  memory          page             disk      faults        cpu
     r b w swap  free re mf pi p fr de sr s0 s1 s2 s3  in  sy  cs us sy id
     0 0 0 11456 4120 1  41 19 1  3  0  2  0  4  0  0  48 112 130  4 14 82
     0 0 1 10132 4280 0   4 44 0  0  0  0  0 23  0  0 211 230 144  3 35 62
     0 0 1 10132 4616 0   0 20 0  0  0  0  0 19  0  0 150 172 146  3 33 64
     0 0 1 10132 5292 0   0  9 0  0  0  0  0 21  0  0 165 105 130  1 21 78

The fields of vmstat’s display are
    procs
            r     in run queue
            b     blocked for resources I/O, paging etc.
           w     swapped

    memory (in Kbytes)
             swap -  amount  of  swap   space   currently   available               
             free   – size of the free list

    page ( in units per second).
          re    page reclaims -  see  -S  option  for  how  this field is modified.
          mf    minor faults -  see  -S  option  for  how    this field is modified.
          pi    kilobytes paged in
          po    kilobytes paged out
          fr    kilobytes freed
          de    anticipated short-term memory shortfall (Kbytes)
          sr    pages scanned by clock algorithm

    disk  ( operations per second )
          There are  slots for up to four disks, labeled with a single letter and number.
          The letter indicates  the  type  of  disk  (s = SCSI, i = IPI, etc) . The number is 
          the logical unit number.

    faults
           in    (non clock) device interrupts
          sy    system calls
          cs    CPU context switches

    cpu -   breakdown of percentage usage of CPU  time.  On multiprocessors  this is an a
               verage across all processors.
          us    user time
          sy    system time
          id    idle time

Results and Solutions:

A.   CPU issues:

Following columns has to be watched to determine if there is any cpu issue

Processes in the run queue (procs r)
User time (cpu us)
System time (cpu sy)
Idle time (cpu id)

     procs      cpu
     r b w    us sy  id
     0 0 0    4  14  82
     0 0 1    3  35  62
     0 0 1    3  33  64
     0 0 1    1  21  78

Problem symptoms:
1.) If the number of processes in run queue (procs r) are consistently greater than the number of CPUs on the system it will slow down system as there are more processes then available CPUs .
2.) if  this number is more than four times the number of available CPUs in the system then system is facing shortage of cpu power and will greatly slow down the processess on the system.
3.) If  the idle time (cpu id) is consistently 0 and if the system time (cpu sy) is double the user time (cpu us)  system is facing shortage of CPU resources.
Resolution :
Resolution to these kind of issues involves tuning of application procedures  to make efficient use of cpu  and as a last resort increasing the cpu power or adding more cpu to the system.  

B.   Memory Issues:
Memory bottlenecks are determined by the scan rate (sr) . The scan rate is the pages scanned by the clock algorithm per second. If the scan rate (sr) is continuously over 200 pages per second then there is a memory shortage.
Resolution :
1. Tune the applications & servers to make  efficient use of memory and cache.
2. Increase system memory .
3. Implement priority paging in s in pre solaris 8 versions by adding line “set priority paging=1″ in
    /etc/system. Remove this line if upgrading from Solaris 7 to 8 & retaining old /etc/system file.