Cleaning Memory on Linux

If your system shows a lot of inactive memory or Swap space in use and you wish to cleanup before starting next job, then here is what you can do:

$ sync

# Cleanup memory held-up in Inactive and return to Unused, also removes unused pages/inodes/delentries from used memory
$ echo 3 > /proc/sys/vm/drop_caches

# Empty used up Swap space
# Disable swap and then re-enable
# This method is an aggressive one and if there isn't enough RAM free to hold up required pages from Swap, the system may crash
$ /sbin/swapoff -a
$ /sbin/swapon -a

Linux Tools

This list will continue to grow as I come across useful tools in Linux. I have also listed the command-line that I most frequently use:

ps - get the process information

• ps -flycae # will give information on scheduling class and kernel function name where process is sleeping

mcelog - Machine check log

• Determine system errors, decode, and log in syslog like /var/log/messages
  mcelog --dmi --syslog
  PS: The decoded hardware module is sometimes not helpful in determining the module at fault

OProfile - Profiling Linux system

• Covered under separate post here. 

mdadm - Manage Software  RAID

• Use mdadm for software mirroring devices:
  mdadm -C --level=raid1 --raid-devices=2 /dev/md0 /dev/sdb1 /dev/sdc1
  mdadm --manage /dev/md0 --fail /dev/sdc1
  mdadm --manage /dev/md0 --remove /dev/sdc1
  mdadm --manage --stop /dev/md0
  For striping, better to use LVM

LVM - Logical Volume Manager

• pvcreate <list of devices>
• vgcreate <vg-name> <list of pv devices>
• lvcreate -i<#of vg devices to stripe across> -I<stripe-size-default64k> -L<size-in-MB> -n<volume-name> <vg-name>

MUTT - The Mutt Mail User Agent

• echo "<message-body>" | mutt -a <attachment-file> -s "<subject>" <To email> [-b <BCC email>] [-c <CC email>] 

OProfile: System / JIT Profiling on Linux

Requirements

• Oprofile tar
• kernel-debuginfo-`uname -r`.rpm
• kernel-debuginfo-commons-`uname -r`.rpm

Setup

Install kernel-debuginfo rpms for the kernel you are running.

As root user:

$ tar zvxf oprofile-0.9.7.tar
$ cd oprofile-0.9.7
$ CFLAGS="-m64" CXXFLAGS="-m64" ./configure \

   --with-java={my_jdk_installdir} \
   --libdir=/usr/local/lib64

Ensure that configure has passed, otherwise resolve any issues with missing headers and libraries and rerun configure as above. One common issue people run into on RHEL 5:
checking libiberty.h usability... no
checking libiberty.h presence... no
checking for libiberty.h... no
checking for cplus_demangle in -liberty... no Solution: Install binutils-devel rpm 

$ make
$ make install

Profiling JIT
For JIT profiling, the following option needs to be added to the Java command-line:

-agentpath:<libdir>/libjvmti_oprofile.so

OR

-agentlib:jvmti_oprofile

Also, add <oprof_install_dir>/lib/oprofile to LD_LIBRARY_PATH

[See Ref]   

Collection

$ opcontrol --reset
$ opcontrol --vmlinux=/usr/lib/debug/lib/modules/`uname -r`/vmlinux
$ opcontrol --session-dir=$DIR
$ opcontrol --start
$ opcontrol --stop
$ opcontrol --dump
$ opcontrol --save <session-name>
$ opcontrol --shutdown

Report

$ opreport session:<session-name>
$ opreport session:<session-name> --image-path /lib/modules/`uname -r`/kernel [--details | -d]
$ opreport session:<session-name> --image-path /lib/modules/`uname -r`/kernel [--symbols | -l]
$ opreport session:<session-name> [--callgraph | -c]

References

• oprofile.sourceforge.net [JVMTI]