Wireshark is a real good GUI packet capture and analysis tool. I have a portable version on a thumbdrive that I occasionally use to diagnose problems on AIX servers.

First, capture the data with iptrace. Refer to the iptrace man page for options. For example, to capture rsh (port 514) from hosta and write it to /tmp run:

startsrc -s iptrace -a "-s hosta -p 514 /tmp/hosta.out"

After your done capturing your data, run:

stopsrc –s iptrace

Then download the output file to your desktop and open the resulting file with Wireshark. Wireshark will automatically determine that the file is from AIX iptrace.

This process uses the savevg command to backup the VG structure information. I used to to this by first excluding any filesystems from the backup by populating the /etc/exclude.vg files with all the filesystems, but IBM introducted the “-r” flag that does the same thing.  We purposely exclude rootvg from this because all the rootvg filesysetems will be in the mksysb backup.

#!/bin/perl
#
# Author: Patrick Vaughan - http://patrickv.info
#
# Purpose:
# Saves VG structure to files for easy recovery
#
# Restore with:
# restvg -f /vgdata/savevg.VGNAME DISKNAME
#
$test=1;
$DEBUG=0;
$" = "";
$GREP = "/usr/bin/grep";
$LSVG = "/usr/sbin/lsvg";
$EXCLUDE_VGS = "rootvg"; # VGs to exclude in a REGEXP
$VGDATA_DIR = "/vgdata";
if ( ! -d "$VGDATA_DIR" ) {
         mkdir("$VGDATA_DIR") || die("Could not create directory $VGDATA_DIR: $@ \n");
}
@VGS = grep(!/$EXCLUDE_VGS/, `$LSVG -o`);
for ( @VGS ) {
         chomp;
         $vg = $_;
#       exclude files are no longer necessary since the -r flag was introduced
#       if ( -f "/etc/exclude.$vg" ) {
#               unlink("/etc/exclude.$vg") || die("Could not remove file /etc/exclude.$vg: $@\n");
#       }
#       (@LVs) = grep(/^\//, split(/\s+/, `$LSVG -l $vg`, -1 ) );
#       open(EXCLUDE_VG, ">/etc/exclude.$vg") or die("Could not create file /etc/exclude.$vg: $@\n");
#       for (@LVs) {
#               # populate /etc/exclude.vg with ^./filesystem/ because we don't want the data
#               print EXCLUDE_VG ("^.$_/\n");
#       }
#       close(EXCLUDE_VG);
#       undef(@LVs);

         print "savevg of $vg to $VGDATA_DIR/savevg.$vg starting.\n";
         system(`savevg -irf $VGDATA_DIR/savevg.$vg $vg`);
         print "savevg of $vg to $VGDATA_DIR/savevg.$vg complete.\n";

# We're done with the exclude files, lets delete them
#       unlink("/etc/exclude.$vg") || die("Could not remove file /etc/exclude.$vg: $@\n");
}

This will create one file in the directory /vgdata for each volume group on the system.  You can then use the restvg -f /vgdata/savevg.xxx to restore the VG data.  Check the restvg man page for more information.

To get information about the backup run:

restvg -l -f savevg.X
VOLUME GROUP:           sharedGvg
BACKUP DATE/TIME:       Tue May 29 09:33:56 EDT 2007
UNAME INFO:             AIX cnpg 3 5 002680EF4C00
BACKUP OSLEVEL:         5.3.0.50
MAINTENANCE LEVEL:      5300-05
BACKUP SIZE (MB):       15008
SHRINK SIZE (MB):       1727
VG DATA ONLY:           no
sharedGvg:
LV NAME             TYPE       LPs   PPs   PVs  LV STATE      MOUNT POINT
shareGlv            jfs2       938   938   1    open/syncd    /shared

You can see the VG size and LV info to make decisions on restoring the VG.

More than one VG can be saved on a mksysb tape, which makes restoring things like your TSM or NIM master servers much easier.   We’ll need to use the rmt tape devices that do not rewind after their operation (rmtX.1).   First, rewind the tape (just to be sure we’re at the beginning) and do the mksysb as normal:

/usr/bin/tctl -f/dev/rmt0 rewind
/usr/bin/mksysb -p -v /dev/rmt0.1

Then just append your VGs on the end of the tape with savevg:

/usr/bin/savevg -p -v -f/dev/rmt0.1 vg1
/usr/bin/savevg -p -v -f/dev/rmt0.1 vg2
/usr/bin/tctl -f/dev/rmt0 rewind

To restore, just do the mksysb restore as normal, then do:

/usr/bin/tctl -f/dev/rmt0.1 rewind
/usr/bin/tctl -f/dev/rmt0.1 fsf 4
/usr/bin/restvg -f/dev/rmt0.1 hdiskX

The “fsf 4” will fast forward the tape to the first saved VG after the mksysb.  To restore the 2nd saved VG, use “fsf 5”.

If you need to restore individual files, you can do it like this:

/usr/bin/tctl -f/dev/rmt0 rewind
restore -x -d -v -s4 -f/dev/rmt0.1 ./[path]/[file]

“-s4” is rootvg, use “-s5” for next VG. The files are restored in your current directory.

Using this you can make a mksysb tape with your OS, your TSM application VGs, and your NIM VGs all on one tape.  And, if you save the VG structure into your rootvg, you can restore your extra filesystems as well.  Then you just have to restore the files from TSM.  I wouldn’t try to backup the TSM log, archivelog, or DB this way, but those can be created with your TSM DB backup.  Just be sure to do a “backup devconfig” and “backup volhist” regularly.

I muddled through my first NIM install, and got it working okay. But, I’ve since heard Steve Knudson several times deliver his NIM sessions? and that has really solidified the concepts better than trying to read the manuals. You can see a replay of Steve’s presentation at the AIX Virtual User Group, which I highly recommend.

I recently had to build a new NIM server, so I thought it was a good time to post my cliff notes version.

Install these on the NIM Master:

• bos.net.tcp.server
• bos.net.nfs.server
• bos.sysmgt.nim.master
• bos.sysmgt.nim.spot

Make these directories:

• /export
• /export/61 ( or 53, or something representative of your AIX version)

Make these filesystems:

• /export/61/lpp_6100-01-07
• /export/61/spot_6100-01-07
• /export/mksysb

The lpp filesystem will be the lppsource for AIX 6.1 TL1 SP7, and the spot filesystem will be the coresponding spot. For AIX 6.1, the lppsource is about 4G and the spot is about 600M.  The mksysb directory is good for customized base install images and mksysb images.

Initialize the NIM master:

smitty nim

Configure the NIM Environment / Advanced Configuration / Initialize the NIM Master Only

Network name:  something like 192_168_1

Primary Network Install Interface: enX

I also let the clients register themselves, it’s just easier that way.

Create the lppsource:

Insert DVD 1

smitty bffcreate

input device: /dev/cd0

directory for storing software: /export/61/lpp_6100-01-07

extend filesystems: yes

process multiple volumes: yes

You’ll be prompted to change DVDs.

Define to lppsource to NIM:

smitty nim

Configure the NIM Environment / Advanced Configuration / Create Basic Installation Resources /  Create a New LPP_SOURCE Only

Resource SERVER: master

LPP_SOURCE Name: lpp_6100-01-07

LPP_SOURCE Directory: /export/61/lpp_6100-01-07

Create new filesystem for LPP_SOURCE: no

Make the SPOT:

smitty nim

Configure the NIM Environment / Advanced Configuration / Create Basic Installation Resources /  Create a New SPOT Only

Resource SERVER: master

Input device: hit F4 and select your lppsource

SPOT Name: spot_6100-01-07

SPOT Directory: /export/61

Create new filesystem for SPOT: no

NIM is great for backing up systems and pushing out packages, but there isn’t a included system for managing mksysb images.  I’ve got a little script that runs on the NIM server and either does a mksysb on all the clients registered or you can specify certain clients.  It also keeps X number of old mksysb images and purges the remaining ones.  It’s needs a little cosmetic work, but you can use it as a basis for your own system.

get_mksysb

Now, since you have NIM working, you really should run a mksysb backup to tape so you have bootable media to get your NIM server back up.  I have two of them, which happen to also be my TSM servers, that I also backup to eachother.  Since you can’t have your NIM master be a client on another NIM server, I have a little script that generates a mksysb on disk (plus any extra VGs) and uses scp to copy the images to the other server. It’s stupid simple, and could use come cosmetics, but it works:

#!/bin/ksh
# Usage: mksysb.ksh dest_hostname [ extra_vg ... ]

hostname=`hostname`
dest_host=$1

shift

scp_file() {
        /usr/bin/scp $1 $dest_host:/export/mksysb/$hostname/

}


/usr/bin/mksysb -e -i -X /shared/mksysb.$hostname

if [ $? -eq 0 ]
then
        scp_file /shared/mksysb.$hostname 
else
        echo "mksysb backup failed!"
        exit 1
fi

if [ $? -eq 0 ]
then
        rm /shared/mksysb.$hostname
fi

while [ "$1" != "" ]
do
        savevg -f /shared/$1.savevg $1
        if [ $? -eq 0 ]
        then
                scp_file /shared/$1.savevg
                rm /shared/$1.savevg
        else
                echo "savevg of $1 failed!"
                exit 1
        fi
        shift
done

If you need to restore this mksysb, you just have to register it as a NIM object, then register the client on the NIM master and do the restore as normal.

UPDATE – Chris Gibson pointed out that with AIX 6.1 you can use a loopback device like Linux.

I recently heard someone discuss how to handle ISO images on AIX. On Linux, you can mount the ISO image as a filesystem, which is pretty slick.

Starting in AIX 6.1, you can use the loopmount command to create a loopback device and mount an ISO directly:

loopmount -i cdrom.iso -o “-V cdrfs -o ro” -m /mnt

And, loopumount unmounts the ISO and removes the loopback device:

loopumount -i cdrom.iso -o "/mnt"

There are a couple of gotcha’s, but nothing unusual. An ISO can only be associated with one loopback device at a time, the mount is only readonly, and it’s not supported in WPARs.

But, AIX previous to 6.1 doesn’t support mounting ISO images directly. The recommendation was to mount the ISO as a virtual optical drive on the VIOS. But, I don’t use VIOS because it hasn’t gained uptake in our environment and with our vendors. You can always burn the image and mount the CD, but if your optical drive is tied up on another LPAR you may not be able to move the optical drive to the right LPAR.

There is another way to mount those images, but it’s not a one-step process. What you need to do is create a LV at least as big as your ISO image, then copy the ISO data to that LV, and then mount the LV:

mklv -y cdlv -s n rootvg 704M hdisk0
dd if=cdrom.iso of=/dev/cdlv
mount -v cdrfs -o ro /dev/cdlv /mnt