Integrated Management Module (IMM) management on IBM x86 M3 server

1. Find IMM version using dmidecode in Linux

dmidecode|awk -F\- ‘/Integrated Management Module/ { gsub(/\[|]/,””);print $2 }’

2. Upgrade the IMM firmware version

download the .bin file  example yuooe3c version upgrade file name “ibm_fw_imm_yuooe3c-1.33_linux_32-64.bin” and execute as root login using -s option

./ibm_fw_imm_yuooe3c-1.33_linux_32-64.bin -s

Note: This will not reboot the server automatically.  New version can be identified through dmidecode only after reboot

3. use asu64 utility to mange IMM from command mode.

Usage: ./asu [apps] <cmds> [<cmd_mod> | <class>] [<options>] [<connect_opts>]

Commands:

 –license           rebootimm
        batch               rebootbmc
        comparedefault      rebootrsa
        delete              replicate
        deletecert          resetrsa
        dump                restore
        encrypt             save
        export              set
        generate            setenc
        help                show
        import              showdefault
        loaddefault         showgroups
        nodes               showlocation
        patchadd            showvalues
        patchextract        version
        patchlist           readraw
        patchremove         writeraw
        rebootimm

4. To check values, first dump all values to a  temp file and use the file for reference

/opt/ibm/toolscenter/asu/asu64 show all 2>/dev/null >/tmp/imm-dump.txt

grep IMM.NTPHost /tmp/imm-dump.txt
IMM.NTPHost=nn.nn.nn.nn

5. To modify individual property value:

 /opt/ibm/toolscenter/asu/asu64 set IMM.NTPHost <IP address>

6. To Modify multiple values, then create a temp file /tmp/imm-values.txt with values

set IMM.DNS_IP_Address1 “nn.nn.nn.nn”
set IMM.DNS_IP_Address2 “nn.nn.nn.nn”
set IMM.NTPHost “nn.nn.nn.nn”
set IMM.SMTP_ServerName “nn.nn.nn.nn”

 then input the temp file /tmp/imm-values.txt to asu64 tool

asu64 batch /tmp/imm-values.txt

As some values will reflect after imm reboot, reboot it using rebootimm

asu64 rebootimm

HP ILO3 upgrade from 1.26 to 1.28 on Proliant BL460c G7

/sbin/hponcfg -g  ==>this will get host info if ILO

Firmware Revision = 1.26 Device type = iLO 3 Driver name = hpilo
Host Information:
                        Server Name: xxxxxxx
                        Server Number:

 ./CP016462.scexe

FLASH_iLO3 v1.06 for Linux (Jan 17 2012)
(C) Copyright 2002-2011 Hewlett-Packard Development Company, L.P.
Firmware image: ilo3_128.bin
Current iLO 3 firmware version  1.26; Serial number ILOxxxx

Component XML file: CP016462.xml
CP016462.xml reports firmware version 1.28
This operation will update the firmware on the
iLO 3 in this server with version 1.28.
Continue (y/N)?y
Current firmware is 1.26 (Aug 26 2011 )
Firmware image is 0x800000(8388608) bytes
Committing to flash part…
******** DO NOT INTERRUPT! ********
Flashing completed.
Attempting to reset device.
Succeeded.
***** iLO 3 reboot in progress (may take up to 60 seconds.)
***** Please ignore console messages, if any.
iLO 3 reboot completed.

ILOM 3.0 Sparc Servers

First let’s start with how to access ILOM:

1. if NetMgmt port configured, using ssh you can login to the ILOM

2.If you configure SerialMgmt Port, then when you connect to serial port, you will access ILOM

3. if you connect to console thorught netmgmt or serial mgmt ports and you connect Operating System. Use #. (Hash symbol with . (dot)) which will take you to the ILOM.

From ILOM to go back to the Host Operating System then start the console service ->start /SP/console

-> start /SP/console  ==>ILOM CLI prompt
Are you sure you want to start /SP/console (y/n)? y

Serial console started.  To stop, type #.

bash-3.2#   –> OS prompt
Serial console stopped.

->  ==>ILOM CLI prompt

Notes on ILOM CLI follow hierarchial Architecture

ILOM Target types:

/SP – configuring ILOM service processor (SP)

/SYS – inventory/environmental and hardware management

/HOST – monitoring and managing host operating system.

Blade Platforms:

/CMM – Blade platform this replace SP, used to configure ILOM Chassis Monitoring Module

/CH – Blade platform this replace /SYS and provides inventory, environmentals and hardware management

ILOM CLI commands:

cd
create
delete
dump
exit
help
load
reset
set
show
start
stop
version

ILOM CLI Syntax:

command [options] [target] [properties

ex.,: set /SP/services/https port=80 servicestate=enabled

User Management ILOM CLI:

Roles in CLI: Admin|Operator (a), User Management (u), Console (c), Reset and Host control (r) and Read Only (o)

1. Add a local user

create /SP/users/user1 password=password  role=a|u|c|r|o|s

2.  Delete a local user

delete /SP/users/user1

 3. Change a local user’s properties

set /SP/users/user1  role=operator

 4. Display information about all local users

show -display [targets|properties|all ] -level all /SP/users

 5. Display information aboutLDAP settings

show /SP/clients/ldap

6. Change LDAP settings

set /SP/clients/ldap binddn=proxyuser bindpw=proxyuserpassword defaultrole=a|u|c|r|o|s address=ipaddress

 Set ILOM clock to sync with NTP server

set /SP/clients/ntp/server/1 address=ntpIPAddress

To configure IP on NetMgmt port set values to the /SP/network properties

Properties:
        commitpending = (Cannot show property)
        dhcp_server_ip = (none)
        ipaddress = (none)
        ipdiscovery = (none)
        ipgateway = (none)
        ipnetmask = (none)
        macaddress = xx:xxx:xx:xx:xx:xx
        managementport = /SYS/MB/SP/NETMGMT
        outofbandmacaddress = xx:xx:xx:xx:xx:xx
        pendingipaddress = (none) –>IPV4 address
        pendingipdiscovery = (none)
        pendingipgateway = (none)  –> Gateway
        pendingipnetmask = (none) –> Netmask
        pendingmanagementport = /SYS/MB/SP/NETMGMT
        sidebandmacaddress = xx:xx:xx:xx:xx:xx
        state = disabled

you can configure above values using set command

set /SP/network pendingipaddress=192.168.1.10

set /SP/network pendingipnetmask=255.255.255.0

set /SP/network pendingipgateway=192.168.1.1

Configuring SNMP and email alert:

set /SP/alertmgmt/rules/1…15 type=snmtrap snmp_version=3 comunity_or_username=username destination=ipaddress level=down|critical|major|minor

set /SP/alertmgmt/rules/1…15 type=email destination=somemail@abc.com level=down|critical|major|minor

To enable web access to ILOM, can http

set /SP/services/http port=80 secureredirect=enabled|disabled servicestate=enabled|disabled

set /SP/services/https port=xxx servicestate=enabled|disabled

HOST system commands (*warning: if your host is up and running, HOST commands can cause outage)

To start the host system ->start /SYS

To stop the host system -> stop /SYS

Force shutdown (required when host hung)->stop -f /SYS

Reset the host system  (required when host hung) ->reset /SYS

start a sessions to connect to host console -> /SP/console

stop the console session -> /SP/console

Filtering output options:

Users with admin roles -> show /SP/users -level all role==”a*”

SNMP trap alerts ->show /SP/alertmgmt -level all type==”snmptrap”

List services which are disabled -> show /SP/services -level all servicestate==disabled

List memory moduels with some part number -> show /SYS -level all type==”DIMM” fru_part_number==07014642

list all hard disks -> show /SYS -level all type==”Hard Disk”

How to backup the ILOM configuration

cd /SP/config

set passphrase=passphrase

set dump_uri=transfer_method://username:password@ipaddress_or_hostname/directorypath/filename

set dump_uri=scp://adminuser:userpswd@x.x.x.x/Backup/Lab9/SP-backup.config

How to Restore the ILOM configuration

Note: you have to login ILOM with user which has Admin,User Management, Console, Reset, Host control and Read only (a,u,c,r,o) roles.

cd /SP/config

set passphrase=passphrase

set load_uri=transfer_method://username:password@ipaddress_or_hostname/directorypath/filename

How to update ILOM Firmware

Pre requisite: Shutdown the host running on the hardware whose ILOM firmware you are upgrading

1.Login to ILOM CLI with user which has Admin role

2. ->version

3. Download the zip file packge and copy to TFTP server that is accessible from network

4.load -source <supported_protocol>://<server ip>/<path_to_firmware_image/<filename.xxx>

Do you want to preserve the configuration (y/n)?y

Once firmware loades it automaticall reboots to complete the firmware update

ILOM CLI is based on DMTF CLP (Distributed Management Task Force specification Server Management Command-Line Specification, Version 11.0a.8 Draft

Reference: http://www.fujitsu.com/downloads/SPARCE/manuals/sparc-t5e/ilom3.0-cli-en-01.pdf

http://www.dmtf.org/

extglob – expression power of rgular expressions in bash/ksh

extglob

In addition to the traditional globs (supported by all Bourne-family shells) that we’ve seen so far, Bash (and Korn Shell) offers extended globs, which have the expressive power of regular expressions. Korn shell enables these by default; in Bash, you must run the command

shopt -s extglob

in your shell (or at the start of your script — see note on parsing below) to use them. The pattern matching reference describes the syntax, which is reproduced here:

?(pattern-list)
Matches zero or one occurrence of the given patterns.
*(pattern-list)
Matches zero or more occurrences of the given patterns.
+(pattern-list)
Matches one or more occurrences of the given patterns.
@(pattern-list)
Matches one of the given patterns.
!(pattern-list)
Matches anything except one of the given patterns.

Patterns in a list are separated by | characters.

Extended globs allow you to solve a number of problems which otherwise require a rather surprising amount of ugly hacking; for example,

# To remove all the files except ones matching *.jpg:
rm !(*.jpg)
# All except *.jpg and *.gif and *.png:
rm !(*.jpg|*.gif|*.png)
# To copy all the MP3 songs except one to your device
cp !(04*).mp3 /mnt

To use an extglob in a parameter expansion (this can also be done in one BASH statement with read):

# To trim leading and trailing whitespace from a variable
x=${x##+([[:space:]])}; x=${x%%+([[:space:]])}

Extended glob patterns can be nested, too.

[[ $fruit = @(ba*(na)|a+(p)le) ]] && echo 'Nice fruit'

Reference:  http://mywiki.wooledge.org/glob

 

script

Requirement:

in below output (4th column) you can observe that some values are higher like 93 , 102 … 

i need a command/script which runs for every 30 min/1 hour and send mail  when ever it detects any process whose value is more thatn 50

db2fenc1 26869926 20775480   0 07:31:00      –  0:01 db2fmp (idle) 0
db2fenc1 35258592 20775480   0   Jan 04      –  0:00 db2fmp (idle) 0
db2fenc1 48824542 20775480  91   Jan 04      – 2390:16 db2fmp (70142) 0
db2fenc1 55509030 20775480 102   Jan 04      – 2429:06 db2fmp (70913) 0
db2fenc1  3408414 20775480  91   Jan 04      – 2396:36 db2fmp (111152) 0
db2fenc1  8651288 20775480  98   Jan 04      – 2432:14 db2fmp (69885) 0
db2fenc1 10683076 20775480  97   Jan 04      – 2401:23 db2fmp (60662) 0
db2fenc1 13238968 20775480   0 10:35:08      –  0:00 db2fmp (95115) 0
db2fenc1 17957440 20775480  85   Jan 04      – 2418:27 db2fmp (74500) 0
db2fenc1 18875132 51970054   0   Dec 15      –  0:01 db2fmp (C) 0
db2fenc1 19268132 20775480  87   Jan 04      – 2442:50 db2fmp (103174) 0
db2fenc1 19792618 20775480   0   Jan 04      –  0:02 db2fmp (idle) 0
db2fenc1 21693040 20775480   0   Jan 04      –  0:00 db2fmp (91774) 0
db2fenc1 23331344 20775480   0   Dec 31      –  0:04 db2fmp (C) 0
db2fenc1 23921376 51970054   0   Dec 15      –  0:00 db2fmp (idle) 0
db2fenc1 25166396 20775480 102   Jan 04      – 2442:36 db2fmp (102109) 0
db2fenc1 26673848 20775480 100   Jan 04      – 2440:56 db2fmp (65756) 0
db2inst1 28115564 66584822   0 15:46:57  pts/1  0:00 grep -i fenc
 
db2inst1@mygbppgodb201:/db2home/db2inst1> ps -ef|grep -i fenc
db2fenc1 22085670 37487042   0 02:55:06      –  0:00 db2fmp (idle) 0
db2inst1 24117472 11665860   0 13:48:02  pts/3  0:00 grep -i fenc
db2fenc1 42205314 37487042   0   05 Jan      –  0:00 db2fmp (idle) 0
db2fenc1  7602520 37487042   0   05 Jan      –  0:00 db2fmp (idle) 0
db2fenc1  8651198 37487042   0   05 Jan      –  0:00 db2fmp (88671) 0
db2fenc1 26214858 37487042   0 13:36:26      –  0:00 db2fmp (C) 0
db2fenc1 27787684 37487042   0 02:50:04      –  0:00 db2fmp (idle) 0
 
Solution:
Use the following line in cron to run for every 30 minutes
#!/usr/bin/bash
#script to parse ps -ef fields
TIMESTAMP=`date +%d%m%Y-%T`
SCRIPT=`basename $0`
LOGFILE=”/var/tmp/$SCRIPT.log”
TMPFILE=/var/tmp/$SCRIPT.$$
FILE=/var/tmp/SCRIPT.$TIMESTAMP
SUBJECT=”Process with above 50 at $TIMESTAMP”
EMAIL=”somemail@domain.com
ps -ef|grep -i fenc|awk ‘$4 > 50 {print $0}’ >$TMPFILE
if [ -s $TMPFILE ]; then
echo “/bin/mail -s “$SUBJECT” “$EMAIL” < $TMPFILE”
if [ $? == 0 ];then
echo “Failed to send email, copied file to $FILE” >> $LOGFILE
mv $TMPFILE $FILE
else
echo “Sent mail successfull to $EMAIL at $TIMESTAMP” >> $LOGFILE
rm $TMPFILE
fi
else
echo “$TIMESTAMP Good news no process with above 50 values” >> $LOGFILE
fi
#end of the script

Configuring Link Based IPMP Solaris 10

Basic requirements for IPMP

  1. All interfaces in an IPMP group must have uniqueMAC addresses
  2. All interfaces in an IPMP group must be of the same media type
  3. All interfaces in an IPMP group must be on the same IP link

Two types of IPMP:

  1. Link-Based FailureDetection
  2. Probe-Based FailureDetection

Terminology:

Data Addresses: Conventional address configured to interface.

Test Addresses: used by in.mpathd for probe based failure detection.

IP link:  Physical connection to  network switch

To configure link based IPMP, we do not required test address. So configure the two interface hostname files

Ex., nxge0 and nxge4

echo “myhost netmask + broadcast + group ipmp0 up” > /etc/hostname.nxge0

echo “group ipmp0 up” > /etc/hostname.nxge4

then restart the network services

svcadm disable network/physical

svcadm enable network/physical

Jan  7 18:26:52 myhost in.mpathd[1501]: No test address configured on interface nxge4; disabling ed failure detection on it

Jan  7 18:26:52 myhost in.mpathd[1501]: No test address configured on interface nxge0; disabling probe-based failure  on it

Then ifconfig output will look like below. Notice that nxge4 do not have  IP configured.

nxge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

        inet 192.168.0.100netmask ffffff80 broadcast 165.40.63.127

        groupname ipmp0

        ether 0:xx:xx:xx:xx:xx

nxge4: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4

        inet 0.0.0.0 netmask ff000000 broadcast 0.255.255.255

        groupname ipmp0

        ether 0:xx:xx:xx:xx:xx

Then verify the failover and back with ‘if_mpadm” command

root@myhost # if_mpadm -d nxge0

Jan  7 18:28:29 myhost in.mpathd[1501]: Successfully failed over from NIC nxge0 to NIC nxge4

root@myhost # if_mpadm -r nxge0

Jan  7 18:28:43 myhost in.mpathd[1501]: Successfully failed back to NIC nxge0

While testing ran a ping test

H:\>ping 192.168.0.100-t

Pinging 192.168.0.100with 32 bytes of data:

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=39ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Request timed out. –here tested failover to nxge4

Reply from 192.168.0.100: bytes=32 time=39ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Request timed out.—here tested failback to nxge0

Reply from 192.168.0.100: bytes=32 time=39ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Reply from 192.168.0.100: bytes=32 time=38ms TTL=244

Ping statistics for 192.168.0.100:

    Packets: Sent = 25, Received = 23, Lost = 2 (8% loss),

Approximate round trip times in milli-seconds:

    Minimum = 38ms, Maximum = 39ms, Average = 38ms

Control-C

Note: For trouble shooting you can test network connectivity from OK prompt with watch-net command

/pci@400/pci@2/pci@0/pci@6/network@0

1000 Mbps full duplex Link up

Looking for Ethernet Packets.

‘.’ is a Good Packet.  ‘X’ is a Bad Packet.

Type any key to stop.

………………………………………

/pci@400/pci@1/pci@0/pci@8/network@0

1000 Mbps link up

Looking for Ethernet Packets.

‘.’ is a Good Packet.

Type any key to stop.

…………………………………………

Reference

http://docs.oracle.com/cd/E23823_01/pdf/816-4554.pdf

http://sunaytripathi.wordpress.com/2010/03/25/solaris-10-networking-the-magic-revealed/#mozTocId342636.125

https://blogs.oracle.com/stw/entry/using_ipmp_with_link_based