Firmware upgrade on T4-1 Sparc server

disclaimer: Please refer vendor provided instructions and follow carefully.

SUMMARY:

  1. cd /tmp/patch/
  2. unzip 148822-05.zip
  3. cd 148822-05
  4. ./sysfwdownload Sun_System_Firmware-8_2_2_c-SPARC_T4-1.pkg
  5. /usr/sbin/shutdown -i0 -g0 -yOK> type #. to go to ILOM mode in console
  6. show /HOST/boodmode (note config parameter value for LDOM servers to set after upgrade)
  7. show /HOST/ (will display the current versions of Hypervisor, OpenBoot, Post and  sysfw_version)
  8. stop /SYS
  9. set /SYS keyswitch_state=Normal
  10. show /SP/firmware/localimage (This will display upload date and version, if by mistake wrong image uploaded, it will display invalid image)
  11. load -source /SP/firmware/localimage (show progress in … and will reboot ILOM on completion)
  12. set /HOST/bootmode=”config-name”” (This is important for the LDOM servers)
  13. start the system “start /SYS”
  14. console /HOST/console….

Detailed explanation, have patience and read below steps

Step: 1

Go to this Oracle site and find the suitable firmware level for your server or refer your internal engineering documents to know upgrade level.

http://www.oracle.com/technetwork/systems/patches/firmware/release-history-jsp-138416.html

I need upgrade to SysFW 8.2.2.c, this comes with Oracle Patch “148822-05

Step2: Login to oracle site with valid support id and download the patch. (Most of the enterprise organizations have engineering team, who test these patches and upload to internal repositories)

Step3: copy the “148822-05.zip” to the servers, where you need upgrade the firmware and unzip it.

Step4: Pre tasks

  • This firmware upgrade require down time. If you are planning on Production/Live servers, take proper down time from application teams.
  • Firmware upgrade require console, hence check your console access.
  • Take all LDOM guest configuration backup. Refer my previous article on saving and restoring Ldom guest configuration.
  • Check current configuration stored in Service Processor(SP). When you upgrade the firmware, it will reset SP bootmode configuration to default-factory. If you save the config name, it will be easy for you to reset. You can also check this from ILOM using “show /HOST/bootmode” save the config property value.
Step5: Load the firmware to System Processor (SP). ILOM has the mechanism to copy the image from OS using sysfwdownload tool. This tool always comes with your firmware patch and it’s recommended to use the same tool.

cd /tmp/patch/148822-05

./sysfwdownload [image].pkg (in my case the image name “Sun_System_Firmware-8_2_2_c-SPARC_T4-1.pkg

(if you observer image file name has all information like this firmware is for Sun Sparc hardware,it’s version is 8.2.2.c and it’s for Sparc T4-1 model. This is first level of defense to avoid loading wrong firmware image)

When downloading it will show progress in format “….(10%)….” it ends with download completed successfully

shutdown -i0 -g0 -y

OK> type #. to go to ILOM mode in console

show /HOST/boodmode (note config parameter value for LDOM servers)

show /HOST/ will display the current versions of Hypervisor, OpenBoot, Post and  sysfw_version

(sysfw_version is the one we mentioned above)

Verify downloaded image is correct or not with “show /SP/firmware/localimage” and verify upload date and Version. (Here second level of defense comes, if valid image is not uploaded instead of version it will show invalid image).

stop the system to poweroff competely “stop /SYS”

Verify the key_switch state “show /SYS keyswitch_state” it should be in normal state. If it’s in locked state change to normal

“set /SYS keyswitch_state=Normal” Check if value set properly.

After completing all pre checks now we are ready to upgrade the firmware

  load -source /SP/firmware/localimage

This will show progress in … and once it complete, it will reboot the ILOM.

Once it’s completed, change the bootmode “set /HOST/bootmode=”config-name”” (This is important for the LDOM servers)

start the system “start /SYS”

go to console to see the server boot progress

console /HOST/console

Advertisements

Rename a guest domain (Solaris LDOM)

Disclaimer: I test following procedure to rename the guest domain. It worked for me, but it’s may not be the right procedure. Do not try on your prod environment

Rename the Guest domain:
==========================
Bring down the guest domain OS to OK prompt

bash-3.2# ldm list old-ldg1
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
old-ldg1 active     -t—-  5000    16    8G       6.2%  23h 17m

#ldm stop-domain old-ldg1
LDom old-ldg1 stopped

bash-3.2# ldm list old-ldg1
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
old-ldg1 bound      ——  5000    16    8G

Then run unbind command, it will change to inactive state

bash-3.2# ldm unbind old-ldg1
bash-3.2# ldm list old-ldg1
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
old-ldg1 inactive   ——          16    8G

Save the guest domain configuration to a xml

bash-3.2# ldm list-constraints -x old-ldg1 > /var/tmp/old-ldg1.xml

Create copy of the xmls

cp -p /var/tmp/old-ldg1.xml /var/tmp/new-ldg1.xml

bash-3.2# grep -i old-ldg1 /var/tmp/old-ldg1.xml
<Content xsi:type=”ovf:VirtualSystem_Type” ovf:id=”old-ldg1″>
<gprop:GenericProperty key=”vol_name”>old-ldg1_sys</gprop:GenericProperty>
<gprop:GenericProperty key=”vol_name”>old-ldg1_sys</gprop:GenericProperty>
<gprop:GenericProperty key=”block_dev”>/dev/zvol/dsk/root/vdsk_old-ldg1_sys</gprop:GenericProperty>
bash-3.2# cp /var/tmp/old-ldg1.xml /var/tmp/new-ldg1.xml
bash-3.2#perl -pi -e ‘s/old-ldg1/new-ldg1/g’ /var/tmp/new-ldg1.xml
bash-3.2# grep -i new-ldg1 /var/tmp/new-ldg1.xml
<Content xsi:type=”ovf:VirtualSystem_Type” ovf:id=”new-ldg1″>
<gprop:GenericProperty key=”vol_name”>new-ldg1_sys</gprop:GenericProperty>
<gprop:GenericProperty key=”vol_name”>new-ldg1_sys</gprop:GenericProperty>
<gprop:GenericProperty key=”block_dev”>/dev/zvol/dsk/root/vdsk_new-ldg1_sys</gprop:GenericProperty>

For ldm 3.x version please remove existing domain configuration to clear uuid. Thanks to Neeraj and Raphael for their comments

bash-3.2# ldm remove-domain old-ldg1

bash-3.2# ldm add-domain -i /var/tmp/new-ldg1.xml
bash-3.2# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  UART    8     1G       1.1%  1d 3h 24m
new-ldg1 inactive   ——          16    8G
bash-3.2# ldm bind new-ldg1
bash-3.2# ldm start new-ldg1
LDom new-ldg1 started
bash-3.2# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  UART    8     1G       4.8%  1d 3h 25m
new-ldg1 active     -t—-  5000    16    8G       3.5%  2s

Verify connecting to new guest domain console:

bash-3.2# telnet localhost 5000
Trying 127.0.0.1…
Connected to localhost.
Escape character is ‘^]’.

Connecting to console “new-ldg1” in group “new-ldg1” ….
Press ~? for control options ..

{0} ok
telnet> quit
Connection to localhost closed.
bash-3.2#

Backup guest domain configuration in xml and recreate the guest domain with xml file

This example is just to demonstrate how to save and restore guest domain.

Backup the xml file

#ldm list-constraints -x test-ldg1 > /var/tmp/test-ldg1.xml

bring down the guest domain OS -> stop the guest domain -> unbind the guest domain

# ldm list test-ldg1
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
test-ldg1 inactive   ——          16    8G

#ldm destroy test-ldg1

#ldm list (will not show test-ldg1 guest domain)

Impot the guest domain using backu xml file  /var/tmp/test-ldg1.xml

#ldm add-domain -i /var/tmp/test-ldg1.xml

Verify the domain details

# ldm list test-ldg1
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
test-ldg1 inactive   ——          16    8G

bind the guest domain -> start the domain -> start the OS on guest domain

#ldm add-domain -i /var/tmp/test-ldg1.xml

Status is inactive
test-ldg1 inactive   ——          16    8G

#ldm bind test-ldg1

Status is bound
test-ldg1 bound      ——  5000    16    8G

#ldm start test-ldg1
LDom test-ldg1 started

status is Active
test-ldg1 active     -t—-  5000    16    8G       3.8%  2s

Installing Oracle VM for Sparc 2.2 (LDOM) and Configuration

I have installed Oracle VM Server for Sparc 2.2 on Sparc T4-1 server

Pre requisites:

supported Platforms: http://docs.oracle.com/cd/E35434_01/html/E23810/gentextid-296.html#LDSRNldomssupportedplatforms

OS: Solaris 10 8/11 or Solaris 11 SRU 7 (Support Repository Update)

http://docs.oracle.com/cd/E35434_01/html/E23810/gentextid-296.html#LDSRNldomsrequiredsoftwarepatches

For Sparc T4-1 required firmware patch:  148822-01

148822-01 Synopsis: Obsoleted by: 148822-02 FIRMWARE: SPARC T4-1 – Sun System Firmware 8.2.0.a
Date: May/22/2012

We have uptodate firmware i.e.: 148822-04

Synopsis: Obsoleted by: 148822-04 FIRMWARE: SPARC T4-1 – Sun System Firmware 8.2.1.b
Date: Aug/31/2012

Installation:

  1. Download and install the package “p14054041_22_SOLARIS64.zip”
  2. cd /var/tmp/ldom (Assuming copied .zip file in this folder)
  3. unzip p14054041_22_SOLARIS64.zip
  4. cd OVM_Server_SPARC-2_2/Install
  5. ./install-ldm

Welcome to the Oracle VM Server for SPARC installer.

You are about to install the LDoms Manager package, SUNWldm, that will enable
you to create, destroy and control other domains on your system. You will
also be given the option of running the Oracle VM Server for SPARC Configuration
Assistant (ldmconfig) to setup the control domain and create guest domains.

If the Solaris Security Toolkit (SST) is installed, you will be prompted to
optionally harden your control domain.

Installing LDoms packages.
pkgadd -n -d “./Product” -a pkg_admin SUNWldm.v
Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.

Installation of <SUNWldm> was successful.
pkgadd -n -d “./Product” -a pkg_admin SUNWldmp2v
Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.

Installation of <SUNWldmp2v> was successful.

Oracle VM Server for SPARC Configuration

Once installed, you may configure your system for a basic LDoms
deployment.  If you select “y” for the following question, the Oracle
VM Server for SPARC Configuration Assistant (tty) will be launched
following a successful installation of the packages.

(You may launch the Configuration Assistant at a later time with the
command:  /usr/sbin/ldmconfig, or use the GUI Configuration Assistant
which is bundled in the Oracle VM Server for SPARC zip file – see
README.GUI for more details)

Select an option for configuration:

y) Yes, launch the Configuration Assistant after install
n) No thanks, I will configure the system manually later

Enter y or n [y]:n

Verify installed packages:

bash-3.2# pkginfo -l SUNWldmp2v
   PKGINST:  SUNWldmp2v
      NAME:  LDoms P2V tool
  CATEGORY:  application
      ARCH:  sparc
   VERSION:  2.2.0.0,REV=2012.04.19.14.10
   BASEDIR:  /
    VENDOR:  Oracle Corporation
      DESC:  Oracle VM Server for SPARC Physical-to-Virtual (P2V) Migration Tool
    PSTAMP:  scapen-on10-020120419141017
  INSTDATE:  Dec 21 2012 15:20
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed
     FILES:       19 installed pathnames
                   6 shared pathnames
                  12 directories
                   3 executables
                 264 blocks used (approx)

bash-3.2# pkginfo -l SUNWldm
   PKGINST:  SUNWldm
      NAME:  LDoms Manager software
  CATEGORY:  application
      ARCH:  sparc.sun4v
   VERSION:  2.2.0.0,REV=2012.04.19.14.10
   BASEDIR:  /
    VENDOR:  Oracle Corporation
      DESC:  Oracle VM Server for SPARC – Virtualization for SPARC T-Series
    PSTAMP:  scapen-on10-020120419141012
  INSTDATE:  Dec 21 2012 15:20
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed
     FILES:       79 installed pathnames
                  18 shared pathnames
                  23 directories
                  22 executables
                5851 blocks used (approx)

Verify the ldms

bash-3.2# svcs|grep ldmd
online         15:20:55 svc:/ldoms/ldmd:default

bash-3.2# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-c–  UART    64    32256M   0.2%  1d 4h 38m

Save and Restore ldm configuration:

bash-3.2# tar -cvpf /var/tmp/autosave.tar var/opt/SUNWldm/autosave-*
a var/opt/SUNWldm/autosave-_Default_SP_Config/ 0K
a var/opt/SUNWldm/autosave-_Default_SP_Config/md-primary.bin 61K
a var/opt/SUNWldm/autosave-_Default_SP_Config/hv-md.bin 22K
a var/opt/SUNWldm/autosave-_Default_SP_Config/timestamp 1K
a var/opt/SUNWldm/autosave-_Default_SP_Config/sp-pri.bin 88K
a var/opt/SUNWldm/autosave-_Default_SP_Config/sp-timestamp 1K

to restore

cd /

tar -xvpf /var/tmp/autosave.tar

Saving and Restorig LDOM constraint Database
backup and restore the xml file: /var/opt/SUNWldm/ldom-db.xml

create default services after installing the LDOM manager:

3 defaults services need to be created on primary ldom:

virtual control concentrator service (vcc) service for use by the virtual network terminal server
daemon (vntsd) and as a concentrator for all logical domain consoles.

virtual disk server (vds) to allowimporting virtual disks into a logical domain

virtual switch service (vsw) to enable networking between virtual network (vnet)
devices in logical domains

VCC # ldm add-vcc port-range=5000-5100 vcc_primary primary
VDS # ldm add-vds vds0_primary primary
VSW # ldm add-vsw net-dev=nxge0 vsw0_primary primary

MAC address to vsw can be manually set using below command: only on Solaris 10:

primary# ldm add-vsw mac-addr=2:04:4f:fb:9f:0d net-dev=nxge0 primary-vsw0 primary

list services:
#ldm list-services primary
VCC
    NAME             LDOM             PORT-RANGE
    vcc_primary primary          5000-5100

VSW
    NAME             LDOM             MAC               NET-DEV   ID   DEVICE     LINKPROP   DEFAULT-VLAN-ID PVID VID                  MTU   MODE   INTER-VNET-LINK
    vsw0_primary primary          00:14:4f:fb:af:0d nxge0     0    switch@0              1               1                         1500         on     

VDS
    NAME             LDOM             VOLUME         OPTIONS          MPGROUP        DEVICE
    vds0_primary primary
bash-3.2# ldm set-vcpu 8 primary
bash-3.2# ldm list-constraints primary

bash-3.2# ldm start-reconf primary
Initiating a delayed reconfiguration operation on the primary domain.
All configuration changes for other domains are disabled until the primary
domain reboots, at which time the new configuration for the primary domain
will also take effect.

bash-3.2# ldm set-memory 1G primary
——————————————————————————
Notice: The primary domain is in the process of a delayed reconfiguration.
Any changes made to the primary domain will only take effect after it reboots.
——————————————————————————
bash-3.2#

bash-3.2# ldm add-config initial
bash-3.2# ldm list-config
factory-default
initial [current]
bash-3.2#sync;sync

bash-3.2# /usr/sbin/shutdown -y -g0 -i6

How to stop all domains:

1. power down all domains OS to OK prompt
2. primary# ldm stop-domain -a
3. primary# ldm unbind-domain ldom

 To remove all guest domains, run above 3 steps and below one step
4. ldm remove-domain -a

 How to upgrade LODM Manager:

1. svcadm disable ldmd
2. pkgrm SUNWldm
3. Instll the new package (copied to /var/tmp)
cd /var/tmp;pkgadd -Gd. SUNWldm.v
4. ldm list (verify)

 How to disable logical domain manager and remove the package?
1. svcadm disable ldmd
2. pkgrm SUNWldm; pkgrm SUNWldmp2v

Documents:

Oracle®VMServer for SPARC 2.2 Administration Guide : E23807.pdf

Solaris LDOMs virtualization : setup guide

Sun Logical Domains or LDoms is a full virtual machine that runs an independent operating system instance and contains virtualized CPU, memory, storage, console, and cryptographic devices. This technology allows you to allocate a system resources into logical groupings and create multiple, discrete systems, each with their own operating system, resources, and identity within a single computer system. We can run a variety of applications software in different logical domains and keep them independent of performance and security purposes. The LDoms environment can help to achieve greater resource usage, better scaling, and increased security and isolation. more..

Deployment and understanding of LDOMS (Logical Domains)

Ldom technology allows us to allocate a system’s various resources such as memory, CPUs and devices (I/O,Networl interfaces etc) into logical groupings and create multiple, discrete systems each having their own operating system, resources, and identity within a single computer system. more