Rename a guest domain (Solaris LDOM)

Disclaimer: I test following procedure to rename the guest domain. It worked for me, but it’s may not be the right procedure. Do not try on your prod environment

Rename the Guest domain:
==========================
Bring down the guest domain OS to OK prompt

bash-3.2# ldm list old-ldg1
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
old-ldg1 active     -t—-  5000    16    8G       6.2%  23h 17m

#ldm stop-domain old-ldg1
LDom old-ldg1 stopped

bash-3.2# ldm list old-ldg1
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
old-ldg1 bound      ——  5000    16    8G

Then run unbind command, it will change to inactive state

bash-3.2# ldm unbind old-ldg1
bash-3.2# ldm list old-ldg1
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
old-ldg1 inactive   ——          16    8G

Save the guest domain configuration to a xml

bash-3.2# ldm list-constraints -x old-ldg1 > /var/tmp/old-ldg1.xml

Create copy of the xmls

cp -p /var/tmp/old-ldg1.xml /var/tmp/new-ldg1.xml

bash-3.2# grep -i old-ldg1 /var/tmp/old-ldg1.xml
<Content xsi:type=”ovf:VirtualSystem_Type” ovf:id=”old-ldg1″>
<gprop:GenericProperty key=”vol_name”>old-ldg1_sys</gprop:GenericProperty>
<gprop:GenericProperty key=”vol_name”>old-ldg1_sys</gprop:GenericProperty>
<gprop:GenericProperty key=”block_dev”>/dev/zvol/dsk/root/vdsk_old-ldg1_sys</gprop:GenericProperty>
bash-3.2# cp /var/tmp/old-ldg1.xml /var/tmp/new-ldg1.xml
bash-3.2#perl -pi -e ‘s/old-ldg1/new-ldg1/g’ /var/tmp/new-ldg1.xml
bash-3.2# grep -i new-ldg1 /var/tmp/new-ldg1.xml
<Content xsi:type=”ovf:VirtualSystem_Type” ovf:id=”new-ldg1″>
<gprop:GenericProperty key=”vol_name”>new-ldg1_sys</gprop:GenericProperty>
<gprop:GenericProperty key=”vol_name”>new-ldg1_sys</gprop:GenericProperty>
<gprop:GenericProperty key=”block_dev”>/dev/zvol/dsk/root/vdsk_new-ldg1_sys</gprop:GenericProperty>

For ldm 3.x version please remove existing domain configuration to clear uuid. Thanks to Neeraj and Raphael for their comments

bash-3.2# ldm remove-domain old-ldg1

bash-3.2# ldm add-domain -i /var/tmp/new-ldg1.xml
bash-3.2# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  UART    8     1G       1.1%  1d 3h 24m
new-ldg1 inactive   ——          16    8G
bash-3.2# ldm bind new-ldg1
bash-3.2# ldm start new-ldg1
LDom new-ldg1 started
bash-3.2# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-cv-  UART    8     1G       4.8%  1d 3h 25m
new-ldg1 active     -t—-  5000    16    8G       3.5%  2s

Verify connecting to new guest domain console:

bash-3.2# telnet localhost 5000
Trying 127.0.0.1…
Connected to localhost.
Escape character is ‘^]’.

Connecting to console “new-ldg1” in group “new-ldg1” ….
Press ~? for control options ..

{0} ok
telnet> quit
Connection to localhost closed.
bash-3.2#

Advertisements

Backup guest domain configuration in xml and recreate the guest domain with xml file

This example is just to demonstrate how to save and restore guest domain.

Backup the xml file

#ldm list-constraints -x test-ldg1 > /var/tmp/test-ldg1.xml

bring down the guest domain OS -> stop the guest domain -> unbind the guest domain

# ldm list test-ldg1
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
test-ldg1 inactive   ——          16    8G

#ldm destroy test-ldg1

#ldm list (will not show test-ldg1 guest domain)

Impot the guest domain using backu xml file  /var/tmp/test-ldg1.xml

#ldm add-domain -i /var/tmp/test-ldg1.xml

Verify the domain details

# ldm list test-ldg1
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
test-ldg1 inactive   ——          16    8G

bind the guest domain -> start the domain -> start the OS on guest domain

#ldm add-domain -i /var/tmp/test-ldg1.xml

Status is inactive
test-ldg1 inactive   ——          16    8G

#ldm bind test-ldg1

Status is bound
test-ldg1 bound      ——  5000    16    8G

#ldm start test-ldg1
LDom test-ldg1 started

status is Active
test-ldg1 active     -t—-  5000    16    8G       3.8%  2s

Installing Oracle VM for Sparc 2.2 (LDOM) and Configuration

I have installed Oracle VM Server for Sparc 2.2 on Sparc T4-1 server

Pre requisites:

supported Platforms: http://docs.oracle.com/cd/E35434_01/html/E23810/gentextid-296.html#LDSRNldomssupportedplatforms

OS: Solaris 10 8/11 or Solaris 11 SRU 7 (Support Repository Update)

http://docs.oracle.com/cd/E35434_01/html/E23810/gentextid-296.html#LDSRNldomsrequiredsoftwarepatches

For Sparc T4-1 required firmware patch:  148822-01

148822-01 Synopsis: Obsoleted by: 148822-02 FIRMWARE: SPARC T4-1 – Sun System Firmware 8.2.0.a
Date: May/22/2012

We have uptodate firmware i.e.: 148822-04

Synopsis: Obsoleted by: 148822-04 FIRMWARE: SPARC T4-1 – Sun System Firmware 8.2.1.b
Date: Aug/31/2012

Installation:

  1. Download and install the package “p14054041_22_SOLARIS64.zip”
  2. cd /var/tmp/ldom (Assuming copied .zip file in this folder)
  3. unzip p14054041_22_SOLARIS64.zip
  4. cd OVM_Server_SPARC-2_2/Install
  5. ./install-ldm

Welcome to the Oracle VM Server for SPARC installer.

You are about to install the LDoms Manager package, SUNWldm, that will enable
you to create, destroy and control other domains on your system. You will
also be given the option of running the Oracle VM Server for SPARC Configuration
Assistant (ldmconfig) to setup the control domain and create guest domains.

If the Solaris Security Toolkit (SST) is installed, you will be prompted to
optionally harden your control domain.

Installing LDoms packages.
pkgadd -n -d “./Product” -a pkg_admin SUNWldm.v
Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.

Installation of <SUNWldm> was successful.
pkgadd -n -d “./Product” -a pkg_admin SUNWldmp2v
Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.

Installation of <SUNWldmp2v> was successful.

Oracle VM Server for SPARC Configuration

Once installed, you may configure your system for a basic LDoms
deployment.  If you select “y” for the following question, the Oracle
VM Server for SPARC Configuration Assistant (tty) will be launched
following a successful installation of the packages.

(You may launch the Configuration Assistant at a later time with the
command:  /usr/sbin/ldmconfig, or use the GUI Configuration Assistant
which is bundled in the Oracle VM Server for SPARC zip file – see
README.GUI for more details)

Select an option for configuration:

y) Yes, launch the Configuration Assistant after install
n) No thanks, I will configure the system manually later

Enter y or n [y]:n

Verify installed packages:

bash-3.2# pkginfo -l SUNWldmp2v
   PKGINST:  SUNWldmp2v
      NAME:  LDoms P2V tool
  CATEGORY:  application
      ARCH:  sparc
   VERSION:  2.2.0.0,REV=2012.04.19.14.10
   BASEDIR:  /
    VENDOR:  Oracle Corporation
      DESC:  Oracle VM Server for SPARC Physical-to-Virtual (P2V) Migration Tool
    PSTAMP:  scapen-on10-020120419141017
  INSTDATE:  Dec 21 2012 15:20
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed
     FILES:       19 installed pathnames
                   6 shared pathnames
                  12 directories
                   3 executables
                 264 blocks used (approx)

bash-3.2# pkginfo -l SUNWldm
   PKGINST:  SUNWldm
      NAME:  LDoms Manager software
  CATEGORY:  application
      ARCH:  sparc.sun4v
   VERSION:  2.2.0.0,REV=2012.04.19.14.10
   BASEDIR:  /
    VENDOR:  Oracle Corporation
      DESC:  Oracle VM Server for SPARC – Virtualization for SPARC T-Series
    PSTAMP:  scapen-on10-020120419141012
  INSTDATE:  Dec 21 2012 15:20
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed
     FILES:       79 installed pathnames
                  18 shared pathnames
                  23 directories
                  22 executables
                5851 blocks used (approx)

Verify the ldms

bash-3.2# svcs|grep ldmd
online         15:20:55 svc:/ldoms/ldmd:default

bash-3.2# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-c–  UART    64    32256M   0.2%  1d 4h 38m

Save and Restore ldm configuration:

bash-3.2# tar -cvpf /var/tmp/autosave.tar var/opt/SUNWldm/autosave-*
a var/opt/SUNWldm/autosave-_Default_SP_Config/ 0K
a var/opt/SUNWldm/autosave-_Default_SP_Config/md-primary.bin 61K
a var/opt/SUNWldm/autosave-_Default_SP_Config/hv-md.bin 22K
a var/opt/SUNWldm/autosave-_Default_SP_Config/timestamp 1K
a var/opt/SUNWldm/autosave-_Default_SP_Config/sp-pri.bin 88K
a var/opt/SUNWldm/autosave-_Default_SP_Config/sp-timestamp 1K

to restore

cd /

tar -xvpf /var/tmp/autosave.tar

Saving and Restorig LDOM constraint Database
backup and restore the xml file: /var/opt/SUNWldm/ldom-db.xml

create default services after installing the LDOM manager:

3 defaults services need to be created on primary ldom:

virtual control concentrator service (vcc) service for use by the virtual network terminal server
daemon (vntsd) and as a concentrator for all logical domain consoles.

virtual disk server (vds) to allowimporting virtual disks into a logical domain

virtual switch service (vsw) to enable networking between virtual network (vnet)
devices in logical domains

VCC # ldm add-vcc port-range=5000-5100 vcc_primary primary
VDS # ldm add-vds vds0_primary primary
VSW # ldm add-vsw net-dev=nxge0 vsw0_primary primary

MAC address to vsw can be manually set using below command: only on Solaris 10:

primary# ldm add-vsw mac-addr=2:04:4f:fb:9f:0d net-dev=nxge0 primary-vsw0 primary

list services:
#ldm list-services primary
VCC
    NAME             LDOM             PORT-RANGE
    vcc_primary primary          5000-5100

VSW
    NAME             LDOM             MAC               NET-DEV   ID   DEVICE     LINKPROP   DEFAULT-VLAN-ID PVID VID                  MTU   MODE   INTER-VNET-LINK
    vsw0_primary primary          00:14:4f:fb:af:0d nxge0     0    switch@0              1               1                         1500         on     

VDS
    NAME             LDOM             VOLUME         OPTIONS          MPGROUP        DEVICE
    vds0_primary primary
bash-3.2# ldm set-vcpu 8 primary
bash-3.2# ldm list-constraints primary

bash-3.2# ldm start-reconf primary
Initiating a delayed reconfiguration operation on the primary domain.
All configuration changes for other domains are disabled until the primary
domain reboots, at which time the new configuration for the primary domain
will also take effect.

bash-3.2# ldm set-memory 1G primary
——————————————————————————
Notice: The primary domain is in the process of a delayed reconfiguration.
Any changes made to the primary domain will only take effect after it reboots.
——————————————————————————
bash-3.2#

bash-3.2# ldm add-config initial
bash-3.2# ldm list-config
factory-default
initial [current]
bash-3.2#sync;sync

bash-3.2# /usr/sbin/shutdown -y -g0 -i6

How to stop all domains:

1. power down all domains OS to OK prompt
2. primary# ldm stop-domain -a
3. primary# ldm unbind-domain ldom

 To remove all guest domains, run above 3 steps and below one step
4. ldm remove-domain -a

 How to upgrade LODM Manager:

1. svcadm disable ldmd
2. pkgrm SUNWldm
3. Instll the new package (copied to /var/tmp)
cd /var/tmp;pkgadd -Gd. SUNWldm.v
4. ldm list (verify)

 How to disable logical domain manager and remove the package?
1. svcadm disable ldmd
2. pkgrm SUNWldm; pkgrm SUNWldmp2v

Documents:

Oracle®VMServer for SPARC 2.2 Administration Guide : E23807.pdf

Network interface mapping on Sparc T4-1

I have sparc T4-1 with onboard quad port and two PCIe quad port network cards installed on slot 0(zero) and slot 1(one)

Slot 0(zero) PCI-E network card:

Slot 0- Port0 – mapped to nxge0

Slot 0- Port1 – mapped to nxge1

Slot 0- Port2 – mapped to nxge2

Slot 0- Port3 – mapped to nxge3

Slot 1 (one) PCI-E network card:

Slot 1- Port0 – mapped to nxge4

Slot 1- Port1 – mapped to nxge5

Slot 1- Port2 – mapped to nxge6

Slot 1- Port3 – mapped to nxge7

Onboard quad port card:

Onboard – Port0 – mapped to igb0

Onboard- Port1 – mapped to igb1

Onboard- Port2 – mapped to igb2

Onboard- Port3 – mapped to igb3

Click on igb  and nxge for more details

igb network card:  Intel 82575 1Gb PCI Express NIC Driver

nxge:  Sun 10/1 Gigabit Ethernet network driver

 

Reset Root password in Solaris 10

{0} ok boot -F failsafe
Boot device: /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@wx000cx00xxxxxfdx,0:a  File and args: -F failsafe
SunOS Release 5.10 Version Generic_147440-01 64-bit
Copyright (c) 1983, 2011, Oracle and/or its affiliates. All rights reserved.
Configuring devices.
Searching for installed OS instances…
ROOT/zfsroot was found on root.
Do you wish to have it mounted read-write on /a? [y,n,?] y
mounting root on /a

An out of sync boot archive was detected on root.
The boot archive is a cache of files used during boot and
should be kept in sync to ensure proper system operation.

Do you wish to automatically update this boot archive? [y,n,?] y
Updating boot archive on root.
The boot archive on root was updated successfully.

# cd /a
# cp etc/shadow etc/shadow.21dec2012
# vi etc/shadow
root::6445::::::
:wq!
# init 6

All about OBP in Sparc Servers

Frequently used commands in OBP:

#1 ok setenv auto-boot? false

to verify use “ok printenv auto-boot?

When you need trouble shoot solaris server at OK prompt, you have to disable the auto boot feature

#2 ok banner
Sun Ultra 5/10 UPA/PCI (UltraSPARC-IIi 333MHz), No Keyboard
OpenBoot 3.15, 128 MB memory installed, Serial #number.
Ethernet address number, Host ID: number.

You can find OBP version, Server model, Serial number, Host ID and Ether net mac address from Banner command

Note: Mac address displayed is Port 0 (zero) of on board network card. For Example I have quad port onboard NIC card and two Quad Port PCI-e Cards. Out of total 12 mac address, one displayed here is from onboard zero port. This clarity will help whe you are troubleshooting “ok boot net – install” trouble shoot errors.

#3 ok setenv local-mac-address? true

When you configure IPMP you have to set local-mac-address? to True, then only each NIC card use it’s Mac address which is very important for IPMP.

#4 set device alias and enable nvram

ok nvalias <alias> <device-path>

ok nvunalias <alias> <device-path>

Example:

{0} ok nvalias bootdisk /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@w5xxxc500xxxxxxfd5,0:a

{0} ok setenv use-nvramrc? true

{0} ok setenv boot-device bootdisk

These two are required to set alis names to the disk and network devices. As we have many disks in system, this will help to set alias as bootdisk and mirrordisk to identify the boot disk and mirror disk  from OK promt among many disk devices.

Note: After using nvunalias command if you refer devalias command it will still show the alias details, when you run reset-all, then it will not show them.

You can do this from OS using eeprom

eeprom nvramrc=”devalias bootdisk /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@wx000cx00xxxxxfd5,0:a devalias mirrordisk /pci@400/pci@2/pci@0/pci@4/scsi@0/disk@wx000cx00xxxxxxxx,0:a

eeprom use-nvramrc?=true

#5 Verify network interface connectivity and check MAC address of the NIC ports

{2} ok watch-net-all

/pci@1f,700000/network@1

/pci@1f,700000/network@1: link down

/pci@1d,700000/pci@2/pci@4/network@2

/pci@1d,700000/pci@2/pci@4/network@2: 100 Mbps half duplex link up

Watch ethernet packets

‘.’ is a good packet and ‘X’ is a bad packet

Press any key to stop

using network device alias

ok> cd /net1

ok> .properties

Using Network device physical path

or cd /pci@1d,700000/pci@2/pci@4/network@2

ok> .properties

 #6 ok show-devs

will list all the devices

#7 ok setenv diag-switch? true

ok setenv diag-level max (available diag levels are: net/max/min/menus)

Other test options available at OK prompt are
probe-scsi (use show-devs for PCI devices)

probe-scsi-all (use show-devs for PCI devices)
test net (tests primary network controller)

test-all (tests all devices)

watch-net and watch-net-all (for all interface)

#8 boot options

ok boot  (boots with default device mentioned in “boot-device”)

ok boot -s (boot to single user mode for maintenance)

ok boot net – install (used to boot through jumpstart boot servers for building OS through jumpstart)

ok boot net:speed=100,duplex=full,dhcp – install (Jumpstart boot through dhcp with preferred network settings instead of rarp , this removes the limitation of both boot servers and client in the same LAN)

ok boot -a (interactive boot, used to supply alternate boot files like /etc/system if default one is corrupted)

ok boot -r (reconfigure reboot, new device added to the sytem, this will create device files in OS)

ok boot disk2 kernel/sparcv9/unix -asv ( Boot single user, interactive, 64-bit and from disk device disk2. I never used this)

ok boot -F failsafe (I beleive it’s only for solaris 10 and above with SMF implementation this maintenance mode option available and saved Sys Admins from searching for boot CD or troubleshooting jumpstart to boot through net to go to maintenance mode for resetting root password or doing fsck on boot file systems)

boot -v or boot -m verbose (to see more messages on boot)

ok boot -m milestone=none (This command instructs the svc.startd daemon to temporarily disable all services and start sulogin on the console)

Reference web links about OBP:

http://it.toolbox.com/wiki/index.php/Open_Boot_PROM_-_OK_Prompt

http://docs.oracle.com/cd/E19963-01/html/821-1451/gkkvd.html

http://irtfweb.ifa.hawaii.edu/~spex/computers/techdocs/1201-hilodog/SunOBP_Quick_Ref.pdf

 http://www.adminschoice.com/booting-problems-in-solaris

http://www.shrubbery.net/~heas/sun-feh-2_1/General/OBP.html