Installing Oracle VM for Sparc 2.2 (LDOM) and Configuration

I have installed Oracle VM Server for Sparc 2.2 on Sparc T4-1 server

Pre requisites:

supported Platforms:

OS: Solaris 10 8/11 or Solaris 11 SRU 7 (Support Repository Update)

For Sparc T4-1 required firmware patch:  148822-01

148822-01 Synopsis: Obsoleted by: 148822-02 FIRMWARE: SPARC T4-1 – Sun System Firmware 8.2.0.a
Date: May/22/2012

We have uptodate firmware i.e.: 148822-04

Synopsis: Obsoleted by: 148822-04 FIRMWARE: SPARC T4-1 – Sun System Firmware 8.2.1.b
Date: Aug/31/2012


  1. Download and install the package “”
  2. cd /var/tmp/ldom (Assuming copied .zip file in this folder)
  3. unzip
  4. cd OVM_Server_SPARC-2_2/Install
  5. ./install-ldm

Welcome to the Oracle VM Server for SPARC installer.

You are about to install the LDoms Manager package, SUNWldm, that will enable
you to create, destroy and control other domains on your system. You will
also be given the option of running the Oracle VM Server for SPARC Configuration
Assistant (ldmconfig) to setup the control domain and create guest domains.

If the Solaris Security Toolkit (SST) is installed, you will be prompted to
optionally harden your control domain.

Installing LDoms packages.
pkgadd -n -d “./Product” -a pkg_admin SUNWldm.v
Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.

Installation of <SUNWldm> was successful.
pkgadd -n -d “./Product” -a pkg_admin SUNWldmp2v
Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.

Installation of <SUNWldmp2v> was successful.

Oracle VM Server for SPARC Configuration

Once installed, you may configure your system for a basic LDoms
deployment.  If you select “y” for the following question, the Oracle
VM Server for SPARC Configuration Assistant (tty) will be launched
following a successful installation of the packages.

(You may launch the Configuration Assistant at a later time with the
command:  /usr/sbin/ldmconfig, or use the GUI Configuration Assistant
which is bundled in the Oracle VM Server for SPARC zip file – see
README.GUI for more details)

Select an option for configuration:

y) Yes, launch the Configuration Assistant after install
n) No thanks, I will configure the system manually later

Enter y or n [y]:n

Verify installed packages:

bash-3.2# pkginfo -l SUNWldmp2v
   PKGINST:  SUNWldmp2v
      NAME:  LDoms P2V tool
  CATEGORY:  application
      ARCH:  sparc
   BASEDIR:  /
    VENDOR:  Oracle Corporation
      DESC:  Oracle VM Server for SPARC Physical-to-Virtual (P2V) Migration Tool
    PSTAMP:  scapen-on10-020120419141017
  INSTDATE:  Dec 21 2012 15:20
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed
     FILES:       19 installed pathnames
                   6 shared pathnames
                  12 directories
                   3 executables
                 264 blocks used (approx)

bash-3.2# pkginfo -l SUNWldm
      NAME:  LDoms Manager software
  CATEGORY:  application
      ARCH:  sparc.sun4v
   BASEDIR:  /
    VENDOR:  Oracle Corporation
      DESC:  Oracle VM Server for SPARC – Virtualization for SPARC T-Series
    PSTAMP:  scapen-on10-020120419141012
  INSTDATE:  Dec 21 2012 15:20
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed
     FILES:       79 installed pathnames
                  18 shared pathnames
                  23 directories
                  22 executables
                5851 blocks used (approx)

Verify the ldms

bash-3.2# svcs|grep ldmd
online         15:20:55 svc:/ldoms/ldmd:default

bash-3.2# ldm list
primary          active     -n-c–  UART    64    32256M   0.2%  1d 4h 38m

Save and Restore ldm configuration:

bash-3.2# tar -cvpf /var/tmp/autosave.tar var/opt/SUNWldm/autosave-*
a var/opt/SUNWldm/autosave-_Default_SP_Config/ 0K
a var/opt/SUNWldm/autosave-_Default_SP_Config/md-primary.bin 61K
a var/opt/SUNWldm/autosave-_Default_SP_Config/hv-md.bin 22K
a var/opt/SUNWldm/autosave-_Default_SP_Config/timestamp 1K
a var/opt/SUNWldm/autosave-_Default_SP_Config/sp-pri.bin 88K
a var/opt/SUNWldm/autosave-_Default_SP_Config/sp-timestamp 1K

to restore

cd /

tar -xvpf /var/tmp/autosave.tar

Saving and Restorig LDOM constraint Database
backup and restore the xml file: /var/opt/SUNWldm/ldom-db.xml

create default services after installing the LDOM manager:

3 defaults services need to be created on primary ldom:

virtual control concentrator service (vcc) service for use by the virtual network terminal server
daemon (vntsd) and as a concentrator for all logical domain consoles.

virtual disk server (vds) to allowimporting virtual disks into a logical domain

virtual switch service (vsw) to enable networking between virtual network (vnet)
devices in logical domains

VCC # ldm add-vcc port-range=5000-5100 vcc_primary primary
VDS # ldm add-vds vds0_primary primary
VSW # ldm add-vsw net-dev=nxge0 vsw0_primary primary

MAC address to vsw can be manually set using below command: only on Solaris 10:

primary# ldm add-vsw mac-addr=2:04:4f:fb:9f:0d net-dev=nxge0 primary-vsw0 primary

list services:
#ldm list-services primary
    NAME             LDOM             PORT-RANGE
    vcc_primary primary          5000-5100

    NAME             LDOM             MAC               NET-DEV   ID   DEVICE     LINKPROP   DEFAULT-VLAN-ID PVID VID                  MTU   MODE   INTER-VNET-LINK
    vsw0_primary primary          00:14:4f:fb:af:0d nxge0     0    switch@0              1               1                         1500         on     

    NAME             LDOM             VOLUME         OPTIONS          MPGROUP        DEVICE
    vds0_primary primary
bash-3.2# ldm set-vcpu 8 primary
bash-3.2# ldm list-constraints primary

bash-3.2# ldm start-reconf primary
Initiating a delayed reconfiguration operation on the primary domain.
All configuration changes for other domains are disabled until the primary
domain reboots, at which time the new configuration for the primary domain
will also take effect.

bash-3.2# ldm set-memory 1G primary
Notice: The primary domain is in the process of a delayed reconfiguration.
Any changes made to the primary domain will only take effect after it reboots.

bash-3.2# ldm add-config initial
bash-3.2# ldm list-config
initial [current]

bash-3.2# /usr/sbin/shutdown -y -g0 -i6

How to stop all domains:

1. power down all domains OS to OK prompt
2. primary# ldm stop-domain -a
3. primary# ldm unbind-domain ldom

 To remove all guest domains, run above 3 steps and below one step
4. ldm remove-domain -a

 How to upgrade LODM Manager:

1. svcadm disable ldmd
2. pkgrm SUNWldm
3. Instll the new package (copied to /var/tmp)
cd /var/tmp;pkgadd -Gd. SUNWldm.v
4. ldm list (verify)

 How to disable logical domain manager and remove the package?
1. svcadm disable ldmd
2. pkgrm SUNWldm; pkgrm SUNWldmp2v


Oracle®VMServer for SPARC 2.2 Administration Guide : E23807.pdf

Network interface mapping on Sparc T4-1

I have sparc T4-1 with onboard quad port and two PCIe quad port network cards installed on slot 0(zero) and slot 1(one)

Slot 0(zero) PCI-E network card:

Slot 0- Port0 – mapped to nxge0

Slot 0- Port1 – mapped to nxge1

Slot 0- Port2 – mapped to nxge2

Slot 0- Port3 – mapped to nxge3

Slot 1 (one) PCI-E network card:

Slot 1- Port0 – mapped to nxge4

Slot 1- Port1 – mapped to nxge5

Slot 1- Port2 – mapped to nxge6

Slot 1- Port3 – mapped to nxge7

Onboard quad port card:

Onboard – Port0 – mapped to igb0

Onboard- Port1 – mapped to igb1

Onboard- Port2 – mapped to igb2

Onboard- Port3 – mapped to igb3

Click on igb  and nxge for more details

igb network card:  Intel 82575 1Gb PCI Express NIC Driver

nxge:  Sun 10/1 Gigabit Ethernet network driver


Reset Root password in Solaris 10

{0} ok boot -F failsafe
Boot device: /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@wx000cx00xxxxxfdx,0:a  File and args: -F failsafe
SunOS Release 5.10 Version Generic_147440-01 64-bit
Copyright (c) 1983, 2011, Oracle and/or its affiliates. All rights reserved.
Configuring devices.
Searching for installed OS instances…
ROOT/zfsroot was found on root.
Do you wish to have it mounted read-write on /a? [y,n,?] y
mounting root on /a

An out of sync boot archive was detected on root.
The boot archive is a cache of files used during boot and
should be kept in sync to ensure proper system operation.

Do you wish to automatically update this boot archive? [y,n,?] y
Updating boot archive on root.
The boot archive on root was updated successfully.

# cd /a
# cp etc/shadow etc/shadow.21dec2012
# vi etc/shadow
# init 6

All about OBP in Sparc Servers

Frequently used commands in OBP:

#1 ok setenv auto-boot? false

to verify use “ok printenv auto-boot?

When you need trouble shoot solaris server at OK prompt, you have to disable the auto boot feature

#2 ok banner
Sun Ultra 5/10 UPA/PCI (UltraSPARC-IIi 333MHz), No Keyboard
OpenBoot 3.15, 128 MB memory installed, Serial #number.
Ethernet address number, Host ID: number.

You can find OBP version, Server model, Serial number, Host ID and Ether net mac address from Banner command

Note: Mac address displayed is Port 0 (zero) of on board network card. For Example I have quad port onboard NIC card and two Quad Port PCI-e Cards. Out of total 12 mac address, one displayed here is from onboard zero port. This clarity will help whe you are troubleshooting “ok boot net – install” trouble shoot errors.

#3 ok setenv local-mac-address? true

When you configure IPMP you have to set local-mac-address? to True, then only each NIC card use it’s Mac address which is very important for IPMP.

#4 set device alias and enable nvram

ok nvalias <alias> <device-path>

ok nvunalias <alias> <device-path>


{0} ok nvalias bootdisk /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@w5xxxc500xxxxxxfd5,0:a

{0} ok setenv use-nvramrc? true

{0} ok setenv boot-device bootdisk

These two are required to set alis names to the disk and network devices. As we have many disks in system, this will help to set alias as bootdisk and mirrordisk to identify the boot disk and mirror disk  from OK promt among many disk devices.

Note: After using nvunalias command if you refer devalias command it will still show the alias details, when you run reset-all, then it will not show them.

You can do this from OS using eeprom

eeprom nvramrc=”devalias bootdisk /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@wx000cx00xxxxxfd5,0:a devalias mirrordisk /pci@400/pci@2/pci@0/pci@4/scsi@0/disk@wx000cx00xxxxxxxx,0:a

eeprom use-nvramrc?=true

#5 Verify network interface connectivity and check MAC address of the NIC ports

{2} ok watch-net-all


/pci@1f,700000/network@1: link down


/pci@1d,700000/pci@2/pci@4/network@2: 100 Mbps half duplex link up

Watch ethernet packets

‘.’ is a good packet and ‘X’ is a bad packet

Press any key to stop

using network device alias

ok> cd /net1

ok> .properties

Using Network device physical path

or cd /pci@1d,700000/pci@2/pci@4/network@2

ok> .properties

 #6 ok show-devs

will list all the devices

#7 ok setenv diag-switch? true

ok setenv diag-level max (available diag levels are: net/max/min/menus)

Other test options available at OK prompt are
probe-scsi (use show-devs for PCI devices)

probe-scsi-all (use show-devs for PCI devices)
test net (tests primary network controller)

test-all (tests all devices)

watch-net and watch-net-all (for all interface)

#8 boot options

ok boot  (boots with default device mentioned in “boot-device”)

ok boot -s (boot to single user mode for maintenance)

ok boot net – install (used to boot through jumpstart boot servers for building OS through jumpstart)

ok boot net:speed=100,duplex=full,dhcp – install (Jumpstart boot through dhcp with preferred network settings instead of rarp , this removes the limitation of both boot servers and client in the same LAN)

ok boot -a (interactive boot, used to supply alternate boot files like /etc/system if default one is corrupted)

ok boot -r (reconfigure reboot, new device added to the sytem, this will create device files in OS)

ok boot disk2 kernel/sparcv9/unix -asv ( Boot single user, interactive, 64-bit and from disk device disk2. I never used this)

ok boot -F failsafe (I beleive it’s only for solaris 10 and above with SMF implementation this maintenance mode option available and saved Sys Admins from searching for boot CD or troubleshooting jumpstart to boot through net to go to maintenance mode for resetting root password or doing fsck on boot file systems)

boot -v or boot -m verbose (to see more messages on boot)

ok boot -m milestone=none (This command instructs the svc.startd daemon to temporarily disable all services and start sulogin on the console)

Reference web links about OBP:

Solaris LDOMs virtualization : setup guide

Sun Logical Domains or LDoms is a full virtual machine that runs an independent operating system instance and contains virtualized CPU, memory, storage, console, and cryptographic devices. This technology allows you to allocate a system resources into logical groupings and create multiple, discrete systems, each with their own operating system, resources, and identity within a single computer system. We can run a variety of applications software in different logical domains and keep them independent of performance and security purposes. The LDoms environment can help to achieve greater resource usage, better scaling, and increased security and isolation. more..

Deployment and understanding of LDOMS (Logical Domains)

Ldom technology allows us to allocate a system’s various resources such as memory, CPUs and devices (I/O,Networl interfaces etc) into logical groupings and create multiple, discrete systems each having their own operating system, resources, and identity within a single computer system. more