Installing Oracle VM for Sparc 2.2 (LDOM) and Configuration

I have installed Oracle VM Server for Sparc 2.2 on Sparc T4-1 server

Pre requisites:

supported Platforms: http://docs.oracle.com/cd/E35434_01/html/E23810/gentextid-296.html#LDSRNldomssupportedplatforms

OS: Solaris 10 8/11 or Solaris 11 SRU 7 (Support Repository Update)

http://docs.oracle.com/cd/E35434_01/html/E23810/gentextid-296.html#LDSRNldomsrequiredsoftwarepatches

For Sparc T4-1 required firmware patch:  148822-01

148822-01 Synopsis: Obsoleted by: 148822-02 FIRMWARE: SPARC T4-1 – Sun System Firmware 8.2.0.a
Date: May/22/2012

We have uptodate firmware i.e.: 148822-04

Synopsis: Obsoleted by: 148822-04 FIRMWARE: SPARC T4-1 – Sun System Firmware 8.2.1.b
Date: Aug/31/2012

Installation:

  1. Download and install the package “p14054041_22_SOLARIS64.zip”
  2. cd /var/tmp/ldom (Assuming copied .zip file in this folder)
  3. unzip p14054041_22_SOLARIS64.zip
  4. cd OVM_Server_SPARC-2_2/Install
  5. ./install-ldm

Welcome to the Oracle VM Server for SPARC installer.

You are about to install the LDoms Manager package, SUNWldm, that will enable
you to create, destroy and control other domains on your system. You will
also be given the option of running the Oracle VM Server for SPARC Configuration
Assistant (ldmconfig) to setup the control domain and create guest domains.

If the Solaris Security Toolkit (SST) is installed, you will be prompted to
optionally harden your control domain.

Installing LDoms packages.
pkgadd -n -d “./Product” -a pkg_admin SUNWldm.v
Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.

Installation of <SUNWldm> was successful.
pkgadd -n -d “./Product” -a pkg_admin SUNWldmp2v
Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.

Installation of <SUNWldmp2v> was successful.

Oracle VM Server for SPARC Configuration

Once installed, you may configure your system for a basic LDoms
deployment.  If you select “y” for the following question, the Oracle
VM Server for SPARC Configuration Assistant (tty) will be launched
following a successful installation of the packages.

(You may launch the Configuration Assistant at a later time with the
command:  /usr/sbin/ldmconfig, or use the GUI Configuration Assistant
which is bundled in the Oracle VM Server for SPARC zip file – see
README.GUI for more details)

Select an option for configuration:

y) Yes, launch the Configuration Assistant after install
n) No thanks, I will configure the system manually later

Enter y or n [y]:n

Verify installed packages:

bash-3.2# pkginfo -l SUNWldmp2v
   PKGINST:  SUNWldmp2v
      NAME:  LDoms P2V tool
  CATEGORY:  application
      ARCH:  sparc
   VERSION:  2.2.0.0,REV=2012.04.19.14.10
   BASEDIR:  /
    VENDOR:  Oracle Corporation
      DESC:  Oracle VM Server for SPARC Physical-to-Virtual (P2V) Migration Tool
    PSTAMP:  scapen-on10-020120419141017
  INSTDATE:  Dec 21 2012 15:20
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed
     FILES:       19 installed pathnames
                   6 shared pathnames
                  12 directories
                   3 executables
                 264 blocks used (approx)

bash-3.2# pkginfo -l SUNWldm
   PKGINST:  SUNWldm
      NAME:  LDoms Manager software
  CATEGORY:  application
      ARCH:  sparc.sun4v
   VERSION:  2.2.0.0,REV=2012.04.19.14.10
   BASEDIR:  /
    VENDOR:  Oracle Corporation
      DESC:  Oracle VM Server for SPARC – Virtualization for SPARC T-Series
    PSTAMP:  scapen-on10-020120419141012
  INSTDATE:  Dec 21 2012 15:20
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed
     FILES:       79 installed pathnames
                  18 shared pathnames
                  23 directories
                  22 executables
                5851 blocks used (approx)

Verify the ldms

bash-3.2# svcs|grep ldmd
online         15:20:55 svc:/ldoms/ldmd:default

bash-3.2# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active     -n-c–  UART    64    32256M   0.2%  1d 4h 38m

Save and Restore ldm configuration:

bash-3.2# tar -cvpf /var/tmp/autosave.tar var/opt/SUNWldm/autosave-*
a var/opt/SUNWldm/autosave-_Default_SP_Config/ 0K
a var/opt/SUNWldm/autosave-_Default_SP_Config/md-primary.bin 61K
a var/opt/SUNWldm/autosave-_Default_SP_Config/hv-md.bin 22K
a var/opt/SUNWldm/autosave-_Default_SP_Config/timestamp 1K
a var/opt/SUNWldm/autosave-_Default_SP_Config/sp-pri.bin 88K
a var/opt/SUNWldm/autosave-_Default_SP_Config/sp-timestamp 1K

to restore

cd /

tar -xvpf /var/tmp/autosave.tar

Saving and Restorig LDOM constraint Database
backup and restore the xml file: /var/opt/SUNWldm/ldom-db.xml

create default services after installing the LDOM manager:

3 defaults services need to be created on primary ldom:

virtual control concentrator service (vcc) service for use by the virtual network terminal server
daemon (vntsd) and as a concentrator for all logical domain consoles.

virtual disk server (vds) to allowimporting virtual disks into a logical domain

virtual switch service (vsw) to enable networking between virtual network (vnet)
devices in logical domains

VCC # ldm add-vcc port-range=5000-5100 vcc_primary primary
VDS # ldm add-vds vds0_primary primary
VSW # ldm add-vsw net-dev=nxge0 vsw0_primary primary

MAC address to vsw can be manually set using below command: only on Solaris 10:

primary# ldm add-vsw mac-addr=2:04:4f:fb:9f:0d net-dev=nxge0 primary-vsw0 primary

list services:
#ldm list-services primary
VCC
    NAME             LDOM             PORT-RANGE
    vcc_primary primary          5000-5100

VSW
    NAME             LDOM             MAC               NET-DEV   ID   DEVICE     LINKPROP   DEFAULT-VLAN-ID PVID VID                  MTU   MODE   INTER-VNET-LINK
    vsw0_primary primary          00:14:4f:fb:af:0d nxge0     0    switch@0              1               1                         1500         on     

VDS
    NAME             LDOM             VOLUME         OPTIONS          MPGROUP        DEVICE
    vds0_primary primary
bash-3.2# ldm set-vcpu 8 primary
bash-3.2# ldm list-constraints primary

bash-3.2# ldm start-reconf primary
Initiating a delayed reconfiguration operation on the primary domain.
All configuration changes for other domains are disabled until the primary
domain reboots, at which time the new configuration for the primary domain
will also take effect.

bash-3.2# ldm set-memory 1G primary
——————————————————————————
Notice: The primary domain is in the process of a delayed reconfiguration.
Any changes made to the primary domain will only take effect after it reboots.
——————————————————————————
bash-3.2#

bash-3.2# ldm add-config initial
bash-3.2# ldm list-config
factory-default
initial [current]
bash-3.2#sync;sync

bash-3.2# /usr/sbin/shutdown -y -g0 -i6

How to stop all domains:

1. power down all domains OS to OK prompt
2. primary# ldm stop-domain -a
3. primary# ldm unbind-domain ldom

 To remove all guest domains, run above 3 steps and below one step
4. ldm remove-domain -a

 How to upgrade LODM Manager:

1. svcadm disable ldmd
2. pkgrm SUNWldm
3. Instll the new package (copied to /var/tmp)
cd /var/tmp;pkgadd -Gd. SUNWldm.v
4. ldm list (verify)

 How to disable logical domain manager and remove the package?
1. svcadm disable ldmd
2. pkgrm SUNWldm; pkgrm SUNWldmp2v

Documents:

Oracle®VMServer for SPARC 2.2 Administration Guide : E23807.pdf

Solaris LDOMs virtualization : setup guide

Sun Logical Domains or LDoms is a full virtual machine that runs an independent operating system instance and contains virtualized CPU, memory, storage, console, and cryptographic devices. This technology allows you to allocate a system resources into logical groupings and create multiple, discrete systems, each with their own operating system, resources, and identity within a single computer system. We can run a variety of applications software in different logical domains and keep them independent of performance and security purposes. The LDoms environment can help to achieve greater resource usage, better scaling, and increased security and isolation. more..

Deployment and understanding of LDOMS (Logical Domains)

Ldom technology allows us to allocate a system’s various resources such as memory, CPUs and devices (I/O,Networl interfaces etc) into logical groupings and create multiple, discrete systems each having their own operating system, resources, and identity within a single computer system. more

 

Difference between LDOMS’s and Zones (containers)

Today’s meeting one guy was speaking LDOM’s and Zones as one and the same. I thought I will write the difference between LDOM’s and Zones

LDOM’s can be created only on sun4v based cpu servers as they come with hypervisor, which is important hardware layer that give power of virtualization. Sun4v cpu latest server models are T4 series servers T4-1 (1 socket, 8 core, 8 threads per core=64 threads per server), T4-2 (2 socket, 2×64 threads), T4-4 (4 socket, 4×64 threads).

LDOMS have independent cpu, memory, file system and most important Kernel. These domains have their own console connectivity and boot prompt,

LDOMs are created and managed from service domain. Every LDOM needs it’s own administrative efforts like patching,

LDOMs now called as Oracle VM servers

 

Zones (containers) do not required any specific hardware as it’s a purely software based virtual technology. Zones also called as containers, reason is you will create compartments in a OS, sharing kernel, cpu, memory and file system

Zones (containers) share everything, CPU, Memory, File Systems and most important thing Kernel. Global zone runs on solaris 10, you can have non global zones with solaris 8, solaris 9 and even linux.

Zones (containers) zones are created and managed through global zones, most of the single administrative task will be applicable to all zones like patching.

 

Below link has a nice presentation on LDOMs, but beware it’s older version

http://unixfoo.blogspot.hk/2008/04/solaris-ldom-virtualization-documents.html

http://en.wikipedia.org/wiki/Logical_Domains

http://en.wikipedia.org/wiki/Solaris_Containers