Solaris LDoms

Solaris LDOMs

See Logical Domains (LDoms) 1.0.3 Administration Guide

Overview

LDoms are based on SPARC Hypervisor - a small firmware layer that provides server virtualization on a hardware level.

A logical domain is a discrete logical grouping with its own operating system, resources, and identity within a single computer system. Each logical domain can be created, destroyed, reconfigured, and rebooted independently, without requiring a power cycle of the server. You can run a variety of applications software in different logical domains and keep them independent for performance and security purposes.

Logical Domains Manager

The Logical Domains Manager is used to create and manage logical domains. There can be only one Logical Domains Manager per server. The Logical Domains Manager maps logical domains to physical resources.

LDoms Manager can be downloaded free from [1]

Logical Domains Roles

Domain RoleDescription
Control Domain Domain in which the Logical Domains Manager runs allowing you to create and manage other logical domains and allocate virtual resources to other domains. There can be only one control domain per server. The initial domain created when installing Logical Domains software is a control domain and is named primary.
Service Domain Domain that provides virtual device services to other domains, such as a virtual switch (vsw), a virtual console concentrator (vcc) and virtual network terminal server (vntsd) that replaces sc, and a virtual disk server (vds).
I/O Domain Domain that has direct ownership of and direct access to physical I/O devices, such as a network card in a PCI Express controller. Shares the devices with other domains in the form of virtual devices when the I/O domain is also the control domain. The number of I/O domains you can have is dependent on your platform architecture. For example, if you are using a Sun UltraSPARC® T1 processor, you can have a maximum of two I/O domains, one of which also must be the control domain.
Guest Domain Domain that is managed by the control domain and uses services from the I/O and service domains.

Dynamic and Delayed Reconfigurations

Solaris 10 supports only dynamic reconfiguration of the number of virtual processors. With delayed reconfiguration, which takes place upon a reboot of a logical domain, all other add/remove resource operations are considered delayed.

Persistent Configuration

All current configurations of the logical domains can be saved in on the SC. Both LDoms Manager and ALOM allow a system administrator to choose which configuration to use upon startup.

Security

The Solaris Security Toolkit software, informally known as the !JumpStart Architecture and Security Scripts toolkit (SUNWjass), adapted for the LDoms Manager and is included in it. It allows the following: * Hardening – Modifying Solaris OS configurations to enable the Solaris Security Toolkit to work with the Logical Domains Manager. Most hardening functions are performed on a secure system anyway by a knowledgeable system administrator * Minimizing – Installing the minimum number of core Solaris OS packages necessary for LDoms and LDoms Management Information Base (MIB) support * Authorization – Setting up authorization using the Solaris OS Role-Based Access Control (RBAC) * Auditing – Using the Solaris OS Basic Security module (BSM) to identify the source of security changes to the system to determine what was done, when it was done, by whom, and what was affected * Compliance – Determining if a system’s configuration is in compliance with a predefined security profile using the Solaris Security Toolkit’s auditing feature

The Security Toolkit can be installed on a Control Domain and on other domains. The Control Domain uses ldm_control-secure.driver, other domains - secure.driver. ldm_control-secure.driver is intended to provide fewer system services than typical for a Solaris OS domain, reserving the control domain for Logical Domains Manager operations, rather than general usage.

Installation

 1. Download [http://www.sun.com/servers/coolthreads/ldoms/get.jsp LDoms_Manager-1_0_3.zip] into /var/spool/pkg
# cd /var/spool/pkg
# unzip -q LDoms_Manager-1_0_3.zip
# cd LDoms_Manager-1_0_3/Product
# pkgadd -Gd . SUNWldm.v
# pkgadd -d . SUNWjass (optional)
# svcadm enable ldmd
# vi /etc/profile
 PATH=$PATH:/opt/SUNWldm/bin:/opt/SUNWjass/bin
 MANPATH=$MANPATH:/opt/SUNWldm/man:/opt/SUNWjass/man
 2. Optionally, harden the control domain
jass-execute -d ldm_control-secure.driver
 3. To validate the driver, use
jass-execute -a secure.driver
 4. To remove hardening, run
jass-execute -u
 5. To see active domains
# ldm list
------------------------------------------------------------------------------
Notice: the LDom Manager is running in configuration mode. Configuration and
resource information is displayed for the configuration under construction;
not the current active configuration. The configuration being constructed
will only take effect after it is downloaded to the system controller and
the host is reset.
------------------------------------------------------------------------------
NAME             STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
primary          active   -n-c-   SP      24    3968M    0.1%  58m

Configuring Logical Domains

Creating Default Services

Virtual Disk (vds)

# ldm add-vds primary-vds0 primary

Virtual Switch (vsw)

ldm add-vsw net-dev=bge0 primary-vsw0 primary

It is important to note that the virtual switch can be configured without a physical network interface associated with it, i.e. without net-dev argument. In this case the packet forwarding must be configured in the control/service domain and the routing information should be entered in guest domains.

The advantages of this configuration are:

Virtual Console Concentrator (vcc)

# ldm add-vcc port-range=5000-5100 primary-vcc0 primary

To verify that all services have been created, run

# ldm list-services primary

Assign resources to the control domain

# ldm set-mau 1 primary
# ldm set-vcpu 4 primary
# ldm set-memory 1G primary

Add Logical Domain configuration called ''initial'' to the SC

# ldm add-config initial

Note - Currently, there is a limit of 8 configurations that can be saved on the SC, not including the factory-default configuration.

Verify that configuration is ready to be used at the next reboot

# ldm list-config

Reboot

# touch /reconfigure
# shutdown -i6 -g0 -y

Configure the virtual switch (vsw0) to be the primary interface

# mv /etc/hostname.bge0 /etc/hostname.vsw0
# reboot

Enable the Virtual Network Terminal Server

# svcadm enable vntsd

Create a Guest Domain

# ldm add-domain betcruise

Add resources to the Guest Domain

It is convinient to use a file, located on a ZFS file system as a boot device for the Guest Domain due to ZFS snapshot/clone features.

  1. umount /export/home
  2. vi /etc/vfstab <commented out /export/home there> }}} 1. Create zfs pool on a disk slice
# zpool create -f ldoms_pool /dev/dsk/c0t0d0s6
# zpool list
NAME                    SIZE    USED   AVAIL    CAP  HEALTH     ALTROOT
ldoms_pool              107G     88K    107G     0%  ONLINE     -
 2. Create ZFS file systems using ldoms_pool, one for each guest domain
# zfs create ldoms_pool/betcruise
# zfs create ldoms_pool/acc
 3. Make a file that will be exported as a block device to a guest domain
# mkfile 32g /ldoms_pool/betcruise/bootdisk
 4. Export this file as a block device
# ldm add-vdsdev /ldoms_pool/betcruise/bootdisk vol1@primary-vds0

Add virtual CPUs

# ldm add-vcpu 8 betcruise

Add memory

# ldm add-memory 1g betcruise

Add network interface

# ldm add-vnet vnet0 primary-vsw0 betcruise

Add virtual disk

# ldm add-vdisk vdisk1 vol1@primary-vds0 betcruise

Set OBP variables

# ldm set-var auto-boot\?=true betcruise
# ldm set-var boot-device=vdisk1 betcruise

Bind domain

# ldm bind-domain betcruise
# ldm list-domain betcruise
NAME             STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
betcruise        bound    -----   5000    8     1G

Start the guest domain

# ldm start-domain betcruise

Save current config to SC

# ldm add-config current
# ldm list-config

Connect to the guest domain's console

# telnet localhost 5000

Install Solaris OS on the guest domain, using [wiki:netboot NetBoot], for example

 ok boot net - install nfs://10.1.1.13/export/home/jumpstart/acc-1/acc-1.tar

To send break signal to the Solaris OS in LDom

telnet> send break
telnet> <return> <return>

Jump-Starting a Guest Domain

It is important to know that disk devices do not have a target; for example, instead of c0t0d0s0, use c0d0s0 in a profile.

Rebooting the Control Domain

If a control domain is restarted, I/O services for guest domains might be unavailable until the control domain has restarted. This is because the control domain functions as a service domain in the Logical Domains Manager 1.0.3 software. Guest domains appear to freeze during the reboot process. Once the control domain has fully restarted, the guest domains resume normal operations.

It is only necessary to shut down guest domains when power is going to be removed from the entire server.

ALOM bootmode

ALOM bootmode has an additional parameter config.

bootmode [normal|reset_nvram|bootscript=strong|config=”config-name”]

The config=”config-name” option enables you to set the configuration on the next power on to another configuration, including the factory-default shipping configuration. You can invoke the command whether the host is powered on or off. It takes effect on the next host reset or power on.