The simplest and most straight forward way to install KIWI is to use the packages that are part of the SUSE distribution you are running. In openSUSE the kiwi packages are part of the "standard" distribution and in SUSE Linux Enterprise kiwi packages are available in the SDK channel.

If you use SUSE Studio to set up your configuration and then export it to build locally on your machine you want to make sure to use the same version of KIWI that SUSE Studio uses to build images. This version of KIWI which most often differs from the version released with a given distribution is available from the openSUSE Build Service. The repository you want to add to your system is http://download.opensuse.org/repositories/home:/ctso/DISTRO.

Once you have the repository added to your system you can search for the kiwi packages and install them as described above.

KIWI is under active development and changes almost on a daily basis. The development code is generally released once a week on Friday. Sometimes the development releases contain new bugs that break existing builds. Therefore, this is not necessarily the best release stream to track if you are looking for critical on time builds of already configured appliances. However, tracking this stream provides a great opportunity for you to help in detecting such bugs and by reporting them on the mailing list you can help the developers. Any regression fixes are generally released as soon as they are completed. Thus, there is no need to wait until the next scheduled release on a Friday. If you add the http://download.opensuse.org/repositories/Virtualization:/Appliances/DISTRO repository to your system you can track the development release.

Once you have the repository added to your system you can search for the kiwi packages and install them as described above.

<image schemaversion="5.2" name="iname"
  displayname="text" 
  inherit="path" 
  kiwirevision="number" 
  id="10 digit number">
  <!-- ... -->
</image>

The image definition starts with an image tag and requires the schema format at version 2.0. The attribute name specifies the name of the image which is also used for the filenames created by KIWI. Because we don’t want spaces in filenames the name attribute must not have any spaces in its name.

The following optional attributes can be inserted in the image tag:

Inside the image section the following mandatory and optional subelements exists. The simplest image description must define the elements description, preferences, repository and packages (at least one of type="bootstrap").

<description type="system">
  <author>an author</author>
  <contact>mail</contact>
  <specification>short info</specification>
</description>

The mandatory description section contains information about the creator of this image description. The attribute type could be either of the value system which indicates this is a system image description or at value boot for boot image descriptions.

<profiles>
   <profile name="name" description="text"/>
   <!-- ... -->
</profiles>

The optional profiles section lets you maintain one image description while allowing for variation of the sections packages and drivers that are included. A separate profile element must be specified for each variation. The profile child element, which has name and description attributes, specifies an alias name used to mark sections as belonging to a profile, and a short description explaining what this profile does.

To mark a set of packages/drivers as belonging to a profile, simply annotate them with the profiles attribute. It is also possible to mark sections as belonging to multiple profiles by separating the names in the profiles attribute with a comma. If a packages or drivers tag does not have a profiles attribute, it is assumed to be present for all profiles.

<preferences profiles="name">
  <version>1.1.2</version>
  <packagemanager>zypper</packagemanager>
  <type image="name" ...>
    <ec2config|systemdisk|oemconfig|pxedeploy|size|split|machine>
  </type> 
</preferences>

The mandatory preferences section contains information about the supported image type(s), the used package manager, the version of this image, and optional attributes. The image version must be a three-part version number of the format: Major.Minor.Release. In case of changes to the image description the following rules should apply:

By default, KIWI uses the zypper package manager but it is also possible to use the non SUSE native package manager called smart.

In general the specification of one preferences section is sufficient. However, it’s possible to specify multiple preferences sections and distinguish between the sections via the profiles attribute. Data may also be shared between different profiles. Using profiles it is possible to, for example, configure specific preferences for OEM image generation. Activation of a given preferences during image generation is triggered by the use of the --add-profile command line argument.

For each preferences block at least one type element must be defined. It is possible to specify multiple type elements in any preferences block. To set a given type description as the default image use the boolean attribute primary and set its value to true. The image type to be created is determined by the value of the image attribute. The following list describes the supported types and possible values of the image attribute:

All of the mentioned types can specify the boot attribute which tells KIWI to call itself to build the requested boot image (initrd). It is possible to tell KIWI to check for an already built boot image which is a so called prebuilt boot image. To activate searching for an appropriate prebuilt boot image the type section also provides the attribute checkprebuilt="true|false". If specified KIWI will search for a prebuilt boot image in a directory named /usr/share/kiwi/image/*boot/*-prebuilt. Example: If the boot attribute was set to isoboot/suse-10.3 and checkprebuilt is set to true KIWI will search the prebuilt boot image in /usr/share/kiwi/image/isoboot/suse-10.3-prebuilt. The directory KIWI searches for the prebuilt boot images can also be specified at the commandline with the --prebuiltbootimage parameter.

Within the preferences section, there are the following optional elements:

The type element may contain child elements to provide specific configuration information for the given type. The following lists the supported child elements:

<users group="group_name" id="number"> 
  <user home="dir" id="number" name="user" pwd="..."
        pwdformat="encrypted|plain" realname="string" shell="path/>
  <!-- ... -->
</users>

The optional users element lists the users belonging to the group specified with the group attribute. At least one user child element must be specified as part of the users element. Multiple users elements may be specified.

The attributes home, id, name, pwd, realname, and shell specify the created users home directory, the user name, the user’s password, the user’s real name, and the user’s login shell, respectively. By default, the value of the password attribute is expected to be an encrypted string. An encrypted password can be created using kiwi --createpassword. It is also possible to specify the password as a non encrypted string by using the pwdformat attribute and setting it’s value to “plain”. KIWI will then encrypt the password prior to the user being added to the system.

All specified users and groups will be created if they do not already exist. By default, the defined users will be part of the group specified with the group attribute of the users element and the default group called “users”. If it is desired to have the specified users to only be part of the given group it is necessary to specify the id attribute. It is recommended to use a group id greater than 100.

<drivers profiles="name"> 
  <file name="filename"/> 
  <!-- ... --> 
</drivers>

The optional drivers element is only useful for boot images (initrd). As a boot image doesn’t need to contain the complete kernel one can save a lot of space if only the required drivers are part of the image. Therefore the drivers section exists. If present only the drivers which matches the file names or glob patterns will be included into the boot image. Each file is specified relative to the /lib/modules/Version/kernel directory.

According to the driver element the specified files are searched in the corresponding directory. The information about the driver names is provided as environment variable named like the value of the type attribute and is processed by the function suseStripKernel. According to this along with a boot image description a script called images.sh must exist which calls this function in order to allow the driver information to have any effect.

<repository type="type" alias="name" imageinclude="true|false"
              password="password" priority="number" status="replaceable"
              username="user-name"> <source path="URL"/>
</repository>

The mandatory repository element specifies the location and type of a repository to be used by the package manager as a package installation source. The mandatory type attribute specifies the repository type. A specified repository can only be accessed by the chosen package manager if the given type is supported by the specified package manager. KIWI supports smart or zypper as package managers, specified with the packagemanager element. The default package manager is zypper. The following table shows the possible supported repository types for each package manager:

The repository element has the following optional attributes:

When specifying an https location for a repository it is generally necessary to include the “openssl-certs” and “cracklib-dict-full” packages in the bootstrap section of the image configuration.

The location of a repository is specified by the path attribute of the mandatory source child element. The location specification may include the %arch macro which will expand to the architecture of the image building host. The value for the path attribute may begin with any of the following location indicators:

<packages type="type" profiles="name" patternType="type"
   <package name="name" arch="arch"/> 
   <package name="name" replaces="name"/>
   <package name="name" bootinclude="true" bootdelete="true"/> 
   <archive name="name" bootinclude="true"/> 
   <package .../> 
   <opensusePattern name="name"/> 
   <opensusePattern .../>
   <opensuseProduct name="name"/> 
   <opensuseProduct .../> 
   <ignore name="name"/> 
   <ignore .../>
</packages>

The mandatory packages element specifies the list of packages (element package) and patterns (element opensusePattern) to be used with the image. The value of the type attribute specifies how the packages and patterns listed are handled, supported values are as follows:

<package name="kernel-default" replaces="kernel-xen"/>

<package name="kernel-xen"  replaces="kernel-default"/> 

If no overlay filesystem is in use but the image filesystem is based on a compressed filesystem KIWI allows to setup which files and directories should be writable in a so called split section. In order to allow to login into the system, at least the /var directory should be writable. This is because the PAM authentication requires to be able to report any login attempt to /var/log/messages which therefore needs to be writable. The following split section can be used if the flag compressed is used:

<split> 
  <persistent> 
     <file name="/var"/>
     <file name="/var/*"/> 
     <file name="/boot"/> 
     <file name="/boot/*"/> 
     <file name="/etc"/> 
     <file name="/etc/*"/> 
     <file name="/home"/> 
     <file name="/home/*"/>
     <file name="/tmp"/>
     <file name="/tmp/*"/>
  </persistent> 
</split>

A hybrid image is a iso image including a partition table and can therefore be attached as a CD/DVD and as a normal disk to the system. This has the advantage that a hybrid iso live system can be burned to a CD/DVD as well as uploaded to a USB stick. In order to activate the hybrid feature the hybrid flag must be set to true as indicated below.

<type image="iso" ... hybrid="true"/>

As indicated above a hybrid iso image also works as USB stick image. If a hybrid iso is used like a disk image on a writable medium like a USB stick it's possible to write into a persistent area on the stick instead of the RAM. kiwi will create an additional ext2 partition to store that information on the disk if the attribute hybridpersistent is set to true.

<type image="iso" ... hybridpersistent="true"/>

In contrast to the hybrid iso image it's also possible to create a oem virtual disk image which is dumped on the stick. The big advantage with this approach is, that it's possible to create a stick which contains a live OS but also a data partition for custom data. The data partition is a fat partition also recognized by the Windows operating system. In order to create such a Windows friendly stick one has to pass the option --fat-storage <size-in-MB>.

kiwi --create ... --fat-storage 500

If this option is set kiwi will use the syslinux bootloader for the image as well as the first partition as fat partition of the specified size. The live OS itself will live in a LVM which allows easy manipulation of the logical root volume. For further information about the OEM image type please refer to the OEM chapter Chapter 13, OEM Image—Preload Systems

<type image="oem" filesystem="clicfs" ramonly="true" .../>

A VMware image is accompanied by a guest configuration file. This file includes information about the hardware to be represented to the guest image by the VMware virtualization environment as well as specification of resources such as memory.

Within the config.xml file it is possible to specify the VMware configuration settings. In addition it is possible to include selected packages in the created image that are specific to the VM image generation. The following config.xml snippet provides general guidance on the elements in config.xml.

<packages type="vmx"> 
   <!-- packages you need in VMware only -->
</packages> 
<type......>
  <machine memory="512"> 
    <vmdisk controller="ide" id="0"/> 
  </machine> 
</type>

Given the specification above KIWI will create a VMware guest configuration specifying the availability of 512 MB of RAM and an IDE disk controller interface for the VM guest. For additional information about the configuration settings please refer to the machine section.

The guest configuration can be loaded through VMware user interface and may be modified through the GUI. The configuration file has the .vmx extension as shown in the example below.

/tmp/myvm-result/suse-##.#-vm-guest.i686-1.1.2.vmx

Using the format="vmdk" attribute of the <type> start tag will create the VMware formatted disk image (.vmdk file) and the required VMware guest configuration (.vmx) file.

In addition it is possible to create an image for the Xen virtualization framework. By adding the bootprofile and bootkernel attributes to the <type> start tag with values of xen and xenboot, respectively. Please refer to the Chapter 14, Xen Image—Paravirtual Systems for additional details.

KIWI also provides support for LVM (Logical Volume Management). In this mode the disk partition table will include one lvm partition and one standard ext2 boot partition. KIWI creates the kiwiVG volume group and adds logical volumes as they are needed and configured according to the image type and filesystem. After boot of the system the user has full control over the volume group and is free to change/resize/increase the group and the volumes inside. Support for LVM has been added for all image types which are disk based. This includes vmx, oem and usb. In order to use LVM for the vmx type just add the --lvm option as part of the KIWI create step or add the attribute lvm="true" as part of the type section in your config.xml file.

kiwi --create /tmp/myvm --type vmx -d /tmp/myvm-result --lvm 

With the optional systemdisk section you can set one or more top level directories into a separate volume. See Chapter 5, KIWI Image Description for a detailed explanation.

In order to setup the atftp server the following steps are required

In contrast to the atftp server setup the following DHCP server setup can only serve as an example. Depending on your network structure, the IP addresses, ranges and domain settings needs to be adapted in order to allow the DHCP server to work within your network. If you already have a DHCP server running in your network, make sure that the filename and next-server information is provided by your server. The following steps describe how to setup a new DHCP server instance:

This section describes the netboot client control file:

hwtype.$<$MAC Address$>$

The control file is primarily used to set up new netboot clients. In this case, there is no configuration file corresponding to the client MAC address available. Using the MAC address information, the control file is created, which is uploaded to the TFTP servers upload directory /var/lib/tftpboot/upload.

This section describes the netboot client configuration file:

config.$<$MAC Address$>$

The configuration file contains data about image, configuration, synchronization, or partition parameters. The configuration file is loaded from the TFTP server directory /var/lib/tftpboot/KIWI via TFTP for previously installed netboot clients. New netboot clients are immediately registered and a new configuration file with the corresponding MAC address is created. The standard case for the deployment of a PXE image is one image file based on a read-write filesystem which is stored onto a local storage device of the client. Below, find an example to cover this case.

DISK=/dev/sda
PART='5;S;x,x;L;/'
IMAGE='/dev/sda2;suse-##.#-pxe-client.i686;1.2.8;192.168.100.2;4096'

The following format is used:

IMAGE='device;name;version;srvip;bsize;compressed,...,'
CONF='src;dest;srvip;bsize;[hash],...,src;dest;srvip;bsize;[hash]'
PART='size;id;Mount,...,size;id;Mount'
DISK=device

By default all downloads controlled by the KIWI linuxrc code are performed by an atftp call and therefore uses the tftp protocol. With PXE the download protocol is fixed and thus you can’t change the way how the kernel and the boot image (initrd) is downloaded. As soon as Linux takes over control the following download protocols http, https and ftp are supported too. KIWI makes use of the curl program to support the additional protocols.

In order to select one of the additional download protocols the following kernel parameters needs to be setup:

To setup this parameters edit the file /srv/tftpboot/pxelinux.cfg/default on your PXE boot server and change the append line accordingly. Please note all downloads except for kernel and initrd are now controlled by the given server and protocol. You need to make sure that this server provides the same directory and file structure as initially provided by the kiwi-pxeboot package.

If there is no local storage and no remote root mount setup the image can be stored into the main memory of the client. Please be aware that there should be still enough RAM space available for the operating system after the image has been deployed into RAM. Below, find an example:

As used in the suse-pxe-client example it is possible to make use of the clicfs overlay filesystems to combine two filesystems into one. In case of thin clients there is often the need for a compressed filesystem due to space limitations. Unfortunately all common compressed filesystems provides only read-only access. Combining a read-only filesystem with a read-write filesystem is a solution for this problem. In order to use a compressed root filesystem make sure your config.xml’s filesystem attribute contains either squashfs or clicfs. Below, find an example:

DISK=/dev/sda
PART='5;S;x,62;L;/,x;L;x'
IMAGE='/dev/sda2;suse-##.#-pxe-client.i386;\
      1.2.8;192.168.100.2;4096'
UNIONFS_CONFIG=/dev/sda3,/dev/sda2,clicfs
KIWI_INITRD=/boot/initrd

As an alternative to clicfs kiwi also supports the fuse based unionfs utility. clicfs writes block lists on the device used for the write operations. unionfs points all write operations into another filesystem which also allows to use remote write locations and to watch them separately. In order to use unionfs to write into ram the following UNIONFS_CONFIG setup is required

UNIONFS_CONFIG=tmpfs,/dev/sda2,unionfs

If you want to write persistently it's required to add the local device name e.g /dev/sda3 instead of tmpfs as shown in the initial example. If the read-only location is on NFS the setup to write into a tmpfs would look like the following

NFSROOT="192.168.100.2;/srv/kiwi-read-only-path"
UNIONFS_CONFIG=tmpfs,nfs,unionfs

Also possible is to read remotely and to write remotely. Some NFS write space is required prior to the following setup:

NFSROOT="192.168.100.2;/srv/kiwi-read-only-path"
UNIONFS_CONFIG=/srv/kiwi-read-write-path,nfs,unionfs

As an alternative to the UNIONFS_CONFIG method it is also possible to create a split image and combine the two portions with the COMBINED_IMAGE method. This allows to use different filesystems without the need for an overlay filesystem to combine them together. Below find an example:

Instead of installing the image onto a local storage device of the client it is also possible to let the client mount the root tree via an NFS remote mount. Below find an example:

As an alternative for root over NFS it is also possible to let the client mount the root tree via a special network block device. Below find an example:

As an alternative for root over NBD it is also possible to let the client mount the root device via a special ATA over Ethernet network block device. Below find an example:

To make use of the grouping functionality, some new configuration files will be required. These configuration files currently have to be manually managed rather than provided, however future versions of KIWI may provide a means of managing groups more effectively once this feature stabilizes. The number of configuration files required will depend on the number of hardware groups that will be created, rather than one configuration file for each MAC address that will reside on the network.

There will be one configuration file that will always be required if using groups, called:

/srv/tftpboot/KIWI/config.group

This file has a new static element that must exist, and one or more dynamic elements depending on the number of groups that will be created. For example, the config.group file defined below lists 3 distinct groups:

KIWI_GROUP="test1, test2, test3"

test1_KIWI_MAC_LIST="11:11:11:11:11:11, 00:11:00:11:22:CA"

test2_KIWI_MAC_LIST="00:22:00:44:00:4D, 99:3F:21:A2:F4:32"

test3_KIWI_MAC_LIST="00:54:33:FA:44:33, 84:3D:45:2F:5F:33"

Note: The above hardware addresses contain random entries, and may not reflect actual hardware.

As we can see in the above example the file contains 1 static element, KIWI_GROUP, and 3 dynamic elements "test1_KIWI_MAC_LIST, test2_KIWI_MAC_LIST and test3_KIWI_MAC_LIST". The definitions of these elements are as follows:

In addition to the config.group file, each defined group will require a config.<GROUP_NAME> file. This file is exactly like a standard KIWI config.<MAC> file, but is assigned to a group of hosts rather than a single unit. If we continue with the example we used in the previous section, we would need the following files:

/srv/tftpboot/KIWI/config.test1
/srv/tftpboot/KIWI/config.test2
/srv/tftpboot/KIWI/config.test3

The contents of these files is the same that would normally reside in a config.<MAC> file, and all definitions that would be supported for a single host, are supported for a group of hosts. In addition, if a host is matched to a group, yet the config.<GROUP_NAME> file does not exist, KIWI will error out.

For example, the following configuration file, called config.test1 would be used for the group called "test1":

DISK=/dev/sda
PART='5;S;x,x;L;/'
IMAGE='/dev/sda2;suse-##.#-pxe-client.i686;1.2.8;192.168.100.2;4096'
CONF='CONFIGURATIONS/xorg.conf.test1;/etc/X11/xorg.conf;192.168.100.2;4096,\
     CONFIGURATIONS/syslog.conf;/etc/sysconfig/syslog.conf;192.168.100.2;4096'

As a result of this configuration file, the image would be configured consistently across all the hosts assigned to test1. The following file called config.test2, contains a small change that may be specific to a function:

DISK=/dev/sda
PART='5;S;x,x;L;/'
IMAGE='/dev/sda2;suse-##.#-pxe-client.i686;1.2.8;192.168.100.2;4096'
CONF='CONFIGURATIONS/xorg.conf.test2;/etc/X11/xorg.conf;192.168.100.2;4096,\
     CONFIGURATIONS/syslog.conf;/etc/sysconfig/syslog.conf;192.168.100.2;4096'

As we can see, while group 1 and 2 share the syslog.conf configuration file, they have different xorg.conf files defined, therefore two distinct groups with one or more hosts assigned to each group can now be configured by managing a smaller number of files.

The only issue with running mixed hardware configurations pertains primarily to hardware differences. For instance, it may be possible to create a single, xorg.conf file that is able to work with all of the hardware, but there is a chance it might not be possible to do so. With this in mind, KIWI provides a mechanism to provide "default" configurations that works with the most common hardware configuration, while providing hardware specific overrides to allow for any differences and yet have all hardware linked to the same group.

HARDWARE_MAP="vendor_name_model"
vendor_name_model_HARDWARE_MAP="00:00:00:11:11:11"

VENDOR_CONF='CONFIGURATIONS/xorg.conf.hardware_name_model;/etc/X11/xorg.conf;192.168.100.2;4096'

cd /srv/tftpboot/KIWI
ls
   config.example1
   config.group
   hardware_config.maxterm_3500

cat config.group

KIWI_GROUP="example1"
example1_KIWI_MAC_LIST=
  "00:00:00:00:00:01 00:00:00:00:00:02 \
   00:00:00:00:00:03 00:00:00:00:00:04 \
   00:00:00:00:00:05 00:00:00:00:00:06 \
   00:00:00:00:00:07 00:00:00:00:00:08 
   00:00:00:00:00:09 00:00:00:00:00:0A"

cat config.example1

KIWI_INITRD=/boot/initrd
KIWI_KERNEL=/boot/linux
DISK=/dev/sda
PART='5;S;x,769;L;/,x;L;x'
IMAGE='/dev/sda2;exmaple-kiosk-opensuse-##.#-pxe-client.i686;0.0.1;192.168.1.2;4096'
UNIONFS_CONFIG=/dev/sda3,/dev/sda2,clicfs
CONF='prefs.js;/home/kioskuser/.mozilla/firefox/07xvl1ty.default/prefs.js;192.168.1.2;4096,xorg.conf;/etc/X11/xorg.conf;192.168.1.2;4096'
RELOAD_IMAGE=yes
RELOAD_CONFIG=yes
HARDWARE_MAP='maxterm_3500'
maxterm_3500_HARDWARE_MAP='00:00:00:00:00:02 00:00:00:00:00:03 00:00:00:00:00:04'

cat hardware_config.maxterm_3500

VENDOR_CONF='xorg.conf.maxterm_3500;/etc/X11/xorg.conf;192.168.1.2;4096,
            someconfig.cfg;/etc/sysconfig/someconfig.cfg;192.168.1.2;4096'

It is possible to specialize the OEM install process by providing shell scripts with the following names. For more information how to pack the scripts and make them work in the boot code, see the chapter Section 3.3, “Boot image Hook-Scripts”.

By default the oemboot process will create/modify a swap, /home and / partition. It is possible to influence the behavior with the oem-* elements. See Chapter 5, KIWI Image Description for details.

KIWI also provides support for LVM (Logical Volume Management). In this mode the disk partition table will include one lvm partition and one standard ext2 boot partition. KIWI creates the kiwiVG volume group, unless the lvmgroup attribute has been set, and adds logical volumes to the group based on the configuration given by the systemdisk block for this type. The filesystem for the volume group is determined by the filesystem attribute of the type element. After booting the system the user has full control over the volume group and is free to change (resize/increase) the group and the volumes inside. Support for LVM has been added for all disk based image types. This includes the vmx and oem image types. In order to use LVM the existence of a systemdisk section is required. The systemdisk specification may be empty. An empty systemdisk specification triggers the creation of one LVM root volume with the default kiwiVG name.

kiwi --create /tmp/myoem --type oem -d /tmp/myoem-result --lvm

With the systemdisk section you can specify to have one or more top level directories in a separate volume. See Chapter 5, KIWI Image Description for a detailed explanation.

The default installation method of an OEM is dumping the entire virtual disk on the selected target disk and repartition the disk to the real geometry. This works but will also wipe everything which was on the disk before. KIWI also supports the installation into already existing partitions. This means the user can setup a disk with free partitions for the KIWI OEM installation process. This way already existing data will not be touched. In order to activate the partition based install mode the following oem option has to be set in config.xml:

<oem-partition-install>true</oem-partition-install>

Compared to the disk based install the following differences should be mentioned:

Instead of manually dumping the OEM image on the target device or creating a kiwi install CD, USB stick, there is a third method of deploying the OEM image on the target device. It's possible to let the image be downloaded from a PXE boot server over the network. This requires a PXE network boot server to be setup properly in the first place For details how to do this refer to the chapter: Chapter 12, PXE Image—Thin Clients. If your pxe server is running the following steps are required to setup the install process over the network

This section is not a replacement for the EC2 documention mentioned earlier. We will only cover the concepts and commands necessary to get you started such that you can launch the KIWI created image in this example.

Prior to launching any instance in EC2 you need to have a keypair. If you do not already have a keypair in EC2 you can create one using the ec2-add-keypair command. This creates a public/private keypair that is used to grant you access to your running instance via the ssh tools. Generate the keypair as shown below, the gsgkey name is arbitrary and used in this example, you can choose any name you like. The use of the key is quite frequent. Therefore, you probably want to choose a name that is easy to remember and not too terribly long to type.

ec2-add-keypair gsgkey 

Save the private key returned by the command in a local file. Using your favorite text editor, paste everything between (and including) the -----BEGIN RSA PRIVATE KEY----- and -----END RSA PRIVATE KEY----- lines into your editor and save the key to a file. The file can have any name. However, it makes sense to name the file after the keypair name you have chosen earlier. If the file is named differently from the keypair you will end up launching instances with -k mykey and then accessing the instance with -i yourkey, which may be a bit weird. As indicated by the heading, this is your private key, thus make sure you safe guard it appropriately. On Linux the ssh tools will complain if the key file does not have the proper permissions. Change the permissions of your private key file to be read-write by you, the owner, only.

chown 600 gsgkey 

The public key of your keypair is stored in the EC2 infrastructure. EC2 allows you to have multiple keypairs, to review your existing keypairs use the ec2-describe-keypairs command.

When you launch an instance of an AMI you must specify a keypair name. This selects the public key to be injected into the instance. The key injection occurs through the amazon init script provided by the suse-ami-tools package. This package, as mentioned previously, is already included in the example's config.xml file. Do not forget to include this package when you create your own image descriptions for EC2 or you will not be able to log into your running instances. Additionally you need to activate this service by adding suseInsertService amazon in your config.sh file.

The key injection mechanism needs to access the network. Therefore, you must configure the network when you build your image. Configuration of the network can be accomplished through the overlay mechanism or via commands in config.sh. The network interface of a guest in EC2 is always eth0 and it needs to be configured to use DHCP. In the example the overlay mechanism is used to setup the network configuration.

Another prerequisite to launching an instance in EC2 is knowing the AMI you want to instantiate. As you have already seen, the ec2-describe-images command with the -a option will provide information about all publicly available AMIs. Using the command without the -a option will provide information about your registered images.

The Amazon EC2 infrastructure uses PVGrub (Para-Virtual Grub) to boot instances of an AMI. This allows instances to run the kernel that is part of the AMI, rather than some kernel provided by the Amazon infrastructure. However, an Amazon provided kernel is still required to kick things off and in the startup process PVGrub eventually picks up the /boot/grub/menu.lst file in your image and then boots the kernel specified. Note, that during the boot process you do not have access to a console and thus it makes no sense to have multiple kernel entries in your menu.lst file. Without console access you do not have an opportunity to choose a kernel. Each EC2 region has it's own independent copy of this boot mechanism and the boot mechanism is differentiated between 32 bit and 64 bit. The boot kernels are named with an ID that starts with the TLA (Three Letter Acronym) aki followed by a dash ("-") and a hex number. The Amazon Kernel Image IDs table below provides guidelines for the selection of the boot kernel ID based on Region and image architecture.

The information in the table above was extracted from the Amazon documentation found at: http://ec2-downloads.s3.amazonaws.com/user_specified_kernels.pdf.

KIWI images created for EC2 do not contain a partition table. Therefore, PVGrub images designated by h0 in the manifest name need to be used with the images created by Kiwi.

Specifying the optional ec2region element as part of the ec2config block in your config.xml file instructs KIWI to embed the appropriate PVGrub Kernel ID in the EC2 manifest file of the bundle. Therefore, you can create region specific bundles. KIWI will create a bundle for each specified region. Supported values for the ec2region element are AP-Northeast, AP-Southeast, EU-West, US-East, US-West, and US-West2.

Public images in EC2 already have the boot kernel information embedded. If you build a generic image with EC2 by not specifying the EC2 region you will need to specify the appropriate boot kernel on the command line when you launch an instance of your AMI as shown below. The example uses the x86 Kernel in the US-East region.

ec2-run-instances ami-... --kernel aki-407d9529 -k gsgkey 

To check the state of your instance(s) use the command ec2-describe-instances.

Once the instance state for an instance is shown as running you can log into the instance using ssh as follows:

ssh -i PATH_TO_PRIVATE_KEY root@PUBLIC_IP_OF_YOUR_INSTANCE

The PUBLIC_IP_OF_YOUR_INSTANCE is displayed as part of the output of the ec2-describe-instances command. If you are unable to log in, it is most likely that the security setting for the instance is blocking the ssh access, or that you did not enable the ssh daemon process on startup. Your config.sh script should have a line activating the ssh daemon as follows, suseInsertService sshd. If you do not have this entry in config.sh you will have to rebuild your image. Addressing the problem with port blocking is acomplished using the ec2-authorize command to open port 22 as shown below.

ec2-authorize default -p 22 

The destination directory, /tmp/myec2-result if you followed the commands given above contains a directory ending with -any. This directory contains the AMI bundle created by KIWI with the Amazon tools for use with the S3 storage model. The -any ending of the directory name indicates that this bundle may be deployed to any EC2 region. As described earlier you can use the ec2region element in config.xml to build bundles for specific EC2 regions. If the ec2region element is used the directory name containing the bundle will end with the appropriate region string.

The generated bundle needs to be transfered to Amazon using the ec2-upload-bundle command line tool. This tool is part of the Amazon API tools. Upload the AMI as follows, replacing AWS_Key_ID and AWS_secret_Key_ID with your Amazon key information. Also you may want to choose a different name for your bucket than myImages. If the bucket does not exist in S3 it will be created.

ec2-upload-bundle  -b myImages  -a AWS_Key_ID  -s AWS_secret_Key_ID \
   -m /tmp/myec2/suse-##.#-ec2-guest.ARCH-Version.ec2-any/ \
      suse-##.#-ec2-guest.ARCH-Version.ami.manifest.xml

Where ARCH indicates the architecture to be used for this image, 32 bit (i686) or 64 bit (x86_64), and Version matches the value of the version element in the config.xml file.

After the upload process is complete, register your image with the EC2 infrastructure using the ec2-register command as shown below. The result of the registration process is an AMI ID returned on the command line in the form “ami-” followed by a random key sequence. Use this AMI ID to launch your instance as described in the Using a registered AMI section.

ec2-register myImages/suse-##.#-ec2-guest.ARCH-Version.ami.manifest.xml

This completes the S3 specific setup using the bundle created with KIWI. The next section explains the use of the disk image file created by KIWI to create and EBS backed AMI.

Inside the destination directory, /tmp/myec2-result if you followed the commands given above, you will find a file with no extension. This file is the filesystem image KIWI created and used to create the AMI bundle suitable for use with S3. This filesystem image is the one we need to install/dump onto an EBS volume to have an EBS backed AMI. In order to create the EBS backed AMI it is necessary to work on a running AMI and the local machine that contains the disk image file. Further, the running instance must have the dd_rescue command available. Using the "regular" dd command will result in failure as it does not properly handle sparse images. The SUSE Linux Enterprise AMIs available in EC2 have the dd_rescue command available. For the openSUSE AMIs available in EC2 it is easy to install the dd_rescue command by executing zypper as shown below.

zypper in dd_rescue

The first step in creating an EBS backed AMI is to create a tarball of the filesystem image file. This is necessary to preserve the sparseness properties of the image. Use the -S option as shown below. The following commands are executed on your build machine.

cd /tmp/myec2-result

tar -cSf myImage.tar suse-##.#-ec2-guest.ARCH-Version

The tarball needs to be uploaded to EC2 and unpacked. This implies that the running instance of your chosen AMI needs to have sufficient space to store the tarball and the unpacked tarball. Therefore, it is recommended to create a storage volume as shown below. The command used will return a volume ID that you want to remember and that will be referred to as STORE_VOL_ID in this example. Execute this command on your local machine.

ec2-create-volume -s X -z AV_ZONE

The X is an integer value representing the size of the volume to be created in GB (Giga Bytes). The AV_ZONE value is one of the Amazon availability zones. For all commands presented here the AV_ZONE value must be the same. Possible values for AV_ZONE are obtained with the ec2-describe-availability-zones

The next step is to launch an instance of your liking in EC2. This instance will be used to create the EBS volume that will eventually function as the backing store for your AMI, therefore this instance must have the dd_rescue as described earlier. Launching the instance is accomplished with the ec2-run-instances command executed on your local machine.

ec2-run-instances -k gsgkey -n 1 -g SECURITY_GROUP_NAME -t INSTANCE_TYPE -z AV_ZONE ami-.....

The SECURITY_GROUP_NAME is optional, but it is best to have a group setup that allows ssh access by default to avoid having to open the port all the time as outlined earlier. The INSTANCE_TYPE specified with the -t depends on the architecture of your image, for example a 64 bit AMI cannot run in a small (m1.small) instance type. The ec2-run-instances command returns the instance ID, and you want to remember this ID. This ID will be referred to as INST_ID in this example.

Wait until the instance is running, check the status with the ec2-describe-instances command. Once the instance is indicated as running attach the previously created volume to your instance, by executing the ec2-attach-volume command on your local machine.

ec2-attach-volume STORE_VOL_ID -i INST_ID -d /dev/sdf

The chosen device specified with the -d is arbitrary, however, you do obviously not want to pick a device node that is already in use.

In a different shell login to the running instance as shown in the Using a registered AMI section.

Check that the storage volume is attached using the ec2-describe-volumes command on your local machine. Once the volume is attached the status will be shown as in use.

With the EBS storage volume attached to your instance create a filesystem on the volume. There is no need to partition the volume. In the EC2 instance that you previously logged into execute the mkfs command.

mkfs -t ext3 /dev/sdf

Once the filesystem creation is complete mount the volume.

mount dev/sdf /mnt

With the storage volume attched, formated and mounted you now have sufficient space to transfer the image tarball you created earlier. From the destination directory on your local machine that contains your tarball initiate an ftp session to your EC2 instance.

sftp -i PATH_TO_PRIVATE_KEY root@PUBLIC_IP_OF_YOUR_INSTANCE

sftp>cd /mnt

sftp>put myImage.tar

sftp>bye

While the image is transfering you can create a new EC2 volume. This new volume will become the backing store for your new AMI and be referred to as VOL_ID in this example. Use the ec2-create-volume command on your local machine as shown previously. This time you want to make sure that the specified size matches the size of your filesystem image file.

ec2-create-volume -s Y -z AV_ZONE

It is good practice to use the size element in your config.xml file when creating EC2 images. Using the size element ensures that you have additional space on your root volume and that you can match the volume size exactly to your image size.

As previously, wait for the volume to be created and check the status with the ec2-describe-volumes command. When the volume creation process is complete, attach the new volume to your running instance by executing the ec2-attach-volume command on your local machine.

ec2-attach-volume VOL_ID -i INST_ID -d /dev/sdg

Wait until the status for the volume changes to in use before proceeding.

By now the upload of your image tarball has probably completed and you can unpack the image on your storage volume in your EC2 instance. In the shell on your EC2 instance unpack the tarball as shown below.

cd /mnt

tar -xSf myImage.tar

After the unpack operation completes you can dump the filesystem image to your EBS volume that will be your backing store for your AMI, this volume if attached to /dev/sdg in this example. In the shell on your EC2 instance execute te dd_rescue command as shown below.

dd_rescue -a /mnt/suse-##.#-ec2-guest.ARCH-Version /dev/sdg

The -a option instructs dd_rescue to handle the sparse file properly. You have to wait until the dump process is complete before proceeding to the next step.

With the dump process complete you can now unmount the storage volume from your EC2 instance by executing the following in the shell of your running instance.

umount /mnt

You can also exit the shell in your running EC2 instance as all remaining commands are executed on your local machine.

You must detach the volume that is intended as your AMI backing store from your running instance. Detaching and deleting the storage volume as well as terminating the running instance are optional.

ec2-detach-volume -i INST_ID -d /dev/sdg VOL_ID

ec2-detach-volume -i INST_ID -d /dev/sdf STORE_VOL_ID

Prior to shutting down the instance or deleting the storage volume, wait until the detach operation has completed. This is indicated by the available status in the output of the ec2-describe-volumes command.

ec2-delete-volume STORE_VOL_ID

ec2-terminate-instances INST_ID

Your next step is to create a snapshot of the EBS volume that contains your image. The ec2-create-snapshot command returns a snapshot ID that you want to remember and will be referred to as SNAP_ID in this example.

ec2-create-snapshot -d A_SHORT_DESCRIPTION VOL_ID

The process of creating the snapshot will take a while and depends on the size of your volume. Check the status of the snapshot creation process using the ec2-describe-snapshots command. When the process is complete the status will change to 100%.

Once the snapshot is complete you can register it as an AMI with the EC2 infrastructure using the ec2-register command.

ec2-register -d A_DESCRIPTION -n A_NAME -s SNAP_ID -a ARCH --kernel BOOT_KERNEL_ID

The ec2-register command will return the AMI ID that you can then use to launch your instance.

The process of creating an EBS backed AMI is a bit tedious. If you create EBS backed AMIs more often it might be well worth your time to script this process using the Amazon REST API. The EC2 tools provided by Amazon that would by name imply to implement this process (ec2-import-instance and ec2-upload-disk-image) do not handle Linux based disk images at the time of this writing.