Wednesday, October 27, 2010

New Amazon EC2 build format

Regular users of SUSE Studio have probably discovered the new Amazon Elastic Compute Cloud (EC2) build format that we sneaked in last week. It was available for openSUSE 11.3 only, and today we released it for SUSE Linux Enterprise Server (SLES) 11 SP1 as well. Support for SLES 10 SP3 will be available next week.

This means you can now easily build customized server appliances for Amazon EC2 in SUSE Studio! Together with Amazon's recent free usage tier announcement, there's hardly a better time to try out EC2.

Introducing Amazon EC2

If you're new to Amazon EC2, it's a web service hosted by Amazon that provides dynamically scalable computing capacity. It is cloud computing and sometimes also referred to as Infrastructure as a Service (IaaS). EC2 allows you to rent reliable computing resources (eg. CPU, memory, disk storage) in a highly flexible and affordable manner, with prices starting at just several cents per hour per instance. These resources can be quickly scaled up or down on demand, reducing or even eliminating the need for companies to buy and maintain expensive server farms in-house. Before we proceed with building an EC2 image in SUSE Studio, there's some Amazon specific terminology we should clarify. An appliance must first be created as an Amazon Machine Image (AMI) before it can run in EC2. There are two main types of AMIs:

  • Amazon Simple Storage Service (S3) backed AMIs
  • Amazon Elastic Block Storage (EBS) backed AMIs

There are several key differences between these two AMI types, though we won't go into the details in this post. What's important to note now is that the EC2 images built by SUSE Studio can technically be used to create both AMI types. To keep things simple, however, the current AMI creation script only supports S3-backed AMIs for openSUSE based images, and EBS-backed AMIs for SLES based images.

Building Amazon EC2 images with SUSE Studio

The easiest way to get started is to find an existing EC2 image in SUSE Gallery, such as this LAMP Server appliance. But let's walk through the steps to create one from scratch. First login to SUSE Studio and create a new appliance based on openSUSE 11.3 JeOS:

Choose your preferred architecture (note that 64-bit S3-backed AMIs are slightly more expensive to run), change the default appliance name, and hit "Create appliance". You will now be in the appliance editor interface where you can perform the usual software selection and configuration, but we'll skip that for now and jump to the Build tab. Change the default image format to "Amazon EC2 image":

There's now an alert in the left sidebar, telling you to add the Amazon EC2 software repository before building:

Simply click on the "Add the 'Amazon EC2 Updates' repository" link there, which will then add the required repository in the software tab:

Once you're satisfied, return to the Build tab and hit "Build". It usually takes only about five minutes (!!) for the build to complete:

Now we can use this image to create an AMI in EC2.

Creating an S3-backed AMI from the Studio-built image

We are working on a minimal user interface to do this directly from SUSE Studio, so meanwhile you have to download the EC2 image and execute the bundled AMI creation script. To do this, click on the "Download" link in the Build tab to download the EC2 image archive. Extract the archive and you'll find the create_ami.sh script and README files along with the EC2 image:

If this is your first time using the Amazon AMI tools, you'll need to do some configuration. The steps are explained in the README file, but I'll repeat them here with a bit more details. First download and install the Amazon EC2 AMI and API tools. Now add your Amazon Web Services (AWS) account information in the following environment variables:

  • $AWS_USER_ID. AWS user ID (eg. 123456789012)
  • $AWS_ACCESS_KEY. AWS access key ID (eg. ABCDEFGHIJKLMNOPQRST)
  • $AWS_SECRET_KEY. AWS secret access key (eg. abcdefghijklmnoprqrtuvwxyzabcdv)
  • $EC2_CERT. Path to EC2 X.509 certification (eg. ~/cert-aws.pem)
  • $EC2_PRIVATE_KEY. Path to EC2 private key (eg. ~/pk-ec2.pem)
We recommend setting these variables in your ~/.profile file so that you don't have to set them up manually each time, eg: export AWS_USER_ID=123456789012 export AWS_ACCESS_KEY=ABCDEFGHIJKLMNOPQRST export AWS_SECRET_KEY=abcdefghijklmnoprqrtuvwxyzabcdv export EC2_CERT=~/cert-aws.pem export EC2_PRIVATE_KEY=~/pk-ec2.pem Once you have all that configured (which you will only need to do once), you can finally create your AMI. AMIs are specific to each Amazon region (eg. us-west-1, us-east-1, eu-west-1, ap-southeast-1), so run the create_ami.sh script with the desired region option, eg. create_ami.sh --region eu-west-1:


Launching new instances of your Studio-built AMI

You can launch as many instances of an AMI as you like. Note that the AMI (referenced by the AMI ID) exists only in the region that it was created in. One way to launch new instances of your AMI is with the command line tools, eg: ec2-run-instances ami-6a380d1e -t m1.small --region eu-west-1 -k jamestyj-eu. You'll want to create your SSH keypair in Amazon first if you don't already have one (specified by the -k option) as password based authentication is automatically disabled (hence the root and user passwords configured in Studio no longer apply). Wait for the instance to boot and check for it's DNS name with ec2-describe-instances, and then you can ssh into it once it fully boots up, eg: ssh -i ~/ec2-eu root@ec2-46-51-133-248.eu-west-1.compute.amazonaws.com. Don't forget to terminate the instance when you're done (eg. ec2-terminate-instances).

To help manage your Amazon EC2 and S3 accounts, we recommend using the ElasticFox and S3Fox Firefox plugins. These plugins have an easy to use graphical user interface that allow you to quickly have an overview of your Amazon accounts, launch new instances, terminate existing ones, etc. Here's a screenshot of ElasticFox:

And one of S3Fox, which is particularly useful for cleaning up any leftover image files from de-registered S3-backed AMIs:

Credits: The ElasticFox and S3Fox screenshots were taken from here.

SUSE Linux Enterprise Server in Amazon EC2

Amazon EC2 has recently collaborated with SUSE to provide supported instances of SUSE Linux Enterprise Server (SLES) running in EC2. These instances have automatic maintenance and access to security updates, patches, and bug fixes.

Creating an EBS-backed AMI

SLES based EC2 instances are currently available as EBS-backed AMI only (otherwise it will not be able to get access to the updates). The creation of these EBS-backed AMIs is very similar to S3-backed ones (as it's all handled by the bundled script), except that the script is called create_ebs_ami.sh. This blog post is getting a bit too long, so we'll go into the details in a later post.


Update 1 (Oct 27 21:26:18 CEST): A number of users have reported problems with building 32-bit PAE-enabled EC2 images. Disabling PAE (Configuration -> Appliance -> Disk and memory -> All formats) should workaround the problem. We are working on a fix.

Update 2 (Nov 01 09:55:18 CEST): The 32-bit PAE-enabled EC2 building issue is now fixed.

17 comments:

  1. Great to have this feature. I have been looking into using SUSE Studio in the future with Amazon...now it appears that will be a drop in the bucket.

    Thanks!

    ReplyDelete
  2. Look forward to being able to deploy Suse studio images directly onto EC2. Do you have an example system(s) installed on EC2?

    ReplyDelete
  3. Neil: We're working on direct EC2 deployment from Studio, will be available in the near future. An example of a EC2 system image would be the LAMP appliance mentioned in the blog post => http://bit.ly/cDl58n.

    ReplyDelete
  4. Will be great to have ability to create OpenSuse ebs AMIs
    right now to play with "AWS Free Usage Tier" (which free for t1.micro\ebs) I needed to create both sles and opensuse images and use fixed script from sles to create ebs based AMI

    ReplyDelete
  5. Also, just a note - by default images from Build Studio is based on Micro SUSE Linux Instance and not compatible with Amazon free tier (i.e. you will be charged).

    ReplyDelete
  6. I'm also looking forward to EBS for OpenSUSE so I can try things out with the new Free Usage Tier.

    ReplyDelete
  7. The script create_abs_ami.sh have error in line 199 root volume parameter has the number 23 and not 24. Must be:root_volume=echo $output | cut -d' ' -f23

    ReplyDelete
  8. >>Девочки, уберите шахматы!
    ну может амазон поменял вывод ?
    и не забудь, что так получится платный имидж.

    If you looking for OpenSuse for Amazon Free Tier :
    https://forums.aws.amazon.com/thread.jspa?messageID=202505#202505

    You can check public AMI with OpenSUSE 11.3 I created - ami-861227f2 (eu-west-1)

    It is image built over Build Studio with JeOS appliance + zypper predinstalled + 5GB.
    AMI by self uploaded via ubuntu, so its free and free tier ready

    Also, ami does not have all drivers (in my case ext4), so after starting you needed to do
    zypper install kernel-ec2-extra
    zypper install kernel-ec2-base

    To have console based yast packager
    zypper install yast2-ncurses-pkg
    zypper install yast2-packager


    Also, if you concerned about security you probably needed to build own AMI.
    So please note - by default images from Build Studio is based on SUSE Linux Instance and not free , if you want to create one you needed start ubuntu (or other free AMI) and replace root partition with image from Build Studio Appliance you created

    ReplyDelete
  9. Public openSUSE 11.3 EBS AMIs are now available, so you can take advantage of Amazon's free tier.

    Please refer to http://blog.susestudio.com/2011/02/public-opensuse-113-amis.html for details.

    ReplyDelete
  10. @Девочки, уберите шахматы!: That error should be fixed already. Please send us a bug report via the feedback link in Studio if you are still having problems, or better yet, a patch to https://github.com/jamestyj/susestudio-ec2.

    ReplyDelete
  11. Update: https://github.com/jamestyj/susestudio-ec2 has been moved to https://github.com/susestudio/susestudio-ec2.

    ReplyDelete
  12. Has support for SLES10.4 been dropped?

    ReplyDelete
    Replies
    1. Charles, SLES10 SP4 is currently not supported in EC2 [1], only SLES10 SP3. Can you move your image to SLES11 instead?

      [1] http://aws.amazon.com/search?searchQuery=SUSE+Linux+Enterprise+Server+10&searchPath=amis&x=0&y=0

      Delete
    2. I'm specifically looking at OES, and if I understand correctly, OES11 is the only SLES11 solution available right now. And OES11 is only 64-bit right? (if this is wrong that'd be great). Which means m1.large or bigger is required. From a cost perspective and due to our small server demands, m1.small (32-bit only) would be preferred.

      Delete
    3. For comparison, a reserved small (32bit) instance with a 1-year heavy utilization linux (non suse, byol?) contract can run ~$490 for the whole year just for the compute time. A large (64bit) is going to run ~$1,900. Using SUSE or Windows pricing makes the gap that much larger.

      Delete
  13. @James Tan

    What we're really looking at is OES, and OES11 is 64-bit only. That means a Small EC2 instance (we don't have large demands on our server), which is only available as 32-bit, cannot be used.

    As a side note, Susestudio only allows you to make a SLES-10-SP4 server, and no longer seems to have an SP3 (EC2 compatible) option.

    ReplyDelete
  14. @Charles - You mean OES2? That's pretty old. Can you still buy licenses for that and is it still supported?

    If you really want to stick to SLES10 SP3 you can find one in the Gallery and clone from it. Not recommended though due to support reasons.

    SUSE itself currently does not support SLES10 SP4 in EC2, so until that happens we can't support it in Studio. The focus is naturally on SLES11 (soon SP2) since SLES10 has been around for a long time.

    ReplyDelete

 
© 2013 SUSE