Creating CentOS server images with Packer

The CentOS images are not available on the AWS Quick Start tab.

CentOS publishes official images on the AWS Marketplace, but you need to subscribe to the image to be able to launch it with an automation software, like Terraform.

Find the latest available CentOS image in the AWS Marketplace

  1. Execute this command to display the list of available images
    aws --region us-east-1 ec2 describe-images --owners aws-marketplace --filters Name=product-code,Values=aw0evgkw8e5c1q413zgy5pjce
  2. Select the latest AMI from the list. The images are NOT ordered by date!
  3. If you try to launch the image before you subscribe to it, the error message is displayed

    Error launching source instance: OptInRequired: In order to use this AWS Marketplace product you need to accept terms and subscribe. To do so please visit http://aws.amazon.com/marketplace/pp?sku=…
    amazon-ebs: status code: 401, request id: []

  4. Follow the link in the error message to http://aws.amazon.com/marketplace/pp?sku=…
  5. Click the Continue to Subscribe button
  6. Once the new image has been created, share it with all the accounts you use.

“Ran out of time waiting for the server with id” with Windows Server 2016 in Chef Test Kitchen

AWS changed how Windows Server EC2 instances send messages during boot.

Windows Server 2012 R2 AWS EC2 instances sent the “Windows is ready” message every time those became available after boot.

When a Windows Server 2016 AWS EC2 instance launches, it only sends the “Windows is ready” message during the first boot. If you create your custom AMI with Packer, the first boot happens during the Packer AMI creation, so Chef Test Kitchen does not receive the expected message and the process times out with the message

>>>>>> Ran out of time waiting for the server with id [i-… to become ready, attempting to destroy it
EC2 instance <i-…> destroyed.

To configure Windows Server 2016 instances to send the expected “Windows is ready” message during every boot, add a PowerShell line to your Packer template that creates your custom AMI:

{
  "type": "powershell",
  "pause_before":"2s",
  "elevated_user": "MY_ADMIN_USER",
  "elevated_password": "MY_ADMIN_PASSWORD",
  "inline": [
    "C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\\Scripts\\SendWindowsIsReady.ps1 -Schedule"
  ]
},

This code creates a scheduled task to execute the configuration script on the instance.

More information is at https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ec2launch.html#ec2launch-sendwinisready

 

Debugging a Packer build

Packer can configure a server instance and create an image of it.

If you need to examine what Packer is doing, you can run Packer in debug mode and RDP into the instance to view files and other settings.

Open the firewall

  1. In the “builders” element add a line to the Packer template to execute a configuration file:
    "user_data_file": "bootstrap-aws.txt",
  2. Create the bootstrap-aws.txt file in the same directory where your Packer template is located,
  3. Add lines to the bootstrap-aws.txt file to create an administrator user, enable the Remote Desktop connection, and open the firewall for the RDP port:
    # Administrator user
    cmd.exe /c net user /add MY_USERNAME MY_PASSWORD
    cmd.exe /c net localgroup administrators MY_USERNAME /add
    
    # RDP
    cmd.exe /c netsh advfirewall firewall add rule name="Open Port 3389" dir=in action=allow protocol=TCP localport=3389
    cmd.exe /c reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server" /v fDenyTSConnections /t REG_DWORD /d 0 /f

Build in debug mode

  1. Execute the build with the -debug option:
    packer build -debug win2012.json

Open the RDP port in the temporary security group

During the build, Packer will prompt you to press enter before major steps.

  1. When the IP address of the instance is displayed on the screen, search for the temporary security group’s name by
    Creating temporary security group for this instance
  2. In the AWS console search for the security group and add a new inbound rule to open port: 3389 for the Remote Desktop Connection:

Connect to the instance

  1. Find the IP address of the instance on the screen,
  2. Start a Remote Desktop connection to the instance,
  3. Use the administrator username and password specified in the bootstrap-aws.txt file.

Log locations

  1. The EC2 log file on the instance is at
    C:\Program Files\Amazon\Ec2ConfigService\Logs\Ec2ConfigLog.txt

 

amazon-ebs: Error waiting for SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

When you launch a Linux AWS EC2 instance with Terraform or create a Linux AWS image with Packer, one of the following errors are displayed:

amazon-ebs: Error waiting for SSH: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

aws_instance.default: 1 error(s) occurred:
* ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

Possible causes

Incorrect username

In the Packer template the “ssh_username”, in the Terraform script the “user” attribute contains the username to connect to the instance. It is important to use the correct username for each operating system. For the list of correct usernames for each operating system, see Create a server image with Packer

Wrong SSH key

Make sure you use the correct SSH key to connect to the instance. In the Terraform file, the “key_name”  attribute of the “aws_instance resource” specifies the SSH key to launch the instance with, and the “private_key” attribute of the “connection” contains the key itself as a string.

 

Create a server image with Packer

Packer is a free, open-source application from Hashicorp. It can generate a server image based on an existing one, and configure it for your special needs. You can use the generated image when you launch a server instance in the cloud or on your local workstation.

Install Packer

Packer script

Packer reads a .json file to generate the new server image.

AMI ID

To find the latest AMI IDs log into the Amazon AWS Console, select EC2 and click the Launch button. On the left side click Quick Start to see the images for

  • Amazon Linux
  • SUSE Linux
  • Red Hat Enterprise Linux
  • Ubuntu Server
  • Microsoft Windows Server

For CentOS images, see the “Official CentOS Linux : Public Images” section at  https://wiki.centos.org/Cloud/AWS The AMI IDs are organized by region.

Username

It is important to use the correct ssh_username for each operating system:

Linux

RHEL: ec2-user
Amazon Linux: ec2-user
CentOS: centos
Ubuntu: ubuntu

Windows

any “winrm_username” and “winrm_password”

The PowerShell script in the user data file, specified in the “user_data_file” attribute, sets the initial username and password with the

cmd.exe /c net user /add ...

command.

The Administrator password is set with the

cmd.exe /c net user Administrator ...

command.

IMPORTANT!!!

For Packer to be able to connect to the server, set the same username and password in the  “winrm_username” and “winrm_password”, “elevated_user”, and “elevated_password” attributes of the .json file.

AMI Name

To allow the Chef Kitchen EC2 driver to read the OS version of the image, include the version in the “Name” as follows.

The Test Kitchen Git repository has the following at https://github.com/test-kitchen/kitchen-ec2

Note that the image_search method requires that the AMI image names be in a specific format. Some examples are:

  • Windows-2012
  • Windows-2012r2
  • Windows-2012r2sp1
  • RHEL-7.2

It is safest to use the same naming convention as used by the public images published by the OS vendors on the AWS marketplace.

Sample acceptable names are

  • my_windows-2012r2_base-01
  • my_amzn-ami-hvm_base-01

To find the name of the AMI

  1. In the AWS Console select EC2, and click the Launch Instance button
  2. On the left side select Quick Start, and find the AMI ID of the image you want to start with.
  3. Navigate back to the EC2 console, select AMIs, in the drop-down select Public images
  4. Search for the AMI by ID
  5. Make sure you copy the name of the AMI with the same ID.

 

Generate the server image with Packer

  1. Open a Bash window,
  2. Navigate to the folder of the Packer JSON script,
  3. Execute the following command. Packer will read the default AWS credentials from  ~/.aws/credentials file on your Macintosh or Linux workstation. On Windows, the file is at C:\Users\YOUR_USER_NAME\.aws\credentials.
    packer build MY_PACKER_SCRIPT.json
    1. To specify the AWS keys in the command line
      packer build -var 'aws_access_key=<access key>' -var 'aws_secret_key=<secret key>' MY_PACKER_SCRIPT.json
  4. The command window will display the ID of the generated image, or you can find it by name in the EC2 section of the AWS console under AMIs.

Share the generated server image with other cloud accounts

If you work in multiple cloud accounts you need to share the generated server image with other accounts

AWS

  1. Log into the AWS account you have used to generate the server image,
  2. On the left side of the EC2 section select AMI and find the new image by name of ID,
  3. On the Permissions tab click the Edit button,
  4. Make sure the Private radio button is selected if you don’t want to share the image publicly,
  5. Enter the account number of the account you want to share the image with (see Find the AWS account number),
  6. Check the Add “create volume” permissions… checkbox,
  7. Click the Add Permission button,
  8. When you have added all accounts to share with, click the Save button.

 

Find the AWS AMI that a RightScale server template is using

When you use RightScale to launch servers in the cloud, you want to use the same base image to test your Chef cookbooks in Test Kitchen.

Packer by Hashicorp is a utility to create custom server images based on cloud images. You supply the image ID and other configuration parameters to create a new custom image.

To find the AMI ID of the base image of the RightScale Server Template

  1. Open the RightScale user interface in your browser,
  2. Select the RightScale account where the server template was created,
  3. In RightScale Cloud Management select  Design / ServerTemplates,
  4. Select the server template,
  5. Select Images,
  6. Select the MultiCloud image,
  7. Select Clouds,
  8. Select the image in the region you are working in,
  9. The Resource UID is the AMI ID of the image.

Keep multiple versions of applications on Macintosh

Most of the DevOps tools are still in beta versions, many times the new version is not compatible with your existing scripts or have an error that stops your scripts working. To be able to keep multiple versions of the applications and easily switch between them, create symbolic links and point to the version you want to execute.

Create version-specific locations

Create a folder for your optional applications.

mkdir /opt

Set you as the owner of the “opt” directory and its children.

YOUR_GROUP should be on:

  • MacOS: wheel
  • Ubuntu: your username 
sudo chown -R YOUR_USER_NAME:YOUR_GROUP /opt

Set the security of the folder.

sudo chmod 755 /opt

We will create a directory structure to place each version of the application into its own folder. This example uses Packer.

Create a symbolic link to point to the application in the appropriate version folder. These are the examples for Packer, Terraform, and Vagrant. Notice, that the Vagrant application is in the “bin” directory, so the symbolic link has to point there.

If there is an existing symbolic link that points to the latest version of the application, rename it to be able to quickly return to the prior version of the application. This can come handy when we need to destroy instances created with the prior version of Terraform.

cd /opt/terraform
mv terraform terraform_v_0.9.11
cd /opt/packer
mv packer packer_v_1.0.4

Create a new symbolic link that points to the latest version of the application.

# For Packer
cd /opt/packer
ln -s /opt/packer/packer_1.2.0/packer packer

# For Terraform
cd /opt/terraform
ln -s /opt/terraform/terraform_0.10.0/terraform terraform

# For Vagrant
cd /opt/vagrant
ln -s /opt/vagrant/vagrant_1.9.5/bin/vagrant vagrant #The vagrant application is in the bin folder

Add the main application folder to the path in the config file of your terminal

  • for iTerm2 with zterminal: ~/.zshrc
  • for other terminals: ~/.bash_profile
PATH=/opt/packer:$PATH
PATH=/opt/terraform:$PATH
PATH=/opt/vagrant:$PATH

Vagrant

For Vagrant installation see Vagrant


To change the version you want to execute

  • Delete the symbolic link,
    cd /opt/packer
    rm packer
  • Create a new symbolic link pointing to the other version of the application as above.