Creating CentOS server images with Packer

The CentOS images are not available on the AWS Quick Start tab.

CentOS publishes official images on the AWS Marketplace, but you need to subscribe to the image to be able to launch it with an automation software, like Terraform.

Find the latest available CentOS image in the AWS Marketplace

  1. Execute this command to display the list of available images 
    aws --region us-east-1 ec2 describe-images --owners aws-marketplace --filters Name=product-code,Values=aw0evgkw8e5c1q413zgy5pjce
  2. Select the latest AMI from the list. The images are NOT ordered by date!

Accept the terms

In every account, you want to launch the image, you have to accept the license agreement terms

If you try to launch the image before you subscribe to it, the error message is displayed

Error launching source instance: OptInRequired: In order to use this AWS Marketplace product you need to accept terms and subscribe. To do so please visit http://aws.amazon.com/marketplace/pp?sku=…
amazon-ebs: status code: 401, request id: []

  1. Follow the link in the error message to http://aws.amazon.com/marketplace/pp?sku=…
  2. Click the Continue to Subscribe button
  3. Click the Accept Terms button
  4. The next page should display
  5. Once the new image has been created, share it with all the accounts you use.

SocketError: Error connecting to … Name or service not known

In an enterprise environment, the company usually operates its own DNS servers.

When a Linux instance launches in AWS, the DNS settings only contain the AWS DNS server. If the company DNS server settings are applied with Chef, during the first Chef Client run those settings do not take effect.

When we reference an internal DNS entry during the first Chef Client run we get the error message:

 

SocketError: Error connecting to https://INTERNAL_ADDRESS.net/… – Failed to open TCP connection to INTERNAL_ADDRESS.net:443 (getaddrinfo: Name or service not known)

  1. There are two solutions to handle this:
  2. Use the IP address of the internal server or load balancer in the URL ( switch from HTTPS to HTTP, because the SSL certificate will not work with the IP address),
  3. Set the DNS servers of the Linux server with Packer when you generate the base image.

The specified version of the NuGet package is not found in Artifactory

The “chocolatey_package” Chef resource can install NuGet packages from Artifactory.

Artifactory is inconsistent in case sensitivity when an application is searching for a NuGet package. When we specify the package ID only, the search is not case sensitive. If the package is called “GoogleChrome” and we search for “googlechrome” the NuGet package is found in Artifactory.

chocolatey_package 'googlechrome' do
  source 'devops-chocolatey'
  options "--allow-empty-checksums --ignore-package-exit-codes"
end

When we specify the version of the package, the search becomes case sensitive in Artifactory. The “chocolatey_package” Chef resource automatically converts the package IDs to lowercase, even if we spell the package ID the same as it is in Artifactory, so “GoogleChrome” can never be found in Artifactory when we specify the NuGet package version.

chocolatey_package 'GoogleChrome' do
  version '66.0.3359.18100'
  source 'devops-chocolatey'
  options "--allow-empty-checksums --ignore-package-exit-codes"
end

The following error message is misleading. In this case, the version was in Artifactory, but the spelling of the package ID is not all lower case in Artifactory.

googlechrome not installed. The package was not found with the source(s) listed.

If you specified a particular version and are receiving this message, it is possible that the package name exists but the version does not.
Version: “66.0.3359.18100”
Source(s): “http://artifactory….”

When we internalize the “googlechrome” Chocolatey package, the package ID in the NuGet package will be spelled as “GoogleChrome”. The “internalize” process downloads the NuGet package from the Chocolatey server, downloads the necessary installer files from the source URL specified in the NuGet package, and creates a new NuGet package that contains all necessary files for the software installation. To have access to these packages any time, even when the Chocolatey package is no longer available at Chocolatey.org, we can upload them to Artifactory or other NuGet repository.

To solve the case sensitivity issue, currently, the only solution is to unzip the package, change the spelling of the ID to lower case and re-create the package again.

  1. Install Chocolatey on your Windows workstation. See Install Chocolatey
  2. Open a Windows command prompt as Administrator
  3. Internalize the package 
    choco download googlechrome --internalize
  4. Unzip the NuGet package
    1. Right-click the .nupkg file and select 7-Zip, Extract to “…”
    2. Open the .nuspec file in a text editor
    3. Change the ID to all lower case 
      <id>googlechrome</id>
    4. Save the .nuspec file
  5. Re-create the NuGet package
    1. Delete the existing _rels folder, the package creation will recreate it with updated information
    2. Right-click the .nuspec file and select Compile Chocolatey Package…
  6. Upload the NuGet package to Artifactory.

 

Scale inserted videos with the correct aspect ratio in Adobe Premiere CC

When we insert a video clip into our timeline that has the same height, but different width than our project, Adobe Premiere CC distorts the inserted clip.

My project is 1920 x 1080, the inserted clip is 1440 x 1080. When I insert the narrower clip into my timeline, Adobe Premiere stretches the inserted clip horizontally.

To set the correct aspect ratio

  1. Select the inserted clip in the timeline
  2. In the upper left region select Effect Controls
  3. Expand the Motion element
  4. Uncheck the Uniform Scale checkbox
  5. Set the Scale Width to the appropriate value, in my case it was 75% (1440 / 1920 )

Copy files between Windows and Linux computers

There are many tools to copy files between Windows and Linux computers, I have found this method the simplest.

It does not require any software installation on the Windows machine, and only one package installation on the Linux machine.

Share a folder on the Windows machine

Share a folder on the Windows machine and allow access to it for a user. If the Windows computer is in the Windows domain, the domain user does not have to be a member of any security group on the Windows machine.

If you copy files from Windows to Linux, make the folder read-only for the user. If you copy files to the Windows machine, allow write access to the folder for the user.

Set up the Linux machine

  1. Install the cifs-utils on the Linux machine
    1. On Red Hat, CentOS, and Amazon Linux 
      sudo yum install cifs-utils
    1. On Ubuntu 
      sudo apt-get install cifs-utils

Mount the shared Windows folder on the Linux machine

  1. On the Linux machine create a directory to mount the Windows folder to 
    mkdir /tmp/windows
  2. Mount the Windows share 
    sudo mount.cifs '\\WINDOWS_SERVER_IP\attachments' /tmp/windows -o domain=MY_DOMAIN,username=MY_USERNAME,password=MY_PASSWORD,vers=1.0

    First, you will be asked for the root password on the Linux machine.
    If you do not specify your password in the line above, you will be also asked to enter your password on the Windows machine.

Access the Windows share

  1. On the Linux machine navigate to the mount directory 
    cd /tmp/windows
  2. List the files of the Windows share 
    ls -al

Troubleshooting

If you get the error message when you issue the mount command

mount error(16): Device or resource busy

try to unmount (umount !) the share first and try the mount again

umount /tmp/windows

 

Edit the HKEY_CURRENT_USER Windows Registry keys of another user

The user-specific settings in the Windows registry are stored under the HKEY_CURRENT_USER key. If you open the Regedit.exe application the HKEY_CURRENT_USER key contains the settings for your user account.

To access the registry keys of another user we need to

Find the Security ID of the user

  1. In Regedit navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\hivelist
  2. The key lists the Security IDs and usernames
  3. Save the Security ID of the user.

Another Security ID list location:

The partial list of the Security IDs is also available at
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList

Not all user profiles are listed here.

Click the Security ID folders on the left to see the username in the ProfileImagePath.

Open the user-specific registry keys

  1. In Regedit navigate to the HKEY_USERS key
  2. Select the Security ID of the user you are looking for
  3. The user-specific values are under that key

 

“Ran out of time waiting for the server with id” with Windows Server 2016 in Chef Test Kitchen

AWS changed how Windows Server EC2 instances send messages during boot.

Windows Server 2012 R2 AWS EC2 instances sent the “Windows is ready” message every time those became available after boot.

When a Windows Server 2016 AWS EC2 instance launches, it only sends the “Windows is ready” message during the first boot. If you create your custom AMI with Packer, the first boot happens during the Packer AMI creation, so Chef Test Kitchen does not receive the expected message and the process times out with the message

>>>>>> Ran out of time waiting for the server with id [i-… to become ready, attempting to destroy it
EC2 instance <i-…> destroyed.

To configure Windows Server 2016 instances to send the expected “Windows is ready” message during every boot, add a PowerShell line to your Packer template that creates your custom AMI:

{
  "type": "powershell",
  "pause_before":"2s",
  "elevated_user": "MY_ADMIN_USER",
  "elevated_password": "MY_ADMIN_PASSWORD",
  "inline": [
    "C:\\ProgramData\\Amazon\\EC2-Windows\\Launch\\Scripts\\SendWindowsIsReady.ps1 -Schedule"
  ]
},

This code creates a scheduled task to execute the configuration script on the instance.

More information is at https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ec2launch.html#ec2launch-sendwinisready

 

Test Chef cookbooks locally on a virtual workstation

When you use a virtual machine to write your Chef cookbooks you may want to test them locally with Vagrant.

This nested virtual machine cannot use a 64 bit operating system, because to run a 64 bit virtual machine, the host computer’s CPU has to provide the CPU Extensions. Currently only physical CPUs can provide the CPU Extensions, no virtualization software can do that.

The Chef Test Kitchen uses Vagrant to launch test instances. In the background Vagrant uses the VirtualBox engine, so your virtual workstation needs to have Vagrant and VirtualBox installed.

To make sure TestKitchen uses the 32 bit operating system first we will set up the 32 bit virtual machine in Vagrant.

Install the necessary software

  1. Install the Chef Development Kit
  2. Install VirtualBox
  3. Install Vagrant

Create the 32 bit virtual machine

  1. Open a terminal window
  2. Initialize the vagrant virtual machine. Vagrant will download the OS image and start the virtual machine. 
     vagrant init hashicorp/precise32
  3. Stop the virtual machine 
    vagrant halt
  4. Make sure all Vagrant virtual machines are off 
     vagrant global-status

    The result looks like this:

    id name provider state directory
----------------------------------------------------------------------------------------------------------------------------
21590cd default virtualbox running C:/Users/interview/Documents
  5. Shut down the running virtual machines. Add the id of the virtual machine to any command to administer it from any directory. 
    vagrant halt 21590cd

Set up the Vagrant connection in the Chef cookbook for Test Kitchen

  1. Navigate to cookbook folder
  2. Edit the .kitchen.yml file:Set the network port forwarding in the driver section: 
    driver:
  name: vagrant
  network:
    - ["forwarded_port", {guest: 2200, host: 2222}]
    - ["private_network", {ip: "127.0.0.1"}]

platforms:
  - name: hashicorp/precise32
  3. Test the cookbook 
    kitchen converge hashicorp-precise32