Failed to complete #create action: [undefined method `version’ for nil:NilClass] on default…

When you execute kitchen converge to launch an EC2 instance in AWS with Chef Test Kitchen, you get the error message:

>>>>> ——Exception——-
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>> Failed to complete #create action: [undefined method `version’ for nil:NilClass] on default-windows-2012r2
>>>>>> ———————-
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose –all` for configuration

The kitchen-ec2 driver tries to read the operating system version from the name of the image, but it cannot find it.

When you create your own AMI (Amazon Machine Image), make sure the version of the operating system is in the name.

The Test Kitchen Git repository has the following at https://github.com/test-kitchen/kitchen-ec2

Note that the image_search method requires that the AMI image names be in a specific format. Some examples are:

  • Windows-2012
  • Windows-2012r2
  • Windows-2012r2sp1
  • RHEL-7.2

It is safest to use the same naming convention as used by the public images published by the OS vendors on the AWS marketplace.

An acceptable name is my_windows-2012r2_base-1

Vagrant installation

 Vagrant Installation

Warning!

The new versions of Vagrant delete the contents of the /opt/vagrant directory. If you want to keep old versions of Vagrant, first move them out of the /opt directory.

  • Download Vagrant from https://www.vagrantup.com/ and follow the instructions to install it.
  • Select DOWNLOAD…
  • Select the operating system of your workstation. For
    • Ubuntu: Debian
    • RedHat: Centos

 

Keep multiple versions on your workstation

Vagrant has an installer, that places the bin and embedded folders to “/opt/vagrant”. To be able to keep multiple versions of Vagrant execute the following:

cd /opt/vagrant
mkdir vagrant_MY_VERSION                                  # Create a subfolder for the new version
sudo chown -R YOUR_USER_NAME:wheel bin                    # Set the owner, so you can move the folder
sudo chown -R YOUR_USER_NAME:wheel embedded               # Set the owner, so you can move the folder
mv bin vagrant_MY_VERSION                                 # Move the folder to the version specific location
mv embedded vagrant_MY_VERSION                            # Move the folder to the version specific location
rm vagrant                                                # Delete the old version of the symbolic link
ln -s /opt/vagrant/vagrant_MY_VERSION/bin/vagrant vagrant # Create the symbolic link for the new version

Important

The default credentials of a Vagrant machine are:

  • UserName: vagrant
  • Password: vagrant

Windows in Vagrant

To test your cookbook on a Windows virtual machine locally, create one for Vagrant. See Launch Windows instances locally with Chef Test Kitchen for the details.

Ubuntu gets stuck in a login loop

When I rebooted my Ubuntu machine it started to repeatedly ask for the password.

When I entered the correct password, the screen went black and asked for the password again.

The problem was an error in the ~/.profile file.

To log into the Ubuntu terminal

  1. Press the Ctrl-Alt-F3 key combination
  2. Enter the username
  3. Enter the password
  4. Open the ~/.profile file with
    /bin/nano ~/.profile

Make sure the

  • The syntax is correct, all open braces are closed,
  • All PATH= statements have :$PATH at the end.

DevOps Engineering part 1. (Ubuntu) – Install the DevOps development tools on Ubuntu

DevOps Software installation on Ubuntu

There are multiple ways to install software on Ubuntu

  • Install a .deb package from the local drive with dpkg
  • Install the software from the internet with apt

Show hidden files

To be able to see hidden files in Files

  1. Start Files,
  2. In the Edit menu select Preferences,
  3. On the Views tab check the Show hidden and backup files,
  4. There is no Save or Ok button, just close the window to save the changes.

Text Editor

Atom

  1. Download the .deb installer from https://atom.io/
  2. Start the terminal by pressing CTRL-ALT-T
  3. Navigate to the Downloads directory
    cd ~/Downloads
  4. Install Atom
    sudo dpkg -i atom-amd64.deb
  5. Add an icon to the desktop
    1. Install the Gnome Panel, but do not install the unnecessary packages
      sudo apt-get install --no-install-recommends gnome-panel
    2. If the message is

      You might want to run ‘apt-get -f install’ to correct these:The following packages have unmet dependencies:…

      1. Execute
        apt-get -f install
    3. Create a new launcher on the desktop
      gnome-desktop-item-edit --create-new ~/Desktop
    4. Enter a name and command for the shortcut,
    5. To select an icon, click the image in the upper left corner and select an icon, and click Open,
    6. Unfortunately, Ubuntu does NOT update the icon in the window, click OK to save the shortcut.

To configure Atom see Atom text editor configuration

The preferences menu item in Linux is in the Edit menu of Atom.

 

Git

  1. Open the Terminal window
  2. Execute
    sudo apt-get install git

Configure Git

To configure Git see Git configuration.


Chef Development Kit

  1. Download the Chef Development Kit from https://downloads.chef.io/chef-dk/
  2. Click the Download button next to your Ubuntu version, and select Save File,
  3. Start the terminal by pressing CTRL-ALT-T
  4. Navigate to the Downloads directory
    cd ~/Downloads
  5. Install the Chef DK. Set the name of the file below for the actual version.
    sudo dpkg -i chefdk_CHEF_DK_VERSION-1_amd64.deb

Vagrant

Important

The default credentials of a Vagrant server are:

  • UserName: vagrant
  • Password: vagrant

Terraform by Hashicorp

 Terraform Installation

  1. Download Terraform from https://www.terraform.io/downloads.html
  2. Click the Linux 64-bit link, and select Save File,
  3. Open the Downloads folder in Files,
  4. Right-click the downloaded ZIP file and select Extract Here to extract the application
  5. Create a version specific directory, terraform_x.x.x for the Terraform application in the /opt/terraform folder
  6. Move the Terraform application into the version specific directory

Add Terraform to the path

  1. In the terminal window execute
    gedit ~/.profile
  2. Add this line to the end of the file
    PATH="/opt/terraform:$PATH"

AWS CLI

The Amazon Web Services command line interface installation will set up your workstation to launch instances in AWS from Test Kitchen. If you know you will work with AWS, see DevOps Engineering part 3. – Working with AWS for the AWS CLI installation.


Packer

We use Packer to create custom AWS AMIs that contain the fundamental configuration and applications that are common in every instance we launch.

Install Packer

  • Add the Packer installation directory to the system path

Configure the Ubuntu virtual machine in VirtualBox

Enable shared folders on the virtual machines

To be able to use shared folders between the host ( your workstation ) and the virtual machine.

  • Start the Ubuntu virtual machine in Virtual Box,
  • Select the virtual machine window on your workstation,
  • In the Devices menu of Virtual Box select Insert Guest Additions CD image…
  • Click the Run button,
  • Enter your admin password and click Authenticate,
  • Select the terminal window and press Enter,

  • Restart the virtual machine for the change to take effect,
  • In Ubuntu
    1. Open a terminal window,
    2. In the upper left corner of the desktop click the Search icon,
    3. Type terminal into the search box,
    4. Select the Terminal icon.
  • In the Terminal window create a directory for the mount point
    mkdir ~/host-share
    sudo chmod 777 ~/host-share
  • Mount the Shared Windows location. Get the Machine Folder name from the virtual machine’s Shared Folders tab in VirtualBox

    (If you shared the root of a Windows C: drive, the name is “C_DRIVE”.)

    sudo mount -t vboxsf -o uid=$UID,gid=$(id -g) MY_MACHINE_FOLDER_NAME ~/host-share

Source: https://help.ubuntu.com/community/VirtualBox/SharedFolders

Make the mounting permanent

  1. Create the folder /opt/scripts
  2. Create a new file in the new folder mount_host_share
    #!/bin/sh
    # Mount the Host Share to see the host computer's file system
    echo MY_PASSWORD | sudo -S mount -t vboxsf -o uid=1000,gid=1000 MY_MACHINE_FOLDER_NAME ~/host-share
  3. Make the file executable
    chmod +x /opt/scripts/mount_host_share
  4. On the Ubuntu desktop click the Search icon and type startup, and click the Startup Applications icon
  5. On the Startup Applications Preferences window click Add,
  6. Enter the Name and this Command, and click Save
    /opt/scripts/mount_host_share

The shared drive appears in the File manager.

Relax the password requirement

The virtual machine is already protected by the operating system of the host, your workstation, so it is not necessary to enter the password every time you want to switch to the guest machine. To remove the password requirement

  1. Click the System Settings icon in the sidebar,
  2. Uncheck the Waking from suspend and Returning from blank screen checkboxes.

Configure the Windows virtual machine in VirtualBox

Enable shared folders on the virtual machine

To be able to use shared folders between the host ( your workstation ) and the Windows virtual machine.

  1. Start the Windows virtual machine in VirtualBox,
  2. Select the virtual machine window on your workstation,
  3. In the Devices menu of VirtualBox select Insert Guest Additions CD image…
  4. In the virtual machine start Windows Explorer,
  5. Open the VirtualBox Guest Additions CD,
  6. Start VBoxWindowsAdditions.exe,

  7. On the User Account Control popup click Yes,
  8. Click Next on the setup page,
  9. Click Next on the path selection page,
  10. Click the Install button,
  11. Save all your documents on the virtual machine, and allow the reboot of the virtual machine for the changes to take effect.

The shared folders will be available in the virtual machine’s Windows Explorer under Network locations as the E: drive

Important

Refresh

When an application copies or deletes files and folders in the displayed shared folder, click the Refresh button to see the new state of the directory. Windows usually does not refresh the view of the shared folder.

 

Command window

On a Windows 10 guest machine, the Git Bash and Command Prompt windows cannot see the mapped shared folders. To execute commands in shared folders, use the PowerShell window. For more see Git Bash and Command Prompt cannot see the VirtualBox shared folder on Windows 10 guest

VirtualBox installation and configuration on Windows

Download Virtual Box from https://www.virtualbox.org/wiki/Downloads and follow the instructions to install it.

At the time of writing this was the section where the installer files were referenced

virtual-box-01-download

Run the downloaded installer file and accept all default values.

The installer starts the Virtual Box application.You can close it, we will use other tools to start it in the future.

To be able to launch 64-bit guest operating systems turn off Hyper-V in Windows Features

  1. In the Windows search box enter Windows Features and select Turn Windows Features on or off,
  2. Uncheck Hyper-V and click OK,
  3. Reboot the computer for the setting to take effect.

Create a virtual machine

To create a new virtual machine

Download the  OS image from the manufacturer’s web site.

  1. Start the VirtualBox application
  2. Click the New icon
  3. Enter a name, select the operating system type and version, and set the memory size,
  4. Set the initial size of the virtual hard disk
  5. Select the virtual machine icon and click the Start button,
  6. Click the Browse icon and select the OS image file in the Downloads directory,
  7. Click the Start button

Configure VirtualBox for each virtual machine

 Networking

To share the connectivity of the host computer with the virtual machine, mainly if you use VPN

  1. In Virtual box click Settings
  2. Select Network
  3. Attach the network adapter to NAT

Copy and paste

To enable copy and paste between the virtual machine and the host ( your workstation )

  1. On the General tab of the Settings page select the Bidirectional shared clipboard

Shared Folders

Specify the shared folder on your workstation

  1. In the VirtualBox menu select Preferences
  2. On the Shared Folders tab click the + icon
  3. To share the entire system drive, select the C: drive of your workstation

Configure the virtual machine OS

To configure the virtual machine operating system in VirtualBox, see Virtual machine configuration on the VirtualBox page.

VirtualBox installation and configuration on Macintosh

Download VirtualBox from https://www.virtualbox.org/wiki/Downloads and follow the instructions to install it.

At the time of writing, this was the section where the installer files were referenced

virtual-box-01-download

Run the downloaded installer file and accept all default values.

The installer starts the application.

 

Create a virtual machine

To create a new virtual machine

Download the OS image from the manufacturer’s website.

  1. Start the VirtualBox application
  2. Click the New icon
  3. Click the Expert Mode button
  4. Enter the name, select the operating system type and version, and set the memory size,
  5. Set the initial size of the virtual hard disk, and click the Create button,
  6. Select the virtual machine icon and click the Start button
  7. Click the Browse icon and select the OS image file in the Downloads directory,
  8. Click the Start button,

Configure VirtualBox for each virtual machine

Networking

To share the connectivity of the host computer with the virtual machine, mainly if you use VPN

  1. In VirtualBox select the virtual machine icon and click Settings
  2. Select Network
  3. Attach the network adapter to NAT

If your company use automated software installation and updates, set up a second network connection using the Bridged Adapter. This way the virtual machine will get an independent IP address in the same IP address range as the host machine.

  1. Select the Adapter 2 tab,
  2. Select the Enable Network Adapter checkbox,
  3. Select the Bridged Adapter in the Attached to drop-down,
  4. In the Name drop-down list select the fastest network connection of the host machine.

Copy and paste

To enable copy and paste between the virtual machine and the host ( your workstation )

  1. On the General tab of the Settings page select the Bidirectional shared clipboard

Shared Folders

Specify the shared folder on your workstation

  • In the VirtualBox menu select Preferences
  • On the Shared Folders tab click the + icon
  • Select your home folder on your workstation

Configure the virtual machine OS

To configure the virtual machine operating system in VirtualBox, see Virtual machine configuration on the VirtualBox page.

Create the AWS credentials file from a Chef Data Bag

When a process on a server instance needs access to an AWS account, the user who will execute the AWS CLI commands needs to be able to automatically authenticate in AWS.

For automatic AWS authentication, the AWS CLI creates two files in the .aws directory:

  • config and
  • credentials.

The location of this directory depends on the operating system and the type of user.

  • On Linux, the location is ~/.aws ( the user’s home directory )
  • On Windows, it is located at C:\Users\USER_NAME\.aws
  • On Windows, if the file was created by SYSTEM, the location is C:\Windows\System32\config\systemprofile\.aws

Store the AWS key values

To create these files, you need to store the AWS Access Key and Secret Key. The safest place for these values is an encrypted data bag. To automatically generate the AWS files, create a data bag file and name it the same as the “id” in the following structure:

{
  "id": "MY_DATA_BAG_ITEM_NAME",
  "MY_PROFiLE_1": {
    "region": "MY_REGION_1",
    "aws_access_key_id": "MY_ACCESSKEY_1",
    "aws_secret_access_key": "MY_SECRET_KEY_1"
  },
  "MY_PROFiLE_2": {
    "region": "MY_REGION_2",
    "aws_access_key_id": "MY_ACCESSKEY_2",
    "aws_secret_access_key": "MY_SECRET_KEY_2"
  }
}

To create and encrypt the data bag see my post on Chef Data Bags

Create the AWS authentication files

  1. In your Chef recipe, first install the AWS CLI and reboot the server, so the new path entry will be available for the Chef process.
  2. The following Chef code will create the AWS config and credential files. The script
    1. opens and decrypts the data bag,
    2. loads it into a hash table,
    3. iterates through the hash items,
    4. skips the “id” item,
    5. stores the AWS key values in a temporary file,
    6. executes the “aws configure” command to generate the AWS config and credential files.
  # Iterate through the data bag and create the credentials file

  puts "***** Creating the AWS credentials file"

  # Load the encrypted data bag into a hash
  aws_credentials = Chef::EncryptedDataBagItem.load('MY_DATA_BAG_NAME', 'MY_DATA_BAG_ITEM_NAME').to_hash

  # Iterate through the items, skip the "id"
  aws_credentials.each_pair do |key, value|

    # skip the "id"
    next if key == "id"

    # Add the credentials to the .aws/credentials file
    puts "Account #{key}, Region #{value['region']}"

    batch "add_aws_credentials_#{key}" do
      code <<-EOF echo #{value["aws_access_key_id"]}> input.txt
        echo #{value["aws_secret_access_key"]}>> input.txt
        echo #{value["region"]}>> input.txt
        echo.>> input.txt
        aws configure --profile #{key} < input.txt
      EOF
    end

  end

 

Splunk configuration

Splunk stores the configuration values in files in the /opt/splunkforwarder directory structure.

Splunk client

Description Location
Splunk Deployment server /opt/splunkforwarder/etc/system/local/deploymentclient.conf
  Example
targetUri = DEPLOYMENT_SERVER_URL:8089
Splunk Forwarder address /opt/splunkforwarder/etc/apps/tcpout-aws/local/outputs.conf
   Example
server = FORWARDER1_ADDRESS:9997,FORWARDER2_ADDRESS:9997
 Linux event log. Splunk tails this file. /var/log/messages
   To log a message in the Linux event log
logger "My message"
   To find a message in the Linux event log
grep "My message" /var/log/messages

 

Splunk server

Description Location
Default data directory /opt/splunk/var/lib/splunk/defaultdb/
Log location /opt/splunk/var/log/splunk/splunkd.log