Waiting for SSH service on …:22, retrying in 3 seconds

When you try to launch a new EC2 instance in AWS using Chef Test Kitchen and the process times out with the message:

  Waiting for SSH service on …:22, retrying in 3 seconds

$$$$$$ [SSH] connection failed, terminating (#<Net::SSH::AuthenticationFailed: Authentication failed for user …@…>)
>>>>>> ——Exception——-
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>> Failed to complete #create action: [SSH session could not be established] on default-rhel7
>>>>>> ———————-
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose –all` for configuration

Check the access to the “ssh_key” which is specified in the platforms: driver: transport: section of the .kitchen.yml file.

The .pem file should be accessible from the cookbook directory with the absolute or relative path.

Failed to complete #create action: [undefined method `version’ for nil:NilClass] on default…

When you execute kitchen converge to launch an EC2 instance in AWS with Chef Test Kitchen, you get the error message:

>>>>> ——Exception——-
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>> Failed to complete #create action: [undefined method `version’ for nil:NilClass] on default-windows-2012r2
>>>>>> ———————-
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose –all` for configuration

The kitchen-ec2 driver tries to read the operating system version from the name of the image, but it cannot find it.

When you create your own AMI (Amazon Machine Image), make sure the version of the operating system is in the name.

The Test Kitchen Git repository has the following at https://github.com/test-kitchen/kitchen-ec2

Note that the image_search method requires that the AMI image names be in a specific format. Some examples are:

  • Windows-2012
  • Windows-2012r2
  • Windows-2012r2sp1
  • RHEL-7.2

It is safest to use the same naming convention as used by the public images published by the OS vendors on the AWS marketplace.

An acceptable name is my_windows-2012r2_base-1

Could not load the ‘vagrant’ driver from the load path

When you execute the kitchen converge command to launch a virtual machine on your workstation with the Vagrant driver, you may get the error message:

>>>>> ——Exception——-
>>>>>> Class: Kitchen::ClientError
>>>>>> Message: Could not load the ‘vagrant’ driver from the load path. Please ensure that your driver is installed as a gem or included in your Gemfile if using Bundler.
>>>>>> ———————-
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose –all` for configuration

An additional Ruby installation on your workstation besides the version came with the Chef Development kit can cause this.

To force Test Kitchen to use the Ruby instance that was installed with the Chef Development kit start your commands with chef exec:

 chef exec kitchen list

To check the Ruby installations on your workstation, execute

which ruby

The path should be


If the result is something else, uninstall the extra Ruby version.

If you use the Ruby Version Manager (RVM), uninstall the current ruby version, then uninstall RVM. See https://richonrails.com/articles/uninstalling-rvm

Reinstall the Chef Development Kit, which will install the recommended Ruby version on your workstation.

If you get the Vagrant error message:

Vagrant failed to initialize at a very early stage …


follow the recommended steps on the screen to reinstall the Vagrant plugins.

Vagrant installation

 Vagrant Installation

  • Download Vagrant from https://www.vagrantup.com/ and follow the instructions to install it.
  • Select DOWNLOAD…
  • Select the operating system of your workstation. For
    • Ubuntu: Debian
    • RedHat: Centos

Keep multiple versions on your workstation

Vagrant has an installer, that places the bin and embedded folders to “/opt/vagrant”. To be able to keep multiple versions of Vagrant execute the following:

cd /opt/vagrant
mkdir vagrant_MY_VERSION                                 # Create a folder for the new version
sudo chown -R YOUR_USER_NAME:wheel bin                   # Set the owner, so you can move it
sudo chown -R YOUR_USER_NAME:wheel embedded              # Set the owner, so you can move it
mv bin vagrant_MY_VERSION                                # Move the folder to the version specific location
mv embedded vagrant_MY_VERSION                           # Move the folder to the version specific location
rm vagrant                                               # Delete the old version of the symbolic link
ln -s /opt/vagrant/vagrant_MY_VERSION/bin/vagrant vagrant # Create the symbolic link


The default credentials of a Vagrant machine are:

  • UserName: vagrant
  • Password: vagrant

Windows in Vagrant

To test your cookbook on a Windows virtual machine locally, create one for Vagrant. See Launch Windows instances locally with Chef Test Kitchen for the details.

Create the AWS credentials file from a Chef Data Bag

When a process on a server instance needs access to an AWS account, the user who will execute the AWS CLI commands needs to be able to automatically authenticate in AWS.

For automatic AWS authentication, the AWS CLI creates two files in the .aws directory:

  • config and
  • credentials.

The location of this directory depends on the operating system and the type of user.

  • On Linux, the location is ~/.aws ( the user’s home directory )
  • On Windows, it is located at C:\Users\USER_NAME\.aws
  • On Windows, if the file was created by SYSTEM, the location is C:\Windows\System32\config\systemprofile\.aws

Store the AWS key values

To create these files, you need to store the AWS Access Key and Secret Key. The safest place for these values is an encrypted data bag. To automatically generate the AWS files, create a data bag file and name it the same as the “id” in the following structure:

  "MY_PROFiLE_1": {
    "region": "MY_REGION_1",
    "aws_access_key_id": "MY_ACCESSKEY_1",
    "aws_secret_access_key": "MY_SECRET_KEY_1"
  "MY_PROFiLE_2": {
    "region": "MY_REGION_2",
    "aws_access_key_id": "MY_ACCESSKEY_2",
    "aws_secret_access_key": "MY_SECRET_KEY_2"

To create and encrypt the data bag see my post on Chef Data Bags

Create the AWS authentication files

  1. In your Chef recipe, first install the AWS CLI and reboot the server, so the new path entry will be available for the Chef process.
  2. The following Chef code will create the AWS config and credential files. The script
    1. opens and decrypts the data bag,
    2. loads it into a hash table,
    3. iterates through the hash items,
    4. skips the “id” item,
    5. stores the AWS key values in a temporary file,
    6. executes the “aws configure” command to generate the AWS config and credential files.
  # Iterate through the data bag and create the credentials file

  puts "***** Creating the AWS credentials file"

  # Load the encrypted data bag into a hash
  aws_credentials = Chef::EncryptedDataBagItem.load('MY_DATA_BAG_NAME', 'MY_DATA_BAG_ITEM_NAME').to_hash

  # Iterate through the items, skip the "id"
  aws_credentials.each_pair do |key, value|

    # skip the "id"
    next if key == "id"

    # Add the credentials to the .aws/credentials file
    puts "Account #{key}, Region #{value['region']}"

    batch "add_aws_credentials_#{key}" do
      code <<-EOF echo #{value["aws_access_key_id"]}> input.txt
        echo #{value["aws_secret_access_key"]}>> input.txt
        echo #{value["region"]}>> input.txt
        echo.>> input.txt
        aws configure --profile #{key} < input.txt



Dynamically set Chef resource attributes

When you need to set a Chef resource attribute based on the current state of the environment, there is a way to dynamically provide the value.

  1. Set the value of a boolean variable with a test,
  2. Declare the Chef resource and assign a reference to it to a variable,
  3. Set the resource attribute based on the value of the boolean variable.
# Set a boolean variable with a test 
  "#{node['domain']}" != ""

# Execute a resource and get a reference to it into a variable
# Set the attribute value based on the boolean variable



No instances for regex `’, try running `kitchen list’ in Chef Test Kitchen

When I tried to execute the “kitchen list” command in Chef Test Kitchen the following error came up:

No instances for regex `’, try running `kitchen list’

I could not find the reason for the error, but when I opened the .kitchen.yml file in the Atom editor, made an insignificant change and saved the file, the error went away.

Chef Attributes

Chef attributes are global variables that are available for every cookbook on the node. There are multiple formats to declare and use an attribute. For important notes on the syntax, please see Undefined method or attribute error in a Chef recipe.

To override the value of an attribute that is defined in another cookbook, use the following syntax

node.override['ATTRIBUTE_NAME'] = 'NEW_VALUE'

During compilation, this line will replace the default value of the attribute with the NEW_VALUE.


Getting started with InSpec

InSpec is an open-source testing framework to verify your infrastructure satisfies the design requirements.

In this article, we will learn to install and use InSpec with Chef.

Install InSpec

  1. Navigate to https://downloads.chef.io/inspec, and download the installer for the operating system of your workstation,
  2. Execute the downloaded installer.

Allow InSpec to verify Red Hat Enterprise Linux instances

InSpec needs “sudo” access to execute the tests, but Red Hat Enterprise Linux prevents that access. Execute the following code on every instance when it runs in Test Kitchen:

if (node.chef_environment == "_default")
  # Running in Test Kitchen

  # Ensure sudo is installed
  package 'Install sudo' do
    package_name 'sudo'
    action :install

  file '/etc/sudoers' do
    mode 0440
    owner 'root'
    group 'root'
    action :create

  delete_lines 'remove hash-comments from /some/file' do
    path '/etc/sudoers'
    pattern '^.*requiretty'



Start to use InSpec

To use InSpec as the default integration testing tool in Chef Test Kitchen

  1. Open the .kitchen.yml file of the cookbook,
  2. Delete the following lines from the platform section if exist:
      sudo: true
  3. Add the following lines to the file between provisioner: and platforms:
      name: inspec
  4. Place the test files into the default location of the InSpec integration test. The “verify” command executes all files in the directory.
  5. To execute the test file with the verify command, add these lines to every suite. This will execute all test files in the default folder.
            - test/smoke/default
  6. To execute only one test file, specify the file name:
            - test/smoke/default/MY_RECIPE_test.rb
  7. To execute the test file of another suite, use relative path, you can use tests from other cookbooks (../ANOTHER_COOKBOOK/test/recipes/ANOTHER_SUITE_NAME).
            - ../ANOTHER_COOKBOOK/test/smoke/default
  8. Create an integration test for your recipe. Create a new file in the test/recipes/THE_SUITE_NAME folder. The name does not matter, if you are planning to create only one test file for the suite, name the file after the suite: default_test.rb,
  9. The following is a simple example of an InSpec integration test:
    # # encoding: utf-8
    # Inspec test for recipe my_cookbook::default
    # The Inspec reference, with examples and extensive documentation, can be
    # found at https://docs.chef.io/inspec_reference.html
    unless os.windows?
     describe user('root') do
     it { should exist }
     skip 'This is an example test, replace with your own test.'
    describe port(80) do
     it { should_not be_listening }
     skip 'This is an example test, replace with your own test.'

As you can see, the syntax of InSpec is (intentionally) very similar to ServerSpec, that it replaces. It is very easy to convert existing ServerSpec integration tests to InSpec compliance tests.

Differences between ServerSpec and InSpec

ServerSpec “process”

Does not work on a Windows host.

On Linux the syntax changed from

 describe process('PROCESS_NAME') do
   it { should be_running }


describe processes('PROCESS_NAME') do
  its('states') { should eq ['R<'] }


The :dword comparison uses the decimal value with no quotes instead of the hex value with quotes

changed from

describe registry_key('HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full') do
  it { should have_property_value('Release', :dword, '70805') } # 460805 decimal


describe registry_key('HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\NET Framework Setup\NDP\v4\Full') do
  it { should have_property_value('Release', :dword, 460805) } # For dword use the decimal value, no quotes



For more information

For more information on the Kitchen InSpec verifier visit https://github.com/chef/kitchen-inspec

Undefined method or attribute error in a Chef recipe

There are multiple reasons Chef can display the following error message

 Undefined method or attribute `...' on `node'


There are many ways to reference an attribute in a Chef recipe:

node['ATTRIBUTE_NAME'] (the recommended style)
node[:ATTRIBUTE_NAME]  (use it only if the single or double quotes (' or ") would cause a problem in the expression. This can happen in the guard of PoweShell resources.)
node.ATTRIBUTE_NAME    (DO NOT USE IT: the bootstrap compiler cannot understand it, chef-client cannot handle it with nil values in if statements)

To check if the attribute value is nil, use the following format:

if ( !node['ATTRIBUTE_NAME'].nil? )

If you use the node.ATTRIBUTE_NAME.nil? to test the value, and the value is nil, Chef throws the above error message.