Chef Attributes

Chef attributes are global variables that are available for every cookbook on the node. There are multiple formats to declare and use an attribute. For important notes on the syntax, please see Undefined method or attribute error in a Chef recipe.

To override the value of an attribute that is defined in another cookbook, use the following syntax

node.override['ATTRIBUTE_NAME'] = 'NEW_VALUE'

During compilation, this line will replace the default value of the attribute with the NEW_VALUE.

 

Getting started with InSpec

InSpec is an open-source testing framework to verify your infrastructure satisfies the design requirements.

In this article, we will learn to install and use InSpec with Chef.

Install InSpec

  1. Navigate to https://downloads.chef.io/inspec, and download the installer for the operating system of your workstation.
  2. Execute the downloaded installer.

Start to use InSpec

To use InSpec as the default integration testing tool in Chef Test Kitchen

  1. Open the .kitchen.yml file of the cookbook,
  2. Delete the following lines from the platform section if exist:
    busser:
      sudo: true
  3. Add the following lines to the file:
    verifier:
      name: inspec
  4. Add this to every suite, so InSpec will search for the test files in the test/recipes directory. Otherwise, the test file needs to be in the test/recipes/SUITE_NAME directory
     verifier:
       inspec_tests:
         - test/recipes
  5. Create a folder structure in the cookbook folder for the InSpec integration tests,
    test
    |--recipes
  6. Create an integration test for your recipe. Create a new file in the test/recipes folder and name it RECIPE_NAME_test.rb. For the default recipe call it default_test.rb,
  7. The following is a simple example of an InSpec integration test:
    # # encoding: utf-8
    
    # Inspec test for recipe my_cookbook::default
    
    # The Inspec reference, with examples and extensive documentation, can be
    # found at https://docs.chef.io/inspec_reference.html
    
    unless os.windows?
     describe user('root') do
     it { should exist }
     skip 'This is an example test, replace with your own test.'
     end
    end
    
    describe port(80) do
     it { should_not be_listening }
     skip 'This is an example test, replace with your own test.'
    end

As you can see, the syntax of InSpec is (intentionally) very similar to ServerSpec, that it replaces. It is very easy to convert existing ServerSpec integration tests to InSpec compliance tests.

Differences between ServerSpec and InSpec

ServerSpec “process”

changed from

 describe process('PROCESS_NAME') do
   it { should be_running }
 end

to

describe processes('PROCESS_NAME') do
  its('states') { should eq ['R<'] }
end

Undefined method or attribute error in a Chef recipe

There are multiple reasons Chef can display the following error message

NoMethodError
 -------------
 Undefined method or attribute `...' on `node'

 

There are multiple ways to reference an attribute in a Chef recipe:

node['ATTRIBUTE_NAME'] (the recommended way)
node["ATTRIBUTE_NAME"] 
node[:ATTRIBUTE_NAME]  ( use it only if the single or double quotes (' or ") would cause a problem in the expression)

node.ATTRIBUTE_NAME

To check if the attribute value is nil, use the following format:

if ( !node['ATTRIBUTE_NAME'].nil? )
   ...
end

If you use the node.ATTRIBUTE_NAME format and the value is nil Chef throws the above error message.

Add SSH key to a Jenkins Git step

To access a Git repository Jenkins can use an SSH key.

To add the SSH key to the Jenkins server use the following Chef script

Store the SSH key in an encrypted data bag called “keys”.

{
 "id": "ci_private_keys",
 "ci_github_key": "-----BEGIN RSA PRIVATE KEY-----\n...\n-----END RSA PRIVATE KEY-----",
}

 

Add the following to the Jenkins Chef recipe

  • Install Git
package 'git'
  • Install the Git and Credentials Jenkins plugins
jenkins_plugin 'git'
jenkins_plugin 'credentials'
  • Copy the SSH key to the Jenkins server
rsa_key = data_bag_item('keys', 'ci_private_keys')
file '/var/lib/jenkins/.ssh/id_rsa' do
  content "#{rsa_key['ci_github_key']}"
  owner 'jenkins'
  group 'jenkins'
  mode '0600'
end
  • Add github.com to the known hosts
bash 'provide github.com RSA fingerprint' do
  code <<-EOF
   ssh-keyscan github.com >> /var/lib/jenkins/.ssh/known_hosts
   chown jenkins.jenkins /var/lib/jenkins/.ssh/known_hosts
  EOF
  not_if{system('grep github.com /var/lib/jenkins/.ssh/known_hosts')}
end

 

To specify the SSH key in the Git step

  1. When the Jenkins server is operational, navigate to the Web interface
  2. Create a new Jenkins project
  3. In the Source Code Management section
    1. Select Git
    2. Enter the SSH URL of the repository
    3. When you are adding the first project, click the Add button to create the credential

      1. Click Jenkins to select the credentials provider
      2. Select SSH Username with private key as the Kind
      3. Enter the username you used when you created the SSH key for the Git repository
      4. Select From the Jenkins master ~/.ssh as the Private Key
      5. Click the Add button
    4. In the Credentials drop down select the credential you have created (the Git user name)

 

Chef custom resource is using the same property name as the called resource

When you create a Chef custom resource, you can call other resources including custom resources you have created. For ease of use it can be convenient to use the same property name as the called resource use.

property :delay_mins,          Fixnum, default: 3

reboot 'Hostname was changed' do
 reason reboot_reason
 delay_mins delay_mins
 action :request_reboot
end

When you you execute the code, chef will display the following error message:

property delay_mins is declared in both reboot[Hostname was changed] and utils_reboot[hostname_reboot] action :request_reboot. Use new_resource.delay_mins instead.

To tell Chef that you want to use the property you have created in this custom resource, add new_resource. in front of your property:

property :delay_mins,          Fixnum, default: 3

reboot 'Hostname was changed' do
 reason reboot_reason
 delay_mins new_resource.delay_mins
 action :request_reboot
end

NoMethodError: undefined method `exists?’ for Chef::Resource::File:Class

When you create a Chef custom resource and use the File class, you need to make slight a change in the syntax you use.

In a Chef recipe you can use

if File.exists?("#{FileName}")

to check for the existence of a file. If we use the same line in a Chef custom resource, we get the error message:

NoMethodError: undefined method `exists?’ for Chef::Resource::File:Class

In a Chef custom resource, you need to add :: in front of the File class to make it work

The leading :: tells Chef not to look for the “File” class in the Chef namespace.

if ::File.exists?("#{FileName}")

 

Chef file locations

The Chef file and folder locations are different on Linux and Windows machines. This article explains the purpose of each file and the location.

Summary

Linux Windows
Cookbook location /var/chef/cache/cookbooks  C:\chef\cache\cookbooks
Chef Client run log /var/log/chef.log First run only
C:\chef\chef-client.log
Subsequent Chef client runs
C:\chef\log\client.log
Error log /var/chef/cache/chef-stacktrace.out C:\chef\cache\chef-stacktrace.out
Ohai output /var/chef/cache/failed-run-data.json C:\chef\cache\failed-run-data.json
Recommended location for custom log files /tmp/cheflog.log C:\Logs\Chef\cheflog.log
Chef Client configuration /etc/chef/client.rb C:\chef\client.rb

When you test your cookbook in Test Kitchen

The .kitchen.yml file contains the username to execute the Chef cookbook. It is specified under platforms:, transport:, username:

Use that value in place of USER-NAME-FROM-KITCHEN-YML below.

Linux Windows
Cookbook location /tmp/kitchen/cookbooks
/tmp/kitchen/cache/cookbooks
 C:\Users\USER-NAME-FROM-KITCHEN-YML\AppData\Local\Temp\kitchen\cookbooks
Error log /tmp/kitchen/cache/chef-stacktrace.out C:\Users\USER-NAME-FROM-KITCHEN-YML\AppData\Local\Temp\kitchen\cache\chef-stacktrace.out
Ohai output /tmp/kitchen/cache/failed-run-data.json C:\Users\USER-NAME-FROM-KITCHEN-YML\AppData\Local\Temp\kitchen\cache\failed-run-data.json
Data bags /tmp/kitchen/data_bags C:\Users\USER-NAME-FROM-KITCHEN-YML\AppData\Local\Temp\kitchen\data_bags

Cookbook location

When the Chef recipes are executed, all cookbooks are stored on the node. You can examine the code to make sure your latest changes are reflected on the machine.

The log of the Chef client run

The output of the Chef cookbook execution is in the chef.log or chef-client.log file

On Windows

The log of the first Chef Client run and subsequent runs are stored in different log files. After the initial Chef Client run, the rest of the log entries are collected in the second file.

Stacktrace

Chef saves information on the hard drive when scripts are executed. If there is a failure, the stack trace of the last error is saved in the chef-stacktrace.out file.

Ohai output

All the information that Ohai collects on the instance, is saved in the failed-run-data.json file, even if there is no error. It is a great resource to get the server specific values.

Cloud info

  • cloud-specific information under “cloud
  • cloud instance information under “ec2

Computer info

  • CPU and memory configuration under “cpu“, “cs_info“, “memory
  • drive sizes and network settings under “filesystem” and “network

Operating system info

  • operating system information under “os_info
  • list of enabled Windows features under “dism_features
  • the list of installed applications under “packages

Chef info

  • Chef client configuration values under “chef_client
  • the Chef node information under “chef_type”: “node
  • the Chef run list under “run_list
  • list of Chef cookbooks and their versions under “cookbooks
  • list of the executed recipes under “recipes
  • the value of the passed in Chef attributes under “normal
  • the value of the Chef cookbook attributes under “chef_client” and the cookbook name
  • all the information on the Chef resources under “json_class
  • the stack trace of the last error under “exception

Internalize Chocolatey packages

To access Chocolatey packages in your private network you need to download them from the Internet and store them at a location where all servers can access them. You can internalize Chocolatey packages if you have a Chocolatey Business subscription.

You can use an Artifactory server to host the internalized packages.

List of available packages

To get the list of the available Chocolatey packages on an Artifactory server

choco list -s http://ARTIFACTORY_SERVER_URL/artifactory/api/nuget/ARTIFACTORY_REPOSITORY_NAME

Non copyrighted applications

If the Chocolatey package does not contain copyrighted components, Chocolatey can download and repackage the entire package, including the application.

Download a Chocolatey package from the Internet and store it on your local drive

choco download PACKAGE_NAME --internalize

To store the Chocolatey packages on an Artifactory server

  • The Artifactory server has to have a Pro license
  • Create a NuGet type local repository

Upload the package from your local drive to an Artifactory server

curl -u ARTIFACTORY_USERNAME:ARTIFACTORY_PASSWORD -X PUT "http://ARTIFACTORY_SERVER_URL/artifactory/ARTIFACTORY_REPOSITORY_NAME/PACKAGE_NAME/PACKAGE_FILE_NAME.nupkg" -T ./PACKAGE_FILE_NAME.nupkg

Copyrighted applications

If the Chocolatey package installs an application that is copyrighted, the author of the package cannot publish the copyrighted source, but can place the download location and installation process into the package.

Internalize the package

To download the necessary files and create a custom Chocolatey package with copyrighted application source:

    1. Download the Chocolatey package without the internalize option
      choco download PACKAGE_NAME

      Example:

      choco download javaruntime
    2. Delete the small .nupkg files. These are too small to contain the installer files. We will recreate them in a subfolder with the downloaded installer files.
    3. Find the package folder in the download sub-folder that has a tools subfolder. In the case of “javaruntime”, the real package is the “jre8”, not “javaruntime”.
    4. Open the tools\chocolateyInstall.ps1 file
    5. Find the download urls of the installer files.

       Some scripts contain variables for the version of the application, so you have to manually assemble the final download URL.
    6. Assemble the actual download URL and download the application source with your web browser,
    7. Place the downloaded installer files into the tools folder,
    8. Update the chocolateyInstall.ps1 to point to the installer files in the package:
      • Add the following before the $url =… and $url64 =… statements
        $toolsDir = "$(Split-Path -parent $MyInvocation.MyCommand.Definition)"
      • Replace the the $url =… and $url64 =… statements. Use the actual file names, and make sure to use double quotes () for the string interpolation to work!
        $url = "$toolsDir\32_BIT_INSTALLER_FILE"
        $url64 = "$toolsDir\64_BIT_INSTALLER_FILE"

      • Add
        -UseOriginalLocation

        to the Install-ChocolateyPackage  line.

    9. Right click the .nuspec file in the package folder and select Compile Chocolatey Package

      (or execute choco pack PATH\TO\NUSPEC.nuspec)
    10. The package is created in the folder of the .nuspec file and the file name is the composite of id + “.” + version tag values in the .nuspec file.

Upload the package to the local repository

  • In the command line execute
    curl -u ARTIFACTORY_USERNAME:ARTIFACTORY_PASSWORD -X PUT "http://ARTIFACTORY_SERVER_URL/artifactory/ARTIFACTORY_REPOSITORY_NAME/PACKAGE_NAME/PACKAGE_FILE_NAME.nupkg" -T PATH/TO/PACKAGE_FILE_NAME.nupkg

    Video tutorials

  • Package Synchronizer

Launch Windows instances locally with Chef Test Kitchen

Most Linux distributions are free, and do not require product keys to launch them.

The steps below are based on the great article at http://kitchen.ci/blog/test-kitchen-windows-test-flight-with-vagrant/

I have summarized the steps below to create a free Virtual Box Windows Server 2012R2 image on your workstation, so Test Kitchen can use Vagrant and Virtual Box to launch Windows instances and test cookbooks locally fast and free.


Install the Vagrant WinRm plugin

vagrant plugin install vagrant-winrm

Get BoxCutter

Create the Windows image with BoxCutter. This process will take 30 minutes or more to fully configure the Windows image. The download step may time out on slower VPN connections. In that case disconnect the VPN connection and try it again without it. In case of any error, just execute the last command again and if it can, the process will continue from the point of error.

cd ~/
mkdir boxcutter
cd boxcutter
git clone https://github.com/boxcutter/windows.git
cd windows

Modify the BoxCutter JSON file

Change the ~/boxcutter/windows/eval-win2012r2-standard.json file to avoid the error message:

virtualbox-iso: Removing floppy drive…
==> virtualbox-iso: Error removing floppy controller: VBoxManage error: VBoxManage: error: The machine ‘eval-win2012r2-standard’ is already locked for a session (or being unlocked)
==> virtualbox-iso: VBoxManage: error: Details: code VBOX_E_INVALID_OBJECT_STATE (0x80bb0007), component MachineWrap, interface IMachine, callee nsISupports
==> virtualbox-iso: VBoxManage: error: Context: “LockMachine(a->session, LockType_Write)” at line 1038 of file VBoxManageStorageController.cpp

  • Change “headless”: “false”, to “headless”: “true”,
  • Add under “headless”: “true”,
"shutdown_timeout": "60m",
"post_shutdown_delay": "120s",

 

  • Add under every occurrence of “headless”: “{{ user `headless` }}”,
"shutdown_timeout": "{{ user `shutdown_timeout` }}",
"post_shutdown_delay": "{{ user `post_shutdown_delay` }}",

Create the virtual machine

make virtualbox/eval-win2012r2-standard

Add the image to Vagrant

vagrant box add windows-2012r2 ./box/virtualbox/eval-win2012r2-standard-nocm-1.0.4.box

Test the virtual machine

Your .kitchen.yml file should look like this

---
driver:
  name: vagrant

provisioner:
  name: chef_zero

platforms:
  - name: windows-2012r2

suites:
  - name: MY_SUITE_NAME
  run_list:
  - recipe[MY_COOKBOOK::MY_RECIPE]

 

You may need to execute the kitchen converge commands as sudo to be able to launch the Windows instance.