Remove Policyfile.rb from your Chef cookbook

When you test your cookbook in Chef Test Kitchen and get the following error, delete the “Policyfile.rb” from your Chef cookbook directory.

$$$$$$ You must set your run_list in your policyfile instead of kitchen config. The run_list your config will be ignored.
$$$$$$ Ignored run_list: ["recipe[...::...]"]
       Preparing dna.json
       Exporting cookbook dependencies from Policyfile /tmp/...
       Error: Invalid lockfile data
       Reason: (ChefDK::DependencyConflict) Cookbook ... (...) has dependency constraints that cannot be met by the existing cookbook set:
       Cookbook ... isn't included in the existing cookbook set.

Chef Data Bags

Create an encrypted Chef data bag

There are secrets in most of the Chef cookbooks that we want to protect. We don’t want to give out user names, passwords and AWS keys.  In Chef the best place to hide these secrets is the Encrypted Data Bag.

A Data Bag is a JSON file that we can encrypt, so we can store it in version control with the rest of the cookbook.

To make  continuous integration and delivery (CI/CD) easier, store the Encrypted Data Bags in the cookbook folder structure and commit them together with the rest of the cookbook into version control (Git)

To make sure the unencrypted secret is not committed into version control, add the following line to the  .gitignore file

# Ignore the unencrypted Data Bags

The structure of Chef folder should look like this. Store the unencrypted Data Bags with the original values in the data_bags_unencrypted folder

Create a folder for the unencrypted Data Bag and create a file for the Data Bag Item. The name of the file and the value of the id element should be the same.

Enter the Data Bag Item values and save the file.

 "id": "access_key",
 "AccessKey": "XXXXX",
 "SecretKey": "YYYYY"

Automate the data bag encryption

Create the folder structure

  1. Create a folder for data bag related files on the same level as the cookbooks folder. Name it data_bags

  2. Get the data bag encryption secret file from your Chef server administrator and place it in the data_bags_unencrypted  folder.
  3. Create a folder for Chef related scripts on the same level as the cookbooks folder. Name it devops-chef-scripts

Create the automation script

Create the following script and name it This script

  1. Encrypts the Data Bag,
  2. Uploads the encrypted data bag to the Chef server,
  3. Saves the encrypted data bag in the data_bags folder on your workstation.

Replace >>>MY_ENCRYPTED_DATABAG_SECRET<<< with the name of the encrypted data bag secret file.

if [ -z $1 ] || [ -z $2 ]
	echo "Please supply the arguments: DATABAG_NAME ITEM_NAME"
	echo "../devops-chef-scripts/ [DATA_BAG_NAME/NAME OF THE FOLDER] [ITEM_NAME/ID]"

  echo -- knife data bag create $1
  knife data bag create $1

  # Encrypt the databag and upload it to the Chef server
  echo -- knife data bag from file $1 $1/$2.json --secret-file ../data_bags_unencrypted/>>>MY_ENCRYPTED_DATABAG_SECRET<<<
  knife data bag from file $1 $1/$2.json --secret-file ../data_bags_unencrypted/>>>MY_ENCRYPTED_DATABAG_SECRET<<<

  # Create a directory for the encrypted databag on the workstation
  echo -- mkdir -p ../data_bags/$1
  mkdir -p ../data_bags/$1

  # Download the encrypted data bag
  echo -- knife data bag show $1 $2 -F json 'to' ../data_bags/$1/$2.json
  knife data bag show $1 $2 -F json > ../data_bags/$1/$2.json

  echo "Encrypted data bag has been created at ../data_bags/"$1"/"$2".json"


echo -n "Press a key to exit"	#'-n' means do not add \n to end of string
read              		# No arg means dump next line of input

Add execution right to the file

chmod +x ./

Encrypt the data bag

Open a Bash window in the data_bags_unencrypted folder

Execute the following command, where

DATA_BAG_NAME is the name of the data bag folder
ITEM_NAME is the value of the id element and the item file name without the ‘.json’ extension,

../devops-chef-scripts/ DATA_BAG_NAME ITEM_NAME

The script will create a folder for the Data Bag in the “data_bags” folder and save the encrypted Data Bag file in it.

The following warning is normal. We did not want to unencrypt the data bag, just download the encrypted version.

WARNING: Encrypted data bag detected, but no secret provided for decoding. Displaying encrypted data.


If you get the error message

ERROR: The object you are looking for could not be found
Response: Cannot load data bag item … for data bag …

make sure you set the name of the data bag item file without the .json extension and the value of the id element the same.


Unable to satisfy the following requirements error message in Chef Test Kitchen

When Berkshelf in the Chef Test Kitchen cannot resolve the cookbook dependencies, it displays the following error message during converge:

Unable to satisfy the following requirements

To help Berks to start a fresh calculation, delete the Berksfile.lock file and enter the following into the command window:

berks install

DevOps Engineering part 2. – Create and test your first cookbook in 5 minutes

In the first part of the series, Beginner’s Guide to DevOps Engineering Part 1. we have already installed the DevOps development tools.

Create and test your first cookbook in 5 minutes

 Set up the Chef working folder

  1. Create a folder for the Chef development
    1. on Mac ~/Chef
    2. on Windows C:\Chef
  2. In the Chef folder create a sub-folder cookbooks
  3. Right-click the cookbooks folder
    1. on Mac select Services, iTerm2 in Finder
    2. on Windows  select Git Bash Here

Create your first cookbook

    • Enter the following command into the Bash window to create a new cookbook
      chef generate cookbook test

Specify the program you want to use to open .rb files

on Mac

  1. In Finder navigate to ~/Chef/test/recipes
  2. Right-click the default.rb file and select Open With, Atom

on Windows

  • In File Explorer navigate to the C:\Chef\test\recipes older
  • Right-click the default.rb file
    • Select Open with
    • Select the Always use… checkbox  and click the More apps link
  • Select Notepad++ and click the OK button


Let’s create a file with Hello world in it.

      • Enter the following to create your first recipe
        • on Mac
          file '/Users/YOUR_USERNAME/Desktop/helloworld.txt' do
           content 'Hello world'
        • on Windows (make sure you use double backslashes!!!)
          file 'C:\\Users\\YOUR_USERNAME\\Desktop\\helloworld.txt' do
           content 'Hello world'
      • Save the file
      • In the bash Window navigate to the test cookbook directory
        cd test
      • Execute your first recipe with the following in the Bash window
        chef-client --local-mode recipes/default.rb
      • Navigate to the Desktop and open the helloworld.txt file to see the result of your script.

Test your cookbook on a virtual machine

We will use Test Kitchen and Vagrant to launch virtual machines. Currently, only Linux images are available for Vagrant, so we will modify our recipe to select between Linux and Windows and act accordingly.

Open the default.rb recipe and update it to look like this

case node['os']
when 'linux'
  file "/tmp/helloworld.txt" do
    content 'This file was created by Chef!'
when 'windows'
  file "C:\\Chef\\helloworld.txt" do
    content 'This file was created by Chef!'

Enter the following commands into the Bash window

      • Display the available test configurations ( suites )
        kitchen list
      • Launch a virtual Linux machine and test your recipe
        kitchen converge ubuntu

        The first time you launch a virtual machine with an operating system you have never used on your workstation, Vagrant has to download the machine image from the Internet. It can take 5-10 minutes. When the virtual machine is created and launched, Test Kitchen will copy your cookbook to it and execute it there.
      • Test kitchen will display the —–> Kitchen is finished message when the cookbook has successfully executed.
      • Log into the virtual machine
        kichen login ubuntu
      • The root prompt appears in the Bash windows. Let’s check if the file has been created or not.
      • To destroy the virtual machine
        kitchen destroy ubuntu

Windows in Vagrant

To test your cookbook on a Windows virtual machine locally, create one for Vagrant. See Launch Windows instances locally with Chef Test Kitchen for the details.

Learn the basics, so you can ask questions

Chef has a steep learning curve. Chef is not just scripting or programming, but you have to understand how Chef works to be able to use it to configure servers. There are many ways to do the same thing and there is not much documentation to recommend the best way. If you search Google, the problems usually have multiple solutions, and many times the “best” answer is selected based on personal preference. To get started, you should familiarize yourself with the tools, because you will use most of them during the development process.

In this guide I will use a Windows computer as a workstation, but all tools work on Mac and Linux computers.


Learn Vagrant to understand how Test Kitchen manages the test servers on your local machine or at AWS. You will not use vagrant directly, but Test Kitchen uses it to launch servers.


Test Kitchen




Working with AWS in Beginner’s Guide to DevOps Engineering part 3.


to the Tutorials page

Beginner’s Guide to DevOps Engineering part 1.

DevOps engineering (Release engineering) is a relatively new profession. There is a very high demand for experienced DevOps engineers, because more and more large corporations want to migrate their data centers into the Cloud. In this series I will introduce you to the tools of the trade through the development of a Chef cookbook.

The tools of the DevOps engineers are new and most of them are in the active development phase with frequent releases. Some of the new versions have show stopping bugs in them, so it is a good idea to test all new versions of the tools before you uninstall the old one.

To work as a DevOps engineer you need a development environment with multiple tools. Luckily all of them are available for free and easy to set up.

  • The Chef Development Kit to write and test your Chef cookbooks and recipes,
  • Vagrant, the orchestrator, to contain the virtual machines to test the scripts,
  • Virtual Box to launch the virtual machines,
  • Chef needs Ruby to run,
  • Git, the version control system.

You can do all development and testing on your workstation for free, but to see your scripts running in a real cloud, you can set up an account at a cloud provider. Amazon Web Services (AWS) offers a free tier where you can launch small server instances for free.

In this guide I will use a Windows computer as a workstation, but all tools work on Mac and Linux computers.

Install the DevOps development tools on Windows

Set up your computer


Display the file extensions

In Windows 10
  • Open Windows explorer
  • On the View tab select
    • File name extensions
    • Hidden items
In Windows 7
  • Open Windows Explorer and select Organize
  • Select Folder and search options
  • On the View tab
    • Select Show hidden files, folders, and drives
    • Uncheck Hide extensions for known file types



Apple Id

If you don’t have an Apple ID create one. You can obtain one without a credit card:

  • Start iTunes on your Macintosh,
  • Click the drop down menu in the upper right corner,
  • If Apps is visible, select it, otherwise click Edit Menu…
    • Select Apps to display it in the drop down.
  • Search for a free app in the App Store and start to download it,
  • Create a new Apple Id and select None for credit card type.

Show the user home directory

  • Open Finder
  • In the Finder menu select Preferences
  • In the Favorites section select the checkbox next to your user name

Create a directory for optional applications

Create the opt folder in the root of the harddisk.

Remote Desktop Client


The Windows operating system already has a Remote Desktop Client


Download the Microsoft Remote Desktop app from the App Store

  • Open the App Store
  • Search for “microsoft remote desktop”
  • Click the Microsoft Remote Desktop icon
  • Click the blue Get button
  • Click the green Install App button

Text Editor


Notepad ++

Install a good text editor. If you do not have a favorite, I recommend Notepad ++

Configure Notepad ++

Tab settings

  • In the Settings menu select Preferences
  • In the Tab Settings section set the Tab size to 2 and select Replace by space



  • Download it from
  • Double click the downloaded ZIP file to extract the application,
  • Drag the Atom application into Applications

Terminal Window


Install iTerm2, a smart terminal emulator to issue Bash commands and log into Linux servers.

Configuration suggestions are at

Enable unlimited scroll back

  • Start iTerm2 and open the preferences window by pressing ⌘, (command-comma)
  • On the Terminal tab click the Unlimited scrollback check box.



  • Navigate to to download Git for Windows. The page automatically downloads the installer for the operating system you use.
  • Install the application
  • Accept the default values, including these:
    • Make sure the Windows Explorer integration for Git Bash is checked.
    • Enable the Git tools in the command prompt too
    • Line endings for Windows computers
    • Terminal emulator
    • Caching
    • Finish the installation

Configure Git for Windows

Enable Page Up and Page Down

  • In Windows Explorer right click in the white area and select Git Bash Here,
  • Stretch the Bash window to the full width of the page to have more room to work later,
  • Right click the Bash window and select Options…,
  • On the Window tab,
    • Click the Current size button to save the size,
    • Select PgUp and PgDn scroll without modifier to be able to scroll quickly up and down in the window with the Page up and Page down buttons.

Set up Git Bash to always run as Administratort

Certain commands need elevated rights to run, so we will set up the Bash window to run as administrator.

  • In Windows 7 click the Windows Start button and type bash
  • Right click the found link and select Properties
  • On the Compatibility tab select Run this program as administrator


  • Navigate to to download Git for Windows. The page automatically downloads the installer for the operating system you use.
  • This app is not trusted by Apple, so to install it
    • Control-click the downloaded file and select Open
    • Click the Open button to confirm the action

Virtual Box

Download Virtual Box from and follow the instructions to install it.

At the time of writing this was the section where the installer files were referenced


Run the downloaded installer file and accept all default values.

The installer starts the application. You can close it, we will use other tools to start it in the future.


Windows and Macintosh


The default credentials of a Vagrant server are:

  • UserName: vagrant
  • Password: vagrant



  • Download Ruby from
  • Install the 32 bit version, as the 64 bit version is fairly new and has compatibility issues. Download the latest Ruby installer. Pick the file that does not have the (x64) at the end.
  • Accept the default values but check Add Ruby executables to your PATH

 Ruby Development Kit

The Ruby Development Kit is need for certain Ruby gems, so install it.

  • Download the Ruby Development Kit from the DEVELOPMENT KIT section of
  • If you have installed the 32 bit version of Ruby make sure you install the 32 bit version of the DevKit. Select the Development kit that matches the Ruby version you installed. At the time of writing the latest development kit is under “For use with Ruby 2.0 and above (32bits version only)
    • Run the installer to extract it to a permanent location (C:\RubyDevKit)
    • Open a command window in the C:\RubyDevKit folder
    • Run these commands in the command prompt to install it.
      • ruby dk.rb init
      • ruby dk.rb install
  • For more information see


Ruby is already a part of the operating system.

Chef Development Kit

Windows and Macintosh

Terraform by Hashicorp

Windows installation

  • Unzip the downloaded package to C:\HashiCorp\Terraform
  • Add the Terraform directory to the path of the computer. It is not added automatically on 64 bit Windows.
    • In Windows Explorer right click Computer and select Properties,
    • On the left side select Advanced System Settings,
    • At the bottom click the Environment Variables… button,
    • In the System Variables box select Path and click the Edit… button,
    • Add the following to the end of the Variable value. (Don’t forget to start with the semicolon as the separator).
  • To create RDS (Relational Database Service) instances in AWS, install the Amazon Web Services Command Line Interface (See instructions below)
  • Configure the GitBash window to run as administrator to be able to execute the “terraform get” command to get local modules during development
    • Right click the shortcut of the Bash window you use and select Properties
    • On the Compatibility tab check Run this program as an administrator

For more details see

Macintosh installation

  • Download Terraform from
  • Double click the downloaded ZIP file to extract the application
  • Create a directory, terraform for the Terraform application in the /opt folder
  • Move the Terraform application into the terraform directoryin the /opt folder
  • Add the location to the path
    • Open the terminal iTerm2 window
    • Navigate to the home directory
      cd ~
    • Find out if the Bash Profile file exists
      ls .bash_profile
    • If the result is ls: .bash_profile: No such file or directory
      • Create the Bash Profile file.
      • Add the following lines
        # PATH Export
        export PATH
        unset DYLD_LIBRARY_PATH

Graphviz – Dependency Graph Visualization Software

We will use this utility to display the Terraform graphs.

  • Download from
    There are really two dots in the address :-)
  • Execute the installer
  • You may need to add the location to the Path environment variable. Make sure you use the actual folder name, as it contains the version of the application.
    ;C:\Program Files (x86)\GraphvizX.XX\bin


Set up the DevOps development environment in Beginner’s Guide to DevOps Engineering part 2.


to the Tutorials page

Enable file sharing on Windows servers in Amazon Web Services (AWS)

When you create an EC2 instance in Amazon Web Services (AWS) the security group (firewall) blocks all ports that are not explicitly opened. To make file sharing possible on Windows servers, open the following ports in the security group of the server:

  • TCP 139
  • TCP 445

How to copy files to the clipboard

When you work in the graphical user interface (GUI) of your operating system, and want to copy the contents of a file to the clipboard, you can use the following commands:

On OS X run: cat FILE_NAME | pbcopy
On Linux run: cat FILE_NAME | xclip
On Windows (via Cygwin/Git Bash) run: cat FILE_NAME | clip

How can I tell if a Chef cookbook is frozen

If a Chef cookbook is “frozen”, the cookbook is protected against accidental overwrite.

When someone tries to overwrite it the
Version ... of cookbook COOKBOOK_NAME is frozen. Use --force to override"
message appears.

To check if a Chef cookbook is frozen on the Chef server execute the following command
knife cookbook show COOKBOOK_NAME latest | grep frozen

To upload a Chef cookbook and freeze it to prevent accidental overwrite
knife cookbook upload COOKBOOK_NAME --freeze

To overwrite a frozen cookbook upload it with
knife cookbook upload COOKBOOK_NAME --force

To freeze the referenced cookbooks when you upload them with BerkShelf upload them with
berks upload
(do not use the –no-freeze option)