Unable to satisfy the following requirements error message in Chef Test Kitchen

When Berkshelf in the Chef Test Kitchen cannot resolve the cookbook dependencies, it displays the following error message during converge:

Unable to satisfy the following requirements

To help Berks to start a fresh calculation, delete the Berksfile.lock file and enter the following into the command window:

berks install

DevOps Engineering part 2. – Create and test your first cookbook in 5 minutes

In the first part of the series, Beginner’s Guide to DevOps Engineering Part 1. we have already installed the DevOps development tools.

Create and test your first cookbook in 5 minutes

 Set up the Chef working folder

  1. Create a folder for the Chef development
    1. on Mac ~/Chef
    2. on Windows C:\Chef
  2. In the Chef folder create a sub-folder cookbooks
  3. Right-click the cookbooks folder
    1. on Mac select Services, iTerm2 in Finder
    2. on Windows  select Git Bash Here

Create your first cookbook

    • Enter the following command into the Bash window to create a new cookbook
      chef generate cookbook test

Specify the program you want to use to open .rb files

on Mac

  1. In Finder navigate to ~/Chef/test/recipes
  2. Right-click the default.rb file and select Open With, Atom

on Windows

  • In File Explorer navigate to the C:\Chef\test\recipes older
  • Right-click the default.rb file
    • Select Open with
    • Select the Always use… checkbox  and click the More apps link
  • Select Notepad++ and click the OK button

 

Let’s create a file with Hello world in it.

      • Enter the following to create your first recipe
        • on Mac
          file '/Users/YOUR_USERNAME/Desktop/helloworld.txt' do
           content 'Hello world'
          end
        • on Windows (make sure you use double backslashes!!!)
          file 'C:\\Users\\YOUR_USERNAME\\Desktop\\helloworld.txt' do
           content 'Hello world'
          end
      • Save the file
      • In the bash Window navigate to the test cookbook directory
        cd test
      • Execute your first recipe with the following in the Bash window
        chef-client --local-mode recipes/default.rb
      • Navigate to the Desktop and open the helloworld.txt file to see the result of your script.

Test your cookbook on a virtual machine

We will use Test Kitchen and Vagrant to launch virtual machines. Currently, only Linux images are available for Vagrant, so we will modify our recipe to select between Linux and Windows and act accordingly.

Open the default.rb recipe and update it to look like this

case node['os']
when 'linux'
  file "/tmp/helloworld.txt" do
    content 'This file was created by Chef!'
  end
when 'windows'
  file "C:\\Chef\\helloworld.txt" do
    content 'This file was created by Chef!'
  end
end

Enter the following commands into the Bash window

      • Display the available test configurations ( suites )
        kitchen list
      • Launch a virtual Linux machine and test your recipe
        kitchen converge ubuntu

        The first time you launch a virtual machine with an operating system you have never used on your workstation, Vagrant has to download the machine image from the Internet. It can take 5-10 minutes. When the virtual machine is created and launched, Test Kitchen will copy your cookbook to it and execute it there.
      • Test kitchen will display the —–> Kitchen is finished message when the cookbook has successfully executed.
      • Log into the virtual machine
        kichen login ubuntu
      • The root prompt appears in the Bash windows. Let’s check if the file has been created or not.
      • To destroy the virtual machine
        kitchen destroy ubuntu

Windows in Vagrant

To test your cookbook on a Windows virtual machine locally, create one for Vagrant. See Launch Windows instances locally with Chef Test Kitchen for the details.

Learn the basics, so you can ask questions

Chef has a steep learning curve. Chef is not just scripting or programming, but you have to understand how Chef works to be able to use it to configure servers. There are many ways to do the same thing and there is not much documentation to recommend the best way. If you search Google, the problems usually have multiple solutions, and many times the “best” answer is selected based on personal preference. To get started, you should familiarize yourself with the tools, because you will use most of them during the development process.

In this guide I will use a Windows computer as a workstation, but all tools work on Mac and Linux computers.

Vagrant

Learn Vagrant to understand how Test Kitchen manages the test servers on your local machine or at AWS. You will not use vagrant directly, but Test Kitchen uses it to launch servers.

Chef

Test Kitchen

Ruby

Terraform


Next:

Working with AWS in Beginner’s Guide to DevOps Engineering part 3.

Back:

to the Tutorials page

Beginner’s Guide to DevOps Engineering part 1.

DevOps engineering (Release engineering) is a relatively new profession. There is a very high demand for experienced DevOps engineers, because more and more large corporations want to migrate their data centers into the Cloud. In this series I will introduce you to the tools of the trade through the development of a Chef cookbook.

The tools of the DevOps engineers are new and most of them are in the active development phase with frequent releases. Some of the new versions have show stopping bugs in them, so it is a good idea to test all new versions of the tools before you uninstall the old one.

To work as a DevOps engineer you need a development environment with multiple tools. Luckily all of them are available for free and easy to set up.

  • The Chef Development Kit to write and test your Chef cookbooks and recipes,
  • Vagrant, the orchestrator, to contain the virtual machines to test the scripts,
  • Virtual Box to launch the virtual machines,
  • Chef needs Ruby to run,
  • Git, the version control system.

You can do all development and testing on your workstation for free, but to see your scripts running in a real cloud, you can set up an account at a cloud provider. Amazon Web Services (AWS) offers a free tier where you can launch small server instances for free.

In this guide I will use a Windows computer as a workstation, but all tools work on Mac and Linux computers.

Install the DevOps development tools on Windows

Set up your computer

Windows

Display the file extensions

In Windows 10
  • Open Windows explorer
  • On the View tab select
    • File name extensions
    • Hidden items
      windows-03-show-file-extensions
In Windows 7
  • Open Windows Explorer and select Organize
  • Select Folder and search options
    windows-02-show-file-extensions
  • On the View tab
    • Select Show hidden files, folders, and drives
    • Uncheck Hide extensions for known file types
      windows-01-show-file-extensions

 

Macintosh

Apple Id

If you don’t have an Apple ID create one. You can obtain one without a credit card:

  • Start iTunes on your Macintosh,
  • Click the drop down menu in the upper right corner,
    apple-id-01
  • If Apps is visible, select it, otherwise click Edit Menu…
    • Select Apps to display it in the drop down.
      apple-id-02
  • Search for a free app in the App Store and start to download it,
    apple-id-03
  • Create a new Apple Id and select None for credit card type.

Show the user home directory

  • Open Finder
  • In the Finder menu select Preferences
  • In the Favorites section select the checkbox next to your user name
    show-home-directory

Create a directory for optional applications

Create the opt folder in the root of the harddisk.

Remote Desktop Client

Windows

The Windows operating system already has a Remote Desktop Client

Macintosh

Download the Microsoft Remote Desktop app from the App Store

  • Open the App Store
  • Search for “microsoft remote desktop”
  • Click the Microsoft Remote Desktop icon
  • Click the blue Get button
  • Click the green Install App button

Text Editor

Windows

Notepad ++

Install a good text editor. If you do not have a favorite, I recommend Notepad ++

Configure Notepad ++

Tab settings

  • In the Settings menu select Preferences
  • In the Tab Settings section set the Tab size to 2 and select Replace by space
    tab-settings-01

Macintosh

Atom

  • Download it from https://atom.io/
  • Double click the downloaded ZIP file to extract the application,
  • Drag the Atom application into Applications

Terminal Window

Macintosh

Install iTerm2, a smart terminal emulator to issue Bash commands and log into Linux servers.

Configuration suggestions are at

https://ruigomes.me/blog/perfect-iterm-osx-terminal-installation/
https://gist.github.com/kevin-smets/8568070

Enable unlimited scroll back

  • Start iTerm2 and open the preferences window by pressing ⌘, (command-comma)
  • On the Terminal tab click the Unlimited scrollback check box.

Git

Windows

  • Navigate to https://git-scm.com/download/win to download Git for Windows. The page automatically downloads the installer for the operating system you use.
  • Install the application
  • Accept the default values, including these:
    • Make sure the Windows Explorer integration for Git Bash is checked.
      git-for-windows-01-intergration
    • Enable the Git tools in the command prompt too
      git-for-windows-02-command-prompt
    • Line endings for Windows computers
      git-for-windows-03-line-endings
    • Terminal emulator
      git-for-windows-04-terminal
    • Caching
      git-for-windows-05-caching
    • Finish the installation
      git-for-windows-05-finish

Configure Git for Windows

Enable Page Up and Page Down

  • In Windows Explorer right click in the white area and select Git Bash Here,
  • Stretch the Bash window to the full width of the page to have more room to work later,
  • Right click the Bash window and select Options…,
  • On the Window tab,
    • Click the Current size button to save the size,
    • Select PgUp and PgDn scroll without modifier to be able to scroll quickly up and down in the window with the Page up and Page down buttons.
      bash-04-window

Set up Git Bash to always run as Administratort

Certain commands need elevated rights to run, so we will set up the Bash window to run as administrator.

  • In Windows 7 click the Windows Start button and type bash
    bash-01-admin
  • Right click the found link and select Properties
    bash-02-admin
  • On the Compatibility tab select Run this program as administrator
    bash-03-admin

Macintosh

  • Navigate to https://git-scm.com/download/mac to download Git for Windows. The page automatically downloads the installer for the operating system you use.
  • This app is not trusted by Apple, so to install it
    • Control-click the downloaded file and select Open
    • Click the Open button to confirm the action

Virtual Box

Download Virtual Box from https://www.virtualbox.org/wiki/Downloads and follow the instructions to install it.

At the time of writing this was the section where the installer files were referenced

virtual-box-01-download

Run the downloaded installer file and accept all default values.

The installer starts the application. You can close it, we will use other tools to start it in the future.

Vagrant

Windows and Macintosh

Important

The default credentials of a Vagrant server are:

  • UserName: vagrant
  • Password: vagrant

Ruby

Windows

  • Download Ruby from http://rubyinstaller.org/downloads
  • Install the 32 bit version, as the 64 bit version is fairly new and has compatibility issues. Download the latest Ruby installer. Pick the file that does not have the (x64) at the end.
  • Accept the default values but check Add Ruby executables to your PATH
    ruby-01-add-to-path

 Ruby Development Kit

The Ruby Development Kit is need for certain Ruby gems, so install it.

  • Download the Ruby Development Kit from the DEVELOPMENT KIT section of  http://rubyinstaller.org/downloads
  • If you have installed the 32 bit version of Ruby make sure you install the 32 bit version of the DevKit. Select the Development kit that matches the Ruby version you installed. At the time of writing the latest development kit is under “For use with Ruby 2.0 and above (32bits version only)
    • Run the installer to extract it to a permanent location (C:\RubyDevKit)
    • Open a command window in the C:\RubyDevKit folder
    • Run these commands in the command prompt to install it.
      • ruby dk.rb init
      • ruby dk.rb install
  • For more information see  http://github.com/oneclick/rubyinstaller/wiki/Development-Kit

Macintosh

Ruby is already a part of the operating system.

Chef Development Kit

Windows and Macintosh

Terraform by Hashicorp

Windows installation

  • Unzip the downloaded package to C:\HashiCorp\Terraform
  • Add the Terraform directory to the path of the computer. It is not added automatically on 64 bit Windows.
    • In Windows Explorer right click Computer and select Properties,
    • On the left side select Advanced System Settings,
    • At the bottom click the Environment Variables… button,
    • In the System Variables box select Path and click the Edit… button,
    • Add the following to the end of the Variable value. (Don’t forget to start with the semicolon as the separator).
      ;C:\HashiCorp\Terraform
  • To create RDS (Relational Database Service) instances in AWS, install the Amazon Web Services Command Line Interface (See instructions below)
  • Configure the GitBash window to run as administrator to be able to execute the “terraform get” command to get local modules during development
    • Right click the shortcut of the Bash window you use and select Properties
    • On the Compatibility tab check Run this program as an administrator

For more details see https://www.terraform.io/intro/getting-started/install.html

Macintosh installation

  • Download Terraform from https://www.terraform.io/downloads.html
  • Double click the downloaded ZIP file to extract the application
  • Create a directory, terraform for the Terraform application in the /opt folder
  • Move the Terraform application into the terraform directoryin the /opt folder
  • Add the location to the path
    • Open the terminal iTerm2 window
    • Navigate to the home directory
      cd ~
    • Find out if the Bash Profile file exists
      ls .bash_profile
    • If the result is ls: .bash_profile: No such file or directory
      • Create the Bash Profile file.
      • Add the following lines
        # PATH Export
        PATH=/opt/terraform:$PATH
        export PATH
        unset DYLD_LIBRARY_PATH
        

Graphviz – Dependency Graph Visualization Software

We will use this utility to display the Terraform graphs.

  • Download from http://www.graphviz.org/Download..php
    There are really two dots in the address :-)
  • Execute the installer
  • You may need to add the location to the Path environment variable. Make sure you use the actual folder name, as it contains the version of the application.
    ;C:\Program Files (x86)\GraphvizX.XX\bin


Next:

Set up the DevOps development environment in Beginner’s Guide to DevOps Engineering part 2.

Back:

to the Tutorials page

Enable file sharing on Windows servers in Amazon Web Services (AWS)

When you create an EC2 instance in Amazon Web Services (AWS) the security group (firewall) blocks all ports that are not explicitly opened. To make file sharing possible on Windows servers, open the following ports in the security group of the server:

  • TCP 139
  • TCP 445

How to copy files to the clipboard

When you work in the graphical user interface (GUI) of your operating system, and want to copy the contents of a file to the clipboard, you can use the following commands:

On OS X run: cat FILE_NAME | pbcopy
On Linux run: cat FILE_NAME | xclip
On Windows (via Cygwin/Git Bash) run: cat FILE_NAME | clip

How can I tell if a Chef cookbook is frozen

If a Chef cookbook is “frozen”, the cookbook is protected against accidental overwrite.

When someone tries to overwrite it the
Version ... of cookbook COOKBOOK_NAME is frozen. Use --force to override"
message appears.

To check if a Chef cookbook is frozen on the Chef server execute the following command
knife cookbook show COOKBOOK_NAME latest | grep frozen

To upload a Chef cookbook and freeze it to prevent accidental overwrite
knife cookbook upload COOKBOOK_NAME --freeze

To overwrite a frozen cookbook upload it with
knife cookbook upload COOKBOOK_NAME --force

To freeze the referenced cookbooks when you upload them with BerkShelf upload them with
berks upload
(do not use the –no-freeze option)

REMOTE HOST IDENTIFICATION HAS CHANGED warning when you try to connect to a server with ssh

If you work in the AWS environment and  launch and terminate servers frequently the same IP is reused many times. When you try to connect to a new server with SSH and the IP address is already in the known_hosts file you will get the following error message.

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:...
Please contact your system administrator.
Add correct host key in /c/Users/.../.ssh/known_hosts to get rid of this message.
Offending ECDSA key in /c/Users/.../.ssh/known_hosts:13
ECDSA host key for ... has changed and you have requested strict checking.
Host key verification failed.

The solution is to delete the line from the “c:/Users/YOUR_USER_NAME/.ssh/known_hosts” file that has the same IP address as the new server.

SSL certificate is missing after installing ChefDK

When you install a new version of ChefDK and try to run Test Kitchen converge you may get the following error message
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: Failed to complete #create action: [SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed

To solve the problem add the following to the Windows environment variables and open a new Command or Bash window for the change to take effect:
SSL_CERT_FILE=C:\opscode\chefdk\embedded\ssl\certs\cacert.pem