Resize the VirtualBox image hard drive

When your VirtualBox virtual machine’s hard drive fills up Virtual Box does not provide a user interface to extend it. If you run VirtualBox on a Macintosh workstation

  1. Stop the virtual machine
  2. On a Macintosh workstation, the virtual machine image files are located at ~/VirtualBox VMs
  3. The name of the subdirectory and the image file matches the name of the machine
  4. Open a terminal window
  5. Navigate to the VirtualBox application directory
    $ cd /Applications/
  6. Check the current size of the hard drive. Place a backslash in front of every space character in the path.
    VBoxManage showhdinfo ~/VirtualBox\ VMs/Win\ 10/Win\ 10.vdi
  7. Resize the drive. Place a backslash in front of every space character in the path, and specify the size in megabytes, so 100 GB is approximately 100000 MB
    VBoxManage modifyhd --resize 100000 ~/VirtualBox\ VMs/Win\ 10/Win\ 10.vdi

    UMG/Win\ 10\ UMG.vdi

  8. Check the size again to make sure the resizing succeeded.

Extend the volume on the enlarged partition

Windows 10 virtual machine

  1. Start the virtual machine
  2. Click the Start button and the Gear icon to start the control panel
  3. Type admin into the search box and select Administrative Tools
  4. Double-click Computer Management
  5. Select Disk Management
  6. Right-click the partition and select Extend Volume
  7. Click Next on every page of the wizard to extend the volume to the maximum available size.

Ruby tips and tricks

Bang methods

(Exclamation point at the end of the method name)

There are methods that have a permanent or dangerous version. The exclamation point designates to use the dangerous version of the method.

String manipulation

The bang versions of the string manipulation methods (with the exclamation point), modify the string variable in place. Some of these methods are

  • sub
  • gsub
  • reverse
  • sort

The original string ‘gsub’ substitution method:

original = 'My old cat'
new = original.gsub('old', 'new')

original = 'My old cat'
new ='My new cat'

The dangerous (bang) ‘gsub’ method with the exclamation point modifies the original variable:

original = 'My old cat'
original.gsub!('old', 'new')

original = 'My new cat'


The script exits with Kernel::exit, but Kernel::exit! causes an immediate exit, bypassing any exit handlers.


Find the AWS account number

The AWS account number uniquely identifies the AWs account you are working with. All AWS “arn” identifiers contain it, and you need to know it when you want to share AMIs with other accounts.

If there are no resources created yet in the account, you can find the account number in the “arn” of your user account.

  1. Log into the AWS Console
  2. In the upper right corner click your username and select My Security Credentials
  3. On the left side select Users, and click any of the usernames in the list
  4. The account number is part of the “User ARN”

Execute Terraform scripts with Octopus

Terraform is a very powerful, free command-line tool to launch servers in any cloud or virtual machine environment. Hashicorp, the creator of Terraform just introduced the paid Terraform Enterprise server, that orchestrates the execution of the Terraform scripts.

Octopus is another tool that added Terraform orchestration functionality in version 2018.3

In this example, we will set up a Multi-Tenant Octopus project to launch servers in AWS with Terraform scripts and configure them with Chef.

Terraform script location

Create a GitHub repository for the Terraform script

GitHub is the most flexible way to store the Terraform scripts. Octopus can download and execute the same script in multiple Octopus projects. This eliminates the code duplication and makes the scripts available for execution from the command line.

Create a GitHub user with read access to the repository

If the GitHub repository is private, Octopus will need a way to access it with a username and password.

Tag the source code version, Octopus needs a version to refer to the version of your script

git tag 1.0.0
git push --tags

Automation with Octopus

We will create a Multi-Tenant Octopus project to be able to use the same Octopus project for multiple server types in multiple AWS environments.

We will create

  • an Octopus project group to hold the AWS launch projects,
  • a generic Octopus project to launch an AWS EC2 instance,
  • an operating system specific Octopus project that calls the generic project
  • environments for each AWS account, application, application environment,
  • tenants for each server.

Create the Environments

  1. On the Infrastructure Octopus page select Environments
  2. On the Environments page click the ADD ENVIRONMENT button
  3. Create the environments you need (aws-dev, aws-qa, aws-uat, aws-prod)

Create a Lifecycle

The lifecycle governs how the project is promoted between environments.

  1. On the Library Octopus page select Lifecycles
  2. Click the ADD LIFECYCLE button
  3. Enter the name and description of the Lifecycle and click the ADD PHASE button

  4. Enter a name for the Phase and click the ADD ENVIRONMENT link to select the environment
  5. Select the environment and the Users will need to manually… radio button
  6. Keep the All environments must be deployed… and click the Save button.

Create a project group

The project group holds your launch projects together.

  1. On the Projects Octopus page select ADD GROUP
  2. Enter a name and description for the project group

Create a GitHub feed

This feed will allow Octopus to connect to GitHub

  1. On the Library Octopus page select External Feeds
  2. Click the ADD FEED button
  3. Select the GitHub Repository Feed type, enter a name and copy the recommended URL. Add the GitHub user’s name and password you have created above, and click the SAVE AND TEST link

  4. Enter the complete name of your Terraform repository and click the SEARCH button to make sure Octopus has access to the private repository. If you do not specify the full path of the repository, the search will display public repositories with similar names.

Store an AWS account reference in Octopus

AWS accounts store the credentials to connect to Amazon Web Sevices

  1. On the Infrastructure page select Accounts
  2. In the Amazon Web Services section click the ADD ACCOUNT button
  3. Enter a name, description, Access Key and Secret Key for the account. Enable the usage in tenanted and untenanted deployments, and click the SAVE AND TEST link.

Store SSH Keys in Octopus

Store your private keys in Octopus, so you can reference them in your variables.

  1. On the infrastructure page select Accounts
  2. In the SSH Key Pairs section click the ADD ACCOUNT button
  3. Enter the SSH key name, the username, and click the up arrow to upload the SSH private key file to the Octopus server.
    1. Select the Include in both tenanted and untenanted deployments radio button.
    2. In the Associated Tenants section select the tenant groups you want to use the key for.
  4. Click the SAVE button to save the certificate in Octopus

Create a Project

The project contains the reference to the Terraform script to launch the server, the connection to the GitHub repository where the Terraform script is stored, and credentials to access the AWS account. The Terraform script and GitHub connection are going to be the same for every server launch, but the AWS account credentials have to be different for each AWS account. If you work with multiple AWS accounts, you need to create separate projects for each AWS account.

  1. On the Projects Octopus page scroll down to the DevOps project group and click the ADD PROJECT button
  2. Enter a name and description for the new project, and click the ADVANCED SETTINGS link
  3. Select the lifecycle you have created earlier, and click the SAVE button.

Create the launch process

  1. On the left side select the Process menu item
  2. On the Process page click the ADD STEP button
  3. Type “terraform” into the filter field and select Apply a Terraform template step

Create the variables

  1. In the project click the Variables menu item
  2. Enter the name of the variable and click the value field
  3. Click the CHANGE TYPE link
  4. Select the Amazon Web Services Account type
  5. Enter a description and select the AWS account you have created above, and click the DONE button.
  6. Create the rest of the variables.
  7. Add prefixes to the variables that will be set in the OS-specific parent project, in the tenant, environment, and target.

Update the variable names in the Terraform script

When we added prefixes to the project variables to show which component responsible for setting it, we have changed the variable name. Update the Terraform script to reflect it.

ami = "#{os.instance_ami}"

Create an OS-specifc project that calls the generic project

There are values that are different for each operating system in each AWS account.

The AMI ID depends on the operating system and the application environment. We can test the new version of the operating system in the development environment, while we are still launching servers with the old, tested version for production.

The common, OS-specific security group ID is different in every AWS account. It depends on the AWS account and the operating system.

To be able to set the value of this variable based on the OS and the AWs account, or the OS and application environment, we need to create an OS-specific project that passes in the appropriate variable values to the generic project.

  1. On the Projects page scroll down to the DevOps project group and click the ADD PROJECT button
  2. Enter the name and description of the new project and click the ADVANCED SETTINGS link
  3. Select the DevOps-aws-lifecycle
  4. Click the Process menu item
  5. Click the ADD STEP button
  6. Select the Deploy a Release step
  7. Enter the name for the step and select the generic AWS EC2 Launch project

Create variables

Set the OS-specific variables in the OS-specific parent project. To avoid variable name collision later add the parent.prefix to the variables passed in from the parent to the child project.


  1. Click the Settings menu item
  2. Require tenant for the deployment, allow the deployment without target, and ask if there is an error. Chef raises an error when it reboots the server, so if the Chef cookbook contains the reboot resource, we need to accept the error for a successful deployment.
  3. Click the Variables menu item
  4. Add the OS-specific variables

Create Library Variable Templates

Library Variable Sets specify the variables that have to be set in a higher level component, like a tenant, environment, and target.

  1. On the Library Octopus page select Variable Sets
  2. On the Variable Sets page click the ADD NEW VARIABLE SET button
  3. Enter a name and description for the library variable set
  4. Click the VARIABLE TEMPLATES tab
  5. On the VARIABLE TEMPLATES tab click the ADD TEMPLATE link
  6. Add the variables to the Library Variable Template. For pre-defined choices use the Drop down control type. To avoid variable name collisions when we pass the variable values to the child project later, add the “parent.” prefix to every variable.

  7. Save the Library Variable Template

Add the Library Variable Templates to the OS-specific project

Add the Library Variable Sets to the OS-specific project, so the tenants can set them with the instance-specific values.

  1. Open the project and select the Variables menu item
  2. Select the Library Sets menu item
  4. Select the newly created Library Variable Set

Pass the variables from the parent project to the child project

The parent project can pass variables into the child project. Select the variable to pass into the child project. You need to do this in every parent project that call the common child project.

  1. In every parent project select Process
  2. Select the process that calls the child project
  3. Click the Variables section
  4. Click the ADD VARIABLE link
  5. Add the OS-specific variables set in the parent project and the host-specific variables defined in the Library Variable Template and set in the tenant.

Create a Tenant

Tenants are server types that use the same Octopus project. A tenant can be an application or a software development project where the servers share common attributes and run on the same operating system type, like all Linux, or all Windows, so the same Terraform script can launch them.

  1. In Octopus select the Tenants page and click the ADD TENANT button
  2. Enter the name of the new tenant
  3. Click the Setting button to select a logo for the tenant
  4. Click the logo placeholder to open the region
  5. Click the file selection button to browse to a 100×100 image file

Tag the Tenant

Tags help you to organize your tenants.

  1. On the Library page select Tenant Tag Sets
  2. Click the ADD TAG SET button
  3. Enter a name and description for the tag set, and create the first tag, and click the ADD link
  4. Click the EDIT TAGS link to add tags to the tenant
  5. Click the small arrow to select the tags

Connect the Tenant to the project

  1. On the Tenants page select the tenant you want to add to the project
  2. Click the CONNECT PROJECT button
  3. Select the project and all environments you want to launch instances for this tenant (software project)

Set the tenant variable values

  1. On the Tenant page select Variables
  2. Select the COMMON VARIABLES tab
  3. Click the name of the template to open it
  4. Set the value and click the Bind icon
  5. When all required variables are set, the SAVE button becomes available.

Create a deployment target (server)

Deployment targets are the servers we are launching. In enterprise settings, all servers have a standardized name that conforms to the company standards.

  1. In the
  2. Click the ADD DEPLOYMENT TARGET button
  3. Select the Cloud Region radio button
  4.  Enter the server name, deployment and a role, and do not press the Save button yet
  5. In the Restrictions region select the Include in both tenanted and untenanted deployments radio button select the Tenant you have created.

Create project releases

If there was a change in the child project, we need to create a release for the child project and all parent projects that call it.

Create a release of the child project

  1. On the project page click the CREATE RELEASE button
  2. If you use GitHub as the Terraform script source, the no version specified message displays in the Packages section
  3. Tag the GitHub repository. Octopus needs a version to link to a GitHub package.
    git tag 1.0.0
    git push --tags
  4. Click the SEARCH link to find the latest version of the repository
  5. Select the latest version of the GitHub repository
  6. Click the SAVE button

Create releases for the parent projects

  1. Open the project
  2. Click the CREATE RELEASE button
  3. Enter the Release Notes, so later you can identify this release

Deploy the server

  1. Open the OS-specific project
  2. Click the Releases menu item
  3. Select the release
  4. Click the DEPLOY TO button and select the environment

    1. If the project was already deployed to an environment, that environment does not show up in the list, so click the ellipsis and select Delpoy To from the dropdown menu.
    2. Select the environment from the list
  5. Click the tenant section to select the tenant (server) to launch
  6. Click the DEPLOY button

Adding new servers, operating systems, environments, and AWS accounts

New Server

  1. Create a new Tenant


  1. Update the AMI in the OS-specific parent project
  2. Open the Releases page of the parent project
  3. Select the latest release
  4. Click the UPDATE VARIABLES link

New Application

  1. Create a new Tenant Tag in the “DevOps Applications” Tenant Tag Set
  2. Add the new Tenants to the Tenant Tag of the application

New Operating System

  1. Clone one of the DevOps AWS EC2 OS… projects
  2. Update the project settings, because those are not cloned
    1. Tenants required for all deployments
    2. Deployments with no target allowed
  3. Update the project variables for the new operating system
  4. Create a release of the new project
  5. Create new Tenants and link them to the new OS project

New Environment or AWS account

  1. Add the new account if necessary. See Add AWS AccountStore an AWS account reference in Octopus above.
  2. Create the DevOps-env-[environment][account number] environment
  3. Add the new environment to the “DevOps-aws-lifecycle”
  4. Add the new environment to the variables in the
    1. Add to the base project: “DevOps AWS EC2 Launch”
      1. account.aws_key
      2. account.aws_key_name
      3. environment.aws_account_credentials
      4. environment.chef_environment
      5. environment.instance_tag_environment
    2. Add to all OS specific projects: “DevOps AWS EC2 OS…
      1. parent.os.account.instance_common_security_group_id

New Chef Attribute

To pass in a new Chef attribute from the Tenant

  1. Add the new attribute to the chef_attributes.json file with variable substitution
     "set_hostname": "#{host.set_hostname}",
     "agent_name": "#{host.agent_name}",
     "add_to_domain": "#{host.add_to_domain}"
  2. Add the new variable to the Library Variable Set’s Variable Template with the parent. prefix
  3. Add the new variable to the passed in variable list in every parent project process. See Pass the variables from the parent project to the child project


To save the variable values to the log add the following two variables to the project

OctopusPrintVariables True
OctopusPrintEvaluatedVariables True

For the change to take effect

Create a new release


Update the variable snapshot of the existing release

  1. On the Project -> Releases  page select the latest release
  2. In the Variable Snapshot section click the SHOW link
  3. Click the UPDATE VARIABLES link

More info at




Migrating from Chef Client version 12 to 13

Chef is under heavy development, every new major version introduces new features, and many times changes, deprecates, or removes some commands or options.

Chef Client 13 introduced a new way of handling reboots and Windows scheduled tasks.

reboot resource

In Chef version 12, the “:reboot_now” action continued the execution of the Chef cookbook but after the specified “delay_mins” the instance was forcefully rebooted even if an application was still running. This could unexpectedly interrupt software installations that did not finish during the specified delay.

This is not a problem if the reboot resource was called with the “:request_reboot” action because in that case, the delay before the reboot starts when the cookbook execution has completed.

In Chef version 13, when the reboot command is ultimately sent to the operating system handling the”:reboot_now” or”:request_reboot” action, a Ruby error is raised to stop the execution of the Chef cookbooks. This prevents the execution of further commands, that would be interrupted by the imminent reboot but puts misleading error messages into the “C:/Chef/Cache/chef-stacktrace.out” file.

Generated at …
Chef::Exceptions::Reboot: Rebooting server at a recipe’s request. Details: {:delay_mins=>1, :reason=>”…”, :timestamp=>…, :requested_by=>”…”}

This is unnecessary in the case of the “:request_reboot” action because the instance is only rebooted when the Chef cookbook execution has already been completed. Chef still raises the error in both cases.

In Test Kitchen

In Chef 12 when the instance was rebooted the DevOps engineer had to wait for the instance to reboot and issue the “kitchen converge” command again to execute the cookbook again.

In Chef 13 Test Kitchen executes the “kitchen converge” command if the cookbook execution failed. This simulates the execution of the Chef Client when the instance starts after the reboot. The error raised by the reboot command helps to test cookbooks with multiple reboots without the need of paying attention to every reboot. This new behavior causes a timeout error when Test Kitchen tries to access the instance while it is still rebooting.

To avoid this we need to add lines to the provisioner section of the .kitchen.yml file

  name: chef_zero
  always_update_cookbooks: true
  retry_on_exit_code: # An array of exit codes that can indicate that kitchen should retry the converge command. Defaults to an empty array.
    - 35              # Reboot is scheduled
    - 20              # The Windows system cannot find the device specified.
    - 1               # Generic failure
  wait_for_retry: 120 # Number of seconds to wait between converge attempts. Defaults to 30.
                     # Chef starts another converge after reboot was requested.
                     # If the 'wait_for_retry' (seconds) is shorter than the 'delay_mins', the converge starts before the reboot happens.
                     # 'wait_for_retry' (set in seconds!) has to be greater than any 'delay_mins' passed to reboot
  max_retries: 10 # Number of times to retry the converge before passing along the failed status. Defaults to 1.

It is very important to set the value of the “wait_for_retry” option greater in seconds, than the longer “delay_mins” set in any “reboot” resources of the cookbook, for Test Kitchen to wait with the retry of the cookbook execution until the instance is rebooted. If Test Kitchen does not wait for the reboot, the next “converge” starts before the instance is rebooted, and the execution will be interrupted by the reboot. After the “wait_for_retry” delay Test Kitchen will try to connect to the instance for “max_retries” times waiting “wait_for_retry” seconds between the tries until the box starts to accept WINRM connections again.


windows_task resource

There are two major changes in the windows_task resource.

If the scheduled task already exists and the resource tries to create it again with the same values, an error is thrown

It looks like the error is in the step that tries to parse the start time of the task. Make sure

To change an existing scheduled task

Chef version 13 does not understand the “:change” action. To change an existing scheduled task, use the “:create” action. If you run your cookbooks on multiple Chef versions, you have to make sure those work on the old and the new versions of Chef.

The transition is not so simple, both versions of Chef throw errors when we switch to the :create action.

If you try to modify an existing scheduled task with the “:create” action in Chef 12:

NoMethodError: undefined method `tr’ for nil:NilClass


in Chef 13:

Invalid starttime value.


Use PowerShell to modify existing scheduled tasks.

To delay the execution of the chef-client scheduled task by two hours to give enough time for the application install between reboots:

$taskpath = "\\"
$taskname = "#{chef_client_task_name}"

$tsDelay = New-TimeSpan -Days 0 -Hours 2 -Minutes 0
$newStartTime = (get-date) + $tsDelay

$tsRepetition = New-TimeSpan -Days 0 -Hours 0 -Minutes 30
$tsDuration = ([timeSpan]::maxvalue)

$trigger = New-ScheduledTaskTrigger -Once -At $newStartTime -RepetitionInterval $tsRepetition -RepetitionDuration $tsDuration

$user = (schtasks.exe /query /s localhost /V /FO CSV | ConvertFrom-Csv | Where { $_.TaskName -eq ($taskpath + $taskname) })."Run As User"
$principal = New-ScheduledTaskPrincipal -UserID $user -LogonType S4U -RunLevel Highest

set-scheduledtask -TaskPath $taskpath -TaskName $taskname -Trigger $trigger -Principal $principal

To set the RunAs user of a Scheduled Task

$user = "NEW_USER_NAME"
$password = "NEW_USER_PASSWORD"

$taskpath = "\\"
$taskname = "TASK_NAME"

set-scheduledtask -TaskPath $taskpath -TaskName $taskname -User $user -Password $password




How to upgrade Jenkins

Jenkins regularly releases new versions. To upgrade Jenkins

On Linux

To install the Generic Java Package (.war)

  1. Write down the current version of Jenkins you have on your server
  2. Download the Jenkins “Generic Java Package (.war)” file from your workstation
  3. Upload the installer to Artifactory or other HTTP repository
  4. SSH into the Jenkins server
  5. Switch to sudo mode
  6. Stop the Jenkins service with
    service jenkins stop
  7. Navigate to the Jenkins directory
    cd /usr/lib/jenkins/
  8. Rename the current Jenkins .war file to include the version, so you can restore it if the new version does not work as expected
    mv jenkins.war jenkins_2.46.3.war
  9. Download the Jenkins war file from Artifactory
    url -O http://THE_IP_ADDRESS_OF_THE REPOSITORY/.../jenkins.war
  10. Start the Jenkins service
    service jenkins start


Calling a resource in the Chef recipe

During major Chef Client version upgrades, some instructions need to be changed based on the version of the Chef Client. For example, upgrading from version 12 to version 13, the “windows_task” resource requires a different action to make changes to existing scheduled tasks.

To call a Chef resource in your cookbook, you need another resource to call from. If you execute the notifies command by itself you get the error message:

undefined method `declared_key’ for “….[…]”:String

In this example, we call the “windows_task” resource to delay the execution of the chef-client by two hours to give enough time for the server configuration to complete. In Chef version 12 we need to set the action to “:change”, in Chef version 13, it needs to be “:create”.

if ( Chef::VERSION.to_f >= 13.0 )
  puts "Version 13.0 or higher"

  ruby_block "call delay chef-client" do
    block do
      # Do nothing, "block do" has to be here
    notifies :create, "windows_task[delay chef-client]", :immediately
  puts "Before version 13.0"

  ruby_block "call delay chef-client" do
    block do
      # Do nothing, "block do" has to be here
    notifies :change, "windows_task[delay chef-client]", :immediately

future_time = + 7200               # Offset is in seconds: 2 hour delay
new_start_day = future_time.strftime('%m/%d/%Y')
new_start_time = future_time.strftime('%H:%M')

windows_task 'delay chef-client' do
  task_name 'chef-client'
  start_day new_start_day
  start_time new_start_time
  action :nothing
  guard_interpreter :powershell_script
  only_if "(Get-ScheduledTask | Where-Object {$_.TaskName -eq 'chef-client'}).TaskName.Length -gt 0"


Make decisions in your Chef recipe based on the version of the Chef Client

There are times when you have to make a decision in your Chef recipe, based on the version of the Chef Client installed on the node.

There are two ways to get the version of the installed Chef Client:


To make a decision based on the installed Chef Client version

if ( Chef::VERSION.to_f >= 14.0 )
  puts "Version 14.0 or higher"
  puts "Before version 14.0"


Knife commands

Knife is a ChefDK command line tool to access the Chef server. We use it to upload our cookbooks, environment files, and data bags to the Chef server, and query the server for information on cookbooks and nodes.

Get the list of cookbooks used by a node with version information

knife node show <node-name> -a cookbooks


Test your cookbook in Chef Test Kitchen against multiple version of the Chef Client

In large environments, during the Chef Client version change, some older servers still run the prior version of the Chef Client, the newly created servers launch with the new version of the Chef Client.

It is very important to test your cookbooks with the old and the new versions of Chef Client.

To specify the Chef Client version for the Test Kitchen “converge” run, add the highlighted three lines to the provisioner section of the .kitchen.yml file:

  name: chef_zero
  product_name: chef          # Install the Chef Client
  product_version: 14.0.0   # Set the Chef Client version, default=latest
  install_strategy: always    # Forces the installation of the specified Chef Client version even if another version is installed

During “converge”, test kitchen will download and install the specified version of Chef Client.


If the installed Chef Client version is newer than the specified version, Test Kitchen downloads the specified version but executes the already installed newer version.