Chef Data Bags

Create a data bag

There are secrets in most of the Chef cookbooks that we want to protect. We don’t want to give out user names, passwords and AWS keys.  In Chef the best place to hide these secrets is the Encrypted Data Bag.

A Data Bag is a JSON file that we can encrypt, so we can store it in version control with the rest of the cookbook.

To make  continuous integration and delivery (CI/CD) easier, store the Encrypted Data Bags in the cookbook folder structure and commit them together with the rest of the cookbook into version control (Git)

To make sure the unencrypted secret is not committed into version control, add the following line to the  .gitignore file

# Ignore the unencrypted Data Bags
data_bags_unencrypted/

The structure of the cookbook should look like this. Store the unencrypted Data Bags with the original values in the data_bags_unencrypted folder

data-bags-01-unencrypted-highlighted

Create a folder for the Data Bag

data-bags-02-unencrypted-databag-folder

Create a file for the Data Bag Item. The name of the file and the Data Bag Item id should be the same.

data-bags-03-unencrypted-item-file

Enter the Data Bag Item values and save the file.

{
 "id": "admin",
 "AccessKey": "XXXXX",
 "SecretKey": "YYYYY"
}

Automate the data bag encryption

Create the folder structure

  1. Create a folder for data bag related files on the same level as the cookbooks folder. Name it data_bags

  2. Get the data bag encryption secret file from your Chef server administrator and place it in the data_bags folder.
  3. Create a folder for Chef related scripts on the same level as the cookbooks folder. Name it devops-chef-scripts

Create the automation script

Create the following script and name it encrypt_databag.sh. This script

  1. Encrypts the Data Bag,
  2. Uploads the encrypted data bag to the Chef server,
  3. Saves the encrypted data bag in the data_bags folder of the cookbook on your workstation.

Enter the name of the secret key file in the middle of the script.

#!/bin/bash
 if [ -z $1 ] || [ -z $2 ]
 then
 echo "Please supply the arguments: DATABAG_NAME ITEM_NAME"
 echo -n "Press a key to exit" #'-n' means do not add \n to end of string
 read # No arg means dump next line of input

else

# Create the data bag if does not exist
 knife data bag create $1

# Encrypt the databag and upload it to the Chef server
 knife data bag from file $1 $1/$2.json --secret-file ../../../data_bags/>>>YOUR_SECRET_KEY_FILE_NAME<<<

# Create a directory for the encrypted databag on the workstation
 mkdir -p ../data_bags/$1

# Download the encrypted data bag
 knife data bag show $1 $2 -F json > ../data_bags/$1/$2.json

echo "Encrypted data bag has been created at ../data_bags/"$1"/"$2".json"
fi

Add execution right to the file

chmod +x ./encrypt_databag.sh

Encrypt the data bag

Open a Bash window in the data_bags_unencrypted folder

Execute the following command

../../../devops-chef-scripts/encrypt_databag.sh [DATA_BAG_NAME (folder name)] [ITEM_NAME (file name)]

The script will create a folder for the Data Bag in the “data_bags” folder and save the encrypted Data Bag file it it.

Leave a Reply