Serialize SSH and RSA private keys to store them in a Chef Data Bag

To send RSA private keys to instances, store them in encrypted data bags. The data bag item is a JSON file that contains keys and values inline.

Use base64 encoding

Base64 encoding converts binary data to ASCII format to represent special characters, like line breaks as ASCII text. The result will be larger, 8 / 6th of the original size, as every 8 bit ASCII character only represents 6 bits of data.

To convert a file to base 64 and save it as another file

cat test.pem | base64 > test.pem.base64

If we need the result on the computer’s clipboard to paste it into a field on the screen

cat test.pem | base64 | pbcopy

To use the encoded string we need to decode it in the Chef cookbook.

Replace the newline characters with \n

To place the multi-line RSA key into the value part of the JSON file, we need to replace the new line characters with the “\n” text.

In Atom

on Mac and Windows

  1. Open the RSA key file in Atom,
  2. Press Command-F on Mac, Ctrl-F on Windows to open the Find and Replace window,
  3. On the right side click the Use Regex button,
  4. In the search field enter 
    \r\n
  5. In the replace with field enter 
    \\n
  6. Press the Replace All button

In Visual Studio Code

on Mac and Windows

  • Press Command-F on Mac, Ctrl-F on Windows to open the Find dialog
  • Select the Use Regular Expression button
  • Enter \n into the find, \\n into the replace field

In Notepad++

on Windows

  • Open the RSA key file in Notepad++,
  • In the Search menu select Replace…,
  • Select Extended mode in the Search Mode section,
  • Enter \r\n to the Find what text box ( if the key was generated on a Windows computer using GitBash, search for \n )
  • Enter \\n  to the Replace with text box
  • Press the Replace All button

You can place the single line key into any encrypted Data Bag file. See Data Bags on Data Bag encryption.

HTTP Request Returned 409 Conflict: Client already exists

The Chef server maintains the list of registered nodes and clients in its database.  When you launch a new instance with Chef you may encounter the following error message:

*** Input CHEF_CLIENT_NODE_NAME is undefined, using: IP-0AFE6965
...
*** Starting chef-client
*** Finished chef-client
Printing Log
...
[...] INFO: Client key C:\chef\client.pem is not present - registering
[...] INFO: HTTP Request Returned 409 Conflict: Client already exists
[...] INFO: HTTP Request Returned 403 Forbidden: error
[...] ERROR: Running exception handlers
[...] ERROR: Exception handlers complete
[...] FATAL: Stacktrace dumped to C:/chef/cache/chef-stacktrace.out
[...] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[...] FATAL: Net::HTTPServerException: 403 "Forbidden"

Cause:

In this case the CHEF_CLIENT_NODE_NAME already exists in the client database.You can search for it from a Bash window.

Solution:

Add a line to the Chef configuration to remove the existing node with the same node name

If you use Terraform to launch server instances and configure Chef, add this line to the Chef provisioner:

recreate_client = true

To delete nodes from the Chef server:

Use the knife command to search for the existing node from a Bash window.

To find the node in the Chef server database

knife search node '*:IP-0AFE6965'

To find the client in the Chef server database

knife search client '*:IP-0AFE6965'

If any of them found, those are leftovers from a previous launch. AWS reuse the IDs, so you have to remove the nodes from the Chef server database when you terminate the instances. To delete the unnecessary entries use the following commands:

knife node delete 'IP-0AFE6965'
knife client delete 'IP-0AFE6965'

To remove the terminated instances from the Chef server database you can also use the Chef web interface.

  • On the Nodes tab enter the IP address of the instance into the search box and hit Enter
  • In the Actions column click the down arrow next to the instance and select Delete

How to edit the GitHub README.md file

The GitHub repositories usually contain a README.md file to describe how to use the project. The GitHub web site has a simple editor, but it has a few limitations

  • If you accidentally refresh the page, you lose your changes,
  • The preview pane is wider than the page that will display your file, so the formatting can be different,
  • You have to switch between the editor and the the preview every time you want to see the result.

One of the simplest ways to edit the README.md is to use the Atom editor.

  • Open the README.md file with Atom,
  • In the Packages menu select Markdown Preview, or press Ctrl-Shitft-M to open the preview pane.

As you type in the editor window the preview pane will show the live review of the file.

For the Markdown syntax visit https://guides.github.com/pdfs/markdown-cheatsheet-online.pdf

An error occurred while signing

When you deploy and application in Microsoft Visual Studio with ClickOnce, you need to sign the installer with a key.

To sign the project Visual Studio needs to use the signtool. If the ClickOnce feature is not enabled in Visual studio, it displays the following error message:

An error occurred while signing: SignTool.exe was not found at path C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\bin\signtool.exe

To install the ClickOnce feature and the signtool

On Windows 7

  • In Windows Control panel select Programs and Features,
  • Right click Visual Studio 2015 and select Modify,
  • In the feature list select ClickOnce tools.

On Windows 10

  • In Windows Control Panel select System,
  • On the left side select Apps & features,
  • In the app list click Microsoft Visual Studio 2015,
  • Click the Modify button,
  • In the Visual Studio setup click the Modify button again,
  • In the feature list select ClickOnce Publishing Tools.

 

Troubleshooting

When you have been able to successfully sign your Click Once applications in the past and suddenly you get the error message

An error occurred while signing: Failed to sign bin\Debug\app.publish\….exe. SignTool Error: No certificates were found that met all the given criteria.

it is possible that the certificate you used to sign your app has expired. To check the validity of the certificate

  1. In Visual Studio open the project’s properties page,
  2. On the Signing tab check the expiration date of the certificate.

Create a new certificate

To create a new certificate

  1. On the Signing tab of the project properties page click the Create Test Certificate button,
  2. Enter a password for the new certificate.
  3. Click the down arrow next to the key file name and select the new key file in the dropdown list.
  4. Save the project file and build your application.

 

 

 

No mapping specified for instances of the EntitySet and AssociationSet in the EntityContainer

When you move your Visual Studio solution to another workstation you may encounter the following error message:

System.Data.Entity.Core.MappingException was unhandled by user code
HResult=-2146232032
Message=Schema specified is not valid. Errors:
: error 2062: No mapping specified for instances of the EntitySet and AssociationSet in the EntityContainer …
Source=EntityFramework

 

In this case it may help if you re-import the database objects into the .edmx file.

To import all objects again:

  • Open the .edmx file,
  • Press the Ctrl-A keys to select all tables in the diagram,
  • Press the delete key on the keyboard to delete all tables,
  • Right click the white area and select Model Browser,
  • In the Model Browser expand the {model name}.Store, and the Stored Procedures / Functions element
  • Delete all functions and stored procedures
  • In the {model name} element (the element above the {model name}.Store) expand the Function Imports element
  • Delete all function imports
  • Save the .edmx file
  • Right click the white area of the .edmx file and select Update Model from Database
  • Select all necessary tables, stored procedures, and functions and import them.

If you re-import the tables, but do not delete the stored procedures, functions, and function imports you may get the following error message:

HResult=-2146233079
Message=The function import ‘Tobacco_WebEntities.spGetAgeGroupId’ cannot be executed because it is not mapped to a store function.
Source=EntityFramework

 

Save the key file password in Visual Studio

When you have already set up the project signing of the C# application for ClickOnce deployment, and move the solution to another workstation you may get the following error message when you build the solution:

Severity Code Description Project File Line Suppression State
Error Cannot import the following key file: .....pfx. The key file may be password protected. To correct this, try to import the certificate again or manually install the certificate to the Strong Name CSP with the following key container name: VS_KEY_....

We get this message, because the new workstation does not know the key file password. To solve this problem, we have to save the key file password on the new workstation.

To make your solution work again

  • Open the Visual Studio project properties window,
  • On the Signing tab click the pull down that shows the name of the key file.
  • Select Browse,
  • Select the key file,
  • Enter the password in the Password text box.

How to rename a TeamCity agent

It is important to keep the TeamCity agent names unique. If you launch a TeamCity agent with a name that is already used by another agent that is connected to the TeamCity server, the new agent will not show up in the Unauthorized agent list.

To change the name of an agent, change the “name” value in the conf/buildAgent.properties file.

To access the file you need to log into the box.

On a Linux agent

  • SSH into the agent,
  • Change the “name” value in the /opt/teamcity/conf/buildAgent.properties file.

On a Windows agent

  • Remote into the agent,
  • Change the “name” value in the C:\TeamCity\conf\buildAgent.properties file.

The location of TeamCity can be different depending on the configuration of the box.

DevOps Engineering part 7. – Launching production instances in the cloud

In the previous parts of this tutorial we have launched instances (servers) in the cloud, but those were created by Test Kitchen, running on our workstation. Those instances are as good as they can be, but the cookbook did not reside on the Chef server.

To launch pre-production and production instances in the cloud, first we need to upload the cookbook to the Chef server. In Beginner’s Guide to DevOps Engineering part 4. Connect to the Chef server we have already connected our workstation to the Chef server.

Upload the cookbook to the Chef server

To make your cookbook available for all the servers of the organization we will upload the cookbook to the Chef server.

  • In the folder of the cookbook open a Bash window
  • Execute the command to upload the cookbook to the Chef server 
    knife cookbook upload COOKBOOK_NAME --freeze

    Don’t forget to add the –freeze option to the end to protect the version of the cookbook from accidental changes.

Launch the server in the Cloud

There are many ways to launch servers in a scripted way, we are going to review a few options. One is to use RightScale, a Cloud Management Platform, that can manage servers in multiple cloud environments.

RightScale

Create a new Deployment

  • Using your web browser log into the RightScale Cloud Management user interface,
  • In the Manage menu select Deployment and click the New button,
  • Enter the name of the new deployment,
  • To add the new deployment to your bookmarks,
    • On the left side in the Bookmarks section click the Add link,
    • If necessary shorten the name of the bookmark and click OK,
    • To reorder the bookmarks
      • Click the Edit link and drag the bookmark to the desired position,
      • Click the Done link to save your changes.

Create the CAT file

CAT files are Ruby like scripts that describe the configuration of the instance. It contain the name of the deployment the server should be placed it, the cloud attributes to assign security groups, regions, availability zones, set the drive size, specify security keys. The easiest way to configure a server is to copy an existing RightScale Self Service CAT file and update it with the new values.

If this is the first CAT file in your organization, get help from RightScale support and get a sample file from them.

If your organization already has a library of CAT files

  • Make a copy of an existing CAT file
  • Update the values to customize it for your server
    • Application name
    • Chef runlist
    • Security group
    • Deployment name
    • Load balancer name
    • Instance type

 

Upload the CAT file to RightScale

  • Open the RightScale Self Service user interface,
  • Select Designer on the left,
  • Click the Upload CAT button and select the CAT file,
  • Click the Upload button in the lower right corner,
  • In the green popup in the upper right corner click the Publish to catalog link,
  • In the lower right corner click the Yes, Publish button.

Launch the server in RightScale Self Service

  • In the RightScale Self Service interface select Catalog on the left side,
  • Select the catalog item you want to launch,
  • Enter a name for the instance to be able to identify it later and select the instance in the dropdown list,
  • Click the Launch CloudApp button.

To monitor the server launch

  • In your web browser open the RightScale user interface,
  • In the design menu select Deployments,
  • Select the deployment of the instance,
  • Click the instance to see in formation on it.

Back:

to the Tutorials page