HTTP Request Returned 409 Conflict: Client already exists

The Chef server maintains the list of registered nodes and clients in its database.  When you launch a new instance with Chef you may encounter the following error message:

*** Input CHEF_CLIENT_NODE_NAME is undefined, using: IP-0AFE6965
...
*** Starting chef-client
*** Finished chef-client
Printing Log
...
[...] INFO: Client key C:\chef\client.pem is not present - registering
[...] INFO: HTTP Request Returned 409 Conflict: Client already exists
[...] INFO: HTTP Request Returned 403 Forbidden: error
[...] ERROR: Running exception handlers
[...] ERROR: Exception handlers complete
[...] FATAL: Stacktrace dumped to C:/chef/cache/chef-stacktrace.out
[...] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[...] FATAL: Net::HTTPServerException: 403 "Forbidden"

Cause:

In this case the CHEF_CLIENT_NODE_NAME already exists in the client database.You can search for it from a Bash window.

Solution:

Add a line to the Chef configuration to remove the existing node with the same node name

If you use Terraform to launch server instances and configure Chef, add this line to the Chef provisioner:

recreate_client = true

To delete nodes from the Chef server:

Use the knife command to search for the existing node from a Bash window.

To find the node in the Chef server database

knife search node '*:IP-0AFE6965'

To find the client in the Chef server database

knife search client '*:IP-0AFE6965'

If any of them found, those are leftovers from a previous launch. AWS reuse the IDs, so you have to remove the nodes from the Chef server database when you terminate the instances. To delete the unnecessary entries use the following commands:

knife node delete 'IP-0AFE6965'
knife client delete 'IP-0AFE6965'

To remove the terminated instances from the Chef server database you can also use the Chef web interface.

  • On the Nodes tab enter the IP address of the instance into the search box and hit Enter
  • In the Actions column click the down arrow next to the instance and select Delete

How to edit the GitHub README.md file

The GitHub repositories usually contain a README.md file to describe how to use the project. The GitHub web site has a simple editor, but it has a few limitations

  • If you accidentally refresh the page, you lose your changes,
  • The preview pane is wider than the page that will display your file, so the formatting can be different,
  • You have to switch between the editor and the the preview every time you want to see the result.

One of the simplest ways to edit the README.md is to use the Atom editor.

  • Open the README.md file with Atom,
  • In the Packages menu select Markdown Preview, or press Ctrl-Shitft-M to open the preview pane.

As you type in the editor window the preview pane will show the live review of the file.

For the Markdown syntax visit https://guides.github.com/pdfs/markdown-cheatsheet-online.pdf

An error occurred while signing

When you deploy and application in Microsoft Visual Studio with ClickOnce, you need to sign the installer with a key.

To sign the project Visual Studio needs to use the signtool. If the ClickOnce feature is not enabled in Visual studio, it displays the following error message:

An error occurred while signing: SignTool.exe was not found at path C:\Program Files (x86)\Microsoft SDKs\Windows\v7.0A\bin\signtool.exe

To install the ClickOnce feature and the signtool

On Windows 7

  • In Windows Control panel select Programs and Features,
  • Right click Visual Studio 2015 and select Modify,
  • In the feature list select ClickOnce tools.

On Windows 10

  • In Windows Control Panel select System,
  • On the left side select Apps & features,
  • In the app list click Microsoft Visual Studio 2015,
  • Click the Modify button,
  • In the Visual Studio setup click the Modify button again,
  • In the feature list select ClickOnce Publishing Tools.

 

Troubleshooting

When you have been able to successfully sign your Click Once applications in the past and suddenly you get the error message

An error occurred while signing: Failed to sign bin\Debug\app.publish\….exe. SignTool Error: No certificates were found that met all the given criteria.

it is possible that the certificate you used to sign your app has expired. To check the validity of the certificate

  1. In Visual Studio open the project’s properties page,
  2. On the Signing tab check the expiration date of the certificate.

Create a new certificate

To create a new certificate

  1. On the Signing tab of the project properties page click the Create Test Certificate button,
  2. Enter a password for the new certificate.
  3. Click the down arrow next to the key file name and select the new key file in the dropdown list.
  4. Save the project file and build your application.

 

 

 

No mapping specified for instances of the EntitySet and AssociationSet in the EntityContainer

When you move your Visual Studio solution to another workstation you may encounter the following error message:

System.Data.Entity.Core.MappingException was unhandled by user code
HResult=-2146232032
Message=Schema specified is not valid. Errors:
: error 2062: No mapping specified for instances of the EntitySet and AssociationSet in the EntityContainer …
Source=EntityFramework

 

In this case it may help if you re-import the database objects into the .edmx file.

To import all objects again:

  • Open the .edmx file,
  • Press the Ctrl-A keys to select all tables in the diagram,
  • Press the delete key on the keyboard to delete all tables,
  • Right click the white area and select Model Browser,
  • In the Model Browser expand the {model name}.Store, and the Stored Procedures / Functions element
  • Delete all functions and stored procedures
  • In the {model name} element (the element above the {model name}.Store) expand the Function Imports element
  • Delete all function imports
  • Save the .edmx file
  • Right click the white area of the .edmx file and select Update Model from Database
  • Select all necessary tables, stored procedures, and functions and import them.

If you re-import the tables, but do not delete the stored procedures, functions, and function imports you may get the following error message:

HResult=-2146233079
Message=The function import ‘Tobacco_WebEntities.spGetAgeGroupId’ cannot be executed because it is not mapped to a store function.
Source=EntityFramework

 

Save the key file password in Visual Studio

When you have already set up the project signing of the C# application for ClickOnce deployment, and move the solution to another workstation you may get the following error message when you build the solution:

Severity Code Description Project File Line Suppression State
Error Cannot import the following key file: .....pfx. The key file may be password protected. To correct this, try to import the certificate again or manually install the certificate to the Strong Name CSP with the following key container name: VS_KEY_....

We get this message, because the new workstation does not know the key file password. To solve this problem, we have to save the key file password on the new workstation.

To make your solution work again

  • Open the Visual Studio project properties window,
  • On the Signing tab click the pull down that shows the name of the key file.
  • Select Browse,
  • Select the key file,
  • Enter the password in the Password text box.

How to rename a TeamCity agent

It is important to keep the TeamCity agent names unique. If you launch a TeamCity agent with a name that is already used by another agent that is connected to the TeamCity server, the new agent will not show up in the Unauthorized agent list.

To change the name of an agent, change the “name” value in the conf/buildAgent.properties file.

To access the file you need to log into the box.

On a Linux agent

  • SSH into the agent,
  • Change the “name” value in the /opt/teamcity/conf/buildAgent.properties file.

On a Windows agent

  • Remote into the agent,
  • Change the “name” value in the C:\TeamCity\conf\buildAgent.properties file.

The location of TeamCity can be different depending on the configuration of the box.

DevOps Engineering part 7. – Launching production instances in the cloud

In the previous parts of this tutorial we have launched instances (servers) in the cloud, but those were created by Test Kitchen, running on our workstation. Those instances are as good as they can be, but the cookbook did not reside on the Chef server.

To launch pre-production and production instances in the cloud, first we need to upload the cookbook to the Chef server. In Beginner’s Guide to DevOps Engineering part 4. Connect to the Chef server we have already connected our workstation to the Chef server.

Upload the cookbook to the Chef server

To make your cookbook available for all the servers of the organization we will upload the cookbook to the Chef server.

  • In the folder of the cookbook open a Bash window
  • Execute the command to upload the cookbook to the Chef server
    knife cookbook upload COOKBOOK_NAME --freeze

    Don’t forget to add the –freeze option to the end to protect the version of the cookbook from accidental changes.

Launch the server in the Cloud

There are many ways to launch servers in a scripted way, we are going to review a few options. One is to use RightScale, a Cloud Management Platform, that can manage servers in multiple cloud environments.

RightScale

Create a new Deployment

  • Using your web browser log into the RightScale Cloud Management user interface,
  • In the Manage menu select Deployment and click the New button,
  • Enter the name of the new deployment,
  • To add the new deployment to your bookmarks,
    • On the left side in the Bookmarks section click the Add link,
    • If necessary shorten the name of the bookmark and click OK,
    • To reorder the bookmarks
      • Click the Edit link and drag the bookmark to the desired position,
      • Click the Done link to save your changes.

Create the CAT file

CAT files are Ruby like scripts that describe the configuration of the instance. It contain the name of the deployment the server should be placed it, the cloud attributes to assign security groups, regions, availability zones, set the drive size, specify security keys. The easiest way to configure a server is to copy an existing RightScale Self Service CAT file and update it with the new values.

If this is the first CAT file in your organization, get help from RightScale support and get a sample file from them.

If your organization already has a library of CAT files

  • Make a copy of an existing CAT file
  • Update the values to customize it for your server
    • Application name
    • Chef runlist
    • Security group
    • Deployment name
    • Load balancer name
    • Instance type

 

Upload the CAT file to RightScale

  • Open the RightScale Self Service user interface,
  • Select Designer on the left,
  • Click the Upload CAT button and select the CAT file,
  • Click the Upload button in the lower right corner,
  • In the green popup in the upper right corner click the Publish to catalog link,
  • In the lower right corner click the Yes, Publish button.

Launch the server in RightScale Self Service

  • In the RightScale Self Service interface select Catalog on the left side,
  • Select the catalog item you want to launch,
  • Enter a name for the instance to be able to identify it later and select the instance in the dropdown list,
  • Click the Launch CloudApp button.

To monitor the server launch

  • In your web browser open the RightScale user interface,
  • In the design menu select Deployments,
  • Select the deployment of the instance,
  • Click the instance to see in formation on it.

Back:

to the Tutorials page

 

DevOps Engineering part 6. – Infrastructure as code

Terraform

If you use Terraform in the corporate environment your company most likely has multiple AWS accounts. One for pre-production, one for production.

To be able to work in multiple AWS accounts, add those keys to the credentials file at C:\Users\YOUR_USERNAME\.aws

[aws01]
aws_access_key_id = MY_ACCESS_KEY_FOR_AWS01
aws_secret_access_key = MY_SECRET_KEY_FOR_AWS01

[aws02]
aws_access_key_id = MY_ACCESS_KEY_FOR_AWS02
aws_secret_access_key = MY_SECRET_KEY_FOR_AWS02

[default]
aws_access_key_id = MY_ACCESS_KEY_FOR_AWS01
aws_secret_access_key = MY_SECRET_KEY_FOR_AWS01

We will use Terraform to create security groups and load balancers in AWS.

  • Create a folder on your workstation for the Git repositories C:\Git
  • In the Git folder create a folder for the Terraform Configurations Terraform Configs
  • In the Terraform Configs folder create a sub-folder for the

When you are reusing existing Terraform configurations

  • Make a copy of the existing Terraform Config folder
  • Rename the folder
  • Rename the .tf files to match the name of the folder name
  • Delete the .tfstate and .tfstate.backup files
  • Update the .tf files with the new values
  • Open a Bash window in the new folder
  • Get the Terraform modules with
    terraform get -update
  • Test the script with
    terraform plan
  • Execute the script with
    terraform apply

More info on Terraform at https://www.terraform.io/docs/providers/index.html


 Next:

Launching production instances in the cloud in Beginner’s Guide to DevOps Engineering part 7.

Back:

to the Tutorials page

Enable user to import projects in TeamCity

It is not enough to belong to the “System Administrator” group to be able to import projects from backup  in TeamCity. The user has to be in the “System administrator” role for the “Root project”.

To enable the user to import projects into TeamCity

  • Click the Administration link in the upper right corner
  • In the User Management section click Users
  • Click the user name in the list
  • On the Roles tab click the + Assign role button
  • Set the Role to System administrator
  • Set the Scope to Root project
  • Select the Replace existing roles with newly selected check box and click the Assign button

DevOps Engineering part 5. – Create an enterprise cookbook

In this exercise we will create a Chef cookbook for a corporation. It will be robust and include all the necessary elements to be used in a large enterprise. If your company already has established standards, as you progress with this tutorial, copy the appropriate file from an existing cookbook. If this is the first cookbook in your organization, or you don’t want to follow the current standards, copy the sample files from this page.

Create the cookbook

  • Navigate to the C:\Chef\…\cookbooks folder and open a Bash window
  • Create an empty cookbook
    chef generate cookbook COOKBOOK_NAME

    Chef creates a new sub-folder with the name of the cookbook.

  • Rename the new folder to cookbook-COOKBOOK_NAME to distinguish it in version control from other repository types.

Update the .gitignore file

  • Chef DK has generated a .gitignore file that contains most of the important entries, but we need to add a few important lines:
    # Certificates
    *.pem
    
    # Unencrypted Data Bags
    data_bags_unencrypted/
    
    # Macintosh folder custom attribute file
    .DS_Store
    
    # If working in a team, personalized Test Kitchen config file 
    .kitchen.yml

Update the metadata.rb file

Add your name, email address, and the version of the cookbook to the top of the metadata.rb file

name 'THE_NAME_OF_THE_COOKBOOK'
maintainer 'YOUR_NAME'
maintainer_email 'YOUR_EMAIL_ADDRESS'
license 'All rights reserved'
description 'Installs/Configures THE_NAME_OF_THE_COOKBOOK'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '1.0.0'

The expression in the long_description line refers to the README.md file that GitHub can create when you set up a new repository. Place the cookbook related information into that file, so others who want to use your cookbook can easily find it.

Update the Berksfile

Add the list of cookbooks this cookbook depends on to the Berksfile file. To download all necessary cookbooks execute

berks install

Update the .kitchen.yml file to be able to test your cookbook.

If you want to test your cookbook in the Cloud, copy the following sections from an existing .kitchen.yml file of your organization. To learn Chef and launch instances with Vagrant on your workstation, the automatically created .kitchen.yml file is perfect.

  • driver
  • provisioner
  • platforms
  • a suite as an example

If your organization use tags to track instances in the Cloud,  update the tags section to reflect the values of the cookbook.

To add multiple recipes to the Chef run list use this syntax:

run_list:
  - recipe[COOKBOOK_NAME1::RECIPE_NAME]
  - recipe[COOKBOOK_NAME2::RECIPE_NAME]

Encrypted Data Bags

If you want to use encrypted data bags, ask your Chef administrator to send you the key file to encrypt and decrypt data bags. Create the data_bags_unencrypted folder for the data bag secret key on the same level where the cookbook and environment folders are.

This is a chicken and egg paradox. We don’t want to commit secrets into version control, so we need to encrypt them. But how can we place the secret encryption key on the server to decrypt the secrets? We will use Packer to create our own server images that will contain the secret key, so when Chef starts to run on the server, the key is going to be there.

Default recipe

We will place code in the default.rb file that is common to all recipes in the cookbook. All custom recipes will call the default recipe as the first step.

Update the header comments of the default.rb recipe with your name and company information

#
# Cookbook Name:: COOKBOOK_NAME
# Recipe:: default
#
# Author:: YOUR_NAME (<YOUR_EMAIL_ADDRESS>)
# Copyright (c) 2015-2016 COMPANY_NAME, All Rights Reserved.

Attribute file

Create the default attribute file with

chef generate attribute default

Add the necessary attributes to the attribute file to store the AWS tag and other cookbook specific values.

Custom recipe

Use the Chef generate command to create the new recipes. It creates all test files in the corect location.

  • Create a new recipe and the test files with
    chef generate recipe MY_RECIPE_NAME
  • Add the following line under the header to call the default recipe, even if it is currently empty
    include_recipe 'COOKBOOK_NAME::default'

Add the cookbook to GitHub

Create the local repository

  • In the Bash window execute the following commands
git init
git add .
git commit -m "Initial commit"

Add the repository to GitHub

In your web browser log into your GitHub account and create a new repository

  • Click the New Repository button
  • Name the repository the same as the name of the folder of the cookbook (cookbook-…)
  • Execute the lines in the section …or push an existing repository….  If you work on a Windows workstation make sure HTTPS is selected
    git remote add origin https://github.com/....git
    git push -u origin master
  • In the Collaborators & teams section of Settings select the group who will have access to the new repository

Test the cookbook

In the Bash window launch the instance with Test Kitchen

List the available instances

kitchen list

Launch the instance

kitchen converge STRING_UNIQUE_TO_THE_INSTANCE

If there are multiple suites or platforms in the .kitchen.yml file you need to type the unique part of the name of the instance to identify it.

Start a Remote Desktop connection to a Windows instance

kitchen login STRING_UNIQUE_TO_THE_INSTANCE

If you work on a Macintosh workstation and testing a Windows server, the best way to remote into the server is

  1. Install Microsoft Remote Desktop for free from the Apple App Store (See the Remote Desktop Client section in Install the DevOps development tools on Macintosh,
  2. Execute the kitchen login STRING_UNIQUE_TO_THE_INSTANCE command in the terminal window,
  3. The Microsoft Remote Desktop window will pop up with the User account name and the IP address. If you click “Connect”, the remote connection opens with the default settings, that are may not optimal for your display. The “login” command already created a new entry in the Microsoft Remote Desktop app with the IP address of the instance. To use custom settings, click Cancel,
  4. Open the Microsoft Remote Desktop app, right click the last entry that contains the IP address, select Edit, and copy the IP address to the clipboard,
  5. Create a new connection with custom display settings and paste the IP address there. You can keep this generic connection to access the Test Kitchen instances, just update the IP address.
  6. Delete the automatically created connection

SSH into a Linux instance

ssh USER_NAME@IP_ADDRESS -i PATH_TO_THE SSH_KEY_FILE

Where

  • USER_NAME is the value of username: in the .kitchen.yml file.
  • IP_ADDRESS is the IP of the instance. Get it from the “Waiting for SSH service on…” line of the Test Kitchen bash or command window, or from the .yml file in the .kitchen/logs folder of the cookbook.
  • PATH_TO_THE_SSH_KEY_FILE is the value of ssh_key: in the .kitchen.yml file.

Terminate the instance

kitchen destroy STRING_UNIQUE_TO_THE_INSTANCE

Upload the cookbook to the Chef server

See Connect to the Chef server in Beginner’s Guide to DevOps Engineering part 4.


Next:

Infrastructure as code in Beginner’s Guide to DevOps Engineering part 6.

Back:

to the Tutorials page