How to duplicate a Windows server that is attached to the Windows Domain

When a Windows server is attached to a Domain only one instance of it can run at a time. To be able to duplicate the server and start a second instance of it

Create an image of the server

  1. Remote into the server
  2. Make sure there is a local administrator account that you can log in with
  3. Remove it from the domain
  4. Reboot the server
  5. In the AWS console create an image of it
  6. Add the server back to the domain with the same username and password that was used to originally add it to the domain

Launch a new instance

  1. In the AWS console launch a new instance with the saved image
  2. Log into the new instance with the local administrator account
  3. Change the server name (only one server can be added to the domain with the same name)
  4. Add the server to the domain
  5. Reboot the server

Bootstrap the new instance in the Chef server

If you configured the original server with Chef, attach the new instance to the Chef server with a new Node Name

  1. Open a terminal window in the cookbooks directory (below the .kitchen directory, so the Chef kitchen command can find the Chef server configuration)
  2. Execute the bootstrap command. See Bootstrap Chef nodes to connect them to the Chef server for the details on the bootstrapping process. Don’t forget to use a double backslash (\\) in front of the username.
knife bootstrap windows winrm MY_NODE_IP -x MY_USERNAME -P MY_PASSWORD --node-name THE_NODE_NAME --environment THE_ENVIRONMENT --run-list 'recipe[MY_COOKBOOK1::default],recipe[MY_COOKBOOK2::default]' --json-attributes '{"MY_ATTRIB1":"MY_VALUE1","MY_ATTRIB2":"MY_VALUE2"}' -V

Unable to insert the virtual optical disk in an Ubuntu virtual machine

When you try to install a new version of the VirtualBox Guest Addition on an Ubuntu virtual machine you may get the error message

Unable to insert the virtual optical disk C:\Program Files\Oracle\VirtualBox\VBoxGuestAdditions.iso into the machine Ubuntu 64.

Could not mount the media/drive ‘C:\Program Files\Oracle\VirtualBox\VBoxGuestAdditions.iso’ (VERR_PDM_MEDIA_LOCKED).

Result Code: E_FAIL (0x80004005)
Component: ConsoleWrap
Interface: IConsole {872da645-4a9b-1727-bee2-5585105b9eed}
Callee: IMachine {85cd948e-a71f-4289-281e-0ca7ad48cd89}

To mount the new CD, first eject the currently mounted virtual CD

  1. Click the CD icon on the left side of the screen
  2. Click the eject icon next of the currently mounted Guest Addition CD

Raw-mode is unavailable courtesy of Hyper-V error in VirtualBox

For VirtualBox to be able to start virtual machines, Hyper-V has to be turned off in Windows 10.

If you get the error message when you try to launch a virtual machine in VirtualBox

Raw-mode is unavailable courtesy of Hyper-V

Check the Hyper-V setting

  1. Open a command prompt as an administrator,
  2. Execute the command: 
    bcdedit

    The default value of the hypervisorlaunchtype is “Auto”

    For VirtualBox to be able to launch virtual machines, we have to turn off hypervisorlaunchtype

Disable Hyper-V

To disable Hyper-V

  1. Open a command prompt as an administrator
  2. Execute the command 
    bcdedit /set hypervisorlaunchtype off
  3. Restart the computer. Do not shut down and start the computer, that does not work.

Enable Hyper-V

Docker for Windows needs the hardware virtualization that Hyper-V provides. To enbale Hyper-V again

  1. Open a command prompt as an administrator
  2. Execute the command 
    bcdedit /set hypervisorlaunchtype auto
  3. Restart the computer for the change to take effect.

Windows application installation error codes

To enable logging of .msi packages open a command prompt as an administrator and execute

MY_APPLICATION.msi /l*vx install.log

Error Code

Explanation
2 ?
1603 Error message from the operating system

  • dll can not register
  • msi installation failed
    • required version of the .NET framework missing
1605 Nothing to uninstall ?
1618 ?
1619 The source directory does not exist?
1622 File not found or access denied
1638

Stop multiple untagged AWS EC2 instances with a Bash script

 List all EC2 instances without a specific tag

One day we have found 499 instances running in our account without any tags. Most likely someone accidentally started a process to launch those, so we needed a way to find them and stop them. Later we will terminate them with the same script below when we can make sure those are not needed.

For simplicity, place the appropriate aws_access_key_id and aws_secret_access_key into the [default] section of the “~/.aws/credentials” file or use the –profile option in every command below.

List all instances

To list all EC2 instances, execute

aws ec2 describe-instances

List all instances missing a specific tag

I have found the command to list those instances that are missing the “Name” tag at https://www.onica.com/blog/using-aws-cli-to-find-untagged-instances/

I have directed the output to a text file with the additional last line.

To get all info on the instances with no “Name” tag into a JSON file

aws ec2 describe-instances \
--query 'Reservations[].Instances[?!not_null(Tags[?Key == `Name`].Value)]' \
> instances-no-name-tag.json

To output multiple properties into a tab-separated file for reporting in Excel.

aws ec2 describe-instances \
--output text \
--filters Name=instance-state-name,Values=running \
--query 'Reservations[].Instances[?!not_null(Tags[?Key == `Name`].Value)] | [].[InstanceId,ImageId,InstanceType,Platform,LaunchTime,SubnetId,KeyName]' \
> instance-info-no-name-tag.csv

Get the list of instance IDs into a text file for batch processing

aws ec2 describe-instances \
--output text \
--filters Name=instance-state-name,Values=running \
--query 'Reservations[].Instances[?!not_null(Tags[?Key == `Name`].Value)] | [].[InstanceId]' \
> instance-ids-no-name-tag.txt

Stop an instance with the instance Id

aws ec2 stop-instances --instance-ids MY_INSTANCE_ID

Stop multiple instances

To stop all instances listed in the “instance-ids-no-name-tag.txt” file created above, create and execute this Bash script:

#!/bin/bash

# The file with the instance IDs
filname=instance-ids-no-name-tag.txt

# Iterate through the lines
while read p; do
  echo "Stopping $p"
  aws ec2 stop-instances --instance-ids $p
done <$filname

 

“incompatible-network” error when launching an AWS RDS instance

When the AWS subnet has no enough IP addresses Terraform displays the following error message:

* aws_db_instance.default: unexpected state ‘incompatible-network’, wanted target ‘available, storage-optimization’. last error: %!s(<nil>)

Make sure the subnet has enough available IP addresses.

Docker commands to run popular images

Maven

Navigate to the Maven project directory on your workstation and launch the Maven container.

To run a Maven project by using the Maven Docker image directly, passing a Maven command to docker run.

docker run -it --rm --name my-maven-project -v "$(pwd)":/usr/src/mymaven -w /usr/src/mymaven maven:3.3-jdk-8 mvn clean install

To launch the Maven Docker container and open a Bash terminal in the container for an interactive session.

docker run -it --name my-maven-project -v "$(pwd)":/usr/src/mymaven -w /usr/src/mymaven maven:3.3-jdk-8 bash

 

Set up a user to connect to a Linux server with a private-public key pair

To secure a Linux server, disable password authentication on it. This way only those users can connect to it, who have access to an authorized private key.

To enable users to connect to a Linux server with a private-public key pair

Generate an RSA key pair

  1. In a Bash terminal on your workstation execute 
    ssh-keygen
  2. Follow the prompts to specify the name of the key file pair. In most of the cases, you don’t need to protect the key with a password.
    1. If you don’t specify the file name, the key will be saved as ~/.ssh/id_rsa
    2. If you specify a file name, the key files will be saved in the current directory
  3. The public key file will get the “.pub” extension, the private file has no extension

Upload the public key to the Linux server

  1. Log into the server with the “ssh” command using a username and password 
    ssh MY_USER_NAME@SERVER_IP_ADDRESS
  2. Add the public part of the key to the user configuration
    1. Switch to sudo mode, this command will ask for the password again 
      sudo -i
    2. Navigate to the user home directory 
      cd /home/USER_NAME/
    3. Add the public key to the user’s authorized_keys file. Open the file with a text editor and copy the public key into a new line. 
      vi authorized_keys
    4. To test the configuration, on your workstation navigate to the directory where the new key is located, and log into the server with 
      ssh -i MY_KEY_NAME MY_USER_NAME@SERVER_IP_ADDRESS

Turn off password authentication

  1. Make sure you can log in with the new key !!!
  2. Execute the command 
    sed -i 's/#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config

Install a ZIP web application package with Octopus Deploy

Install the Octopus Tentacle on the application server.

  1. Automated Tentacle installation instructions are at https://octopus.com/docs/infrastructure/windows-targets/automating-tentacle-installation
    (Note for our DevOps team: execute ~/Git/devops-scripts/OctopusDeploy/Tentacle_Installation/install_tentacle.ps1.)
  2. Make sure the box registered itself in the Octopus server. On the Deployment Targets tab of the Infrastructure menu enter the server name into the search box. The script already created the Environment and the Role and attached them to the target.

To manually create the application-specific environments

  1. On the Environments tab of the Infrastructure menu push the ADD ENVIRONMENT button

Create an application-specific lifecycle

  1. On the Lifecycles tab of the Library menu click the ADD LIFECYCLE button
  2. Click the ADD PHASE button to add a new phase for the environments
  3. Click the ADD ENVIRONMENT link to add the environments to the phase
  4. Create a phase for every environment. For automated deployment, select the Deploy automatically… radio button, and select the application-specific environments, and click the Ok button
  5. The list of phases is at the bottom of the page. Click the Save button to save the lifecycle.

Create a project for the application deployment

  1. Create a new project group
  2. Create a new project
  3. Set the project group and the lifecycle
  4. Click the DEFINE YOUR DEPLOYMENT PROCESS button
  5. Click the ADD STEP button
  6. Select the Deploy to IIS template
  7. Enter the name of the step and select the Role and Package ID

    …

Trigger an Octopus Deploy application package installation from TeamCity

Push the package to Octopus Deploy from TeamCity

Create project level parameters

In TeamCity create the parameters to centralize the configuration of reused values

  1. On the project level create a system parameter for the Octopus package name including the build number interpolation

Push the artifact to Octopus Deploy

  1. Create a build configuration, and on the General Settings page specify a build number format that Octopus can understand
  2. Add the build step as a dependency
  3. Create a trigger on the same dependency to automatically execute the Octopus push on a successful build
  4. Create a Command Line build step to get the artifact from the built-in TeamCity repository. Use curl to download the file to the working directory, use the octopus_package_name parameter you created above. 
    curl -o %system.MY_APP_api_octopus_package_name% http://%system.tc_server%/guestAuth/repository/downloadAll/MY_PROJECT_ID/.lastSuccessful/artifacts.zip

  5. Create an OctopusDeploy: Create and Push Packages build step to upload the package to the Octopus server. Use the system parameters to specify the Octopus server URL, and the package name, enter the API key that has enough rights to push a package to the Octopus server. The user needs the BuiltInFeedPush permission to push a package. Octopus Deploy provides a built-in role called Package Publisher that has been granted the BuiltInFeedPush permission.
  6. To check the push to Octopus Deploy, run the TeamCity configuration to upload the package to the OctopusDeploy built-in repository. in Octopus Deploy click the name of the package to see the versions.
  7. The next page shows the package versions

Create an Octopus Deploy release

  1. In the TeamCity configuration, add an OctopusDeploy: Create release type build step to create a release in Octopus Deploy. Use the %OctopusURL% parameter from the root project level, enter the Octopus API key, and use the %build_number% macro that you have specified in the Release number field on the General Settings tab. If you have not created a custom channel in the Octopus project, the default is “Default”.
    List the environments where the release can be deployed, and check the Show deployment process checkbox.
  2. To check the release creation in Octopus Deploy, navigate to the overview page of the Octopus Deploy project.

Deploy the Octopus Deploy release

To install the package add an OctopusDeploy: Deploy Release build step

  1. Create a project parameter for the Octopus Deploy project name
  2. Create a new build step using the project level parameters.