Pinter Computing – Page 58 – Knowledge Base for IT Professionals, Teachers and Astronauts

Error waiting for instance (i-…) to become ready: unexpected state ‘terminated’, wanted target ‘running’

When you launch a server instance with Terraform, sometimes the error message does not contain the underlying cause. When the cloud provider cannot complete the request, many times Terraform displays a generic error message:

Error waiting for instance (i-...) to become ready: unexpected state 'terminated', wanted target 'running'

To find the root cause of the error in AWS

Log into the AWS console and navigate to the EC2 section,
Search for the instance by the instance Id,
You can find the error message at the bottom of the Description tab

In our specific case, it was Client.VolumeLimitExceeded: Volume limit exceeded

We had to increase the volume limit to be able to launch more large EC2 instances.

Upgrade a server on an AWS EC2 instance with minimum risk and downtime

When you need to upgrade an application on an AWS EC2 instance with minimum downtime, there are many options.

Upgrade an EC2 instance

Stop the application, so users don’t make more changes,
Create a backup of the database (snapshot of the RDS instance),
Create a backup of the server (backup image of the EC2 instance),
Upgrade the application on the server,
Start the application,
Test the new version of the application.

In case the upgrade fails, or the post-upgrade test fails

Stop the application,
If you use RightScale to launch servers:
1. stop the RightLink service on the failed server to prevent RightScale auto terminating the restored server
  (When RightScale finds a new identical server instance, it automatically shuts it down to avoid multiple instances with the same identifier.)
2. Terminate the failed server
Stop the failed server,
Restore the database from the pre-upgrade backup with a new name,
Restore the server instance from the pre-upgrade backup,
Start the restored server,
Change the database address in the application’s config file to point to the restored database,
Start the application,
Test the restored version of the application.

To make the backup image of the AWS EC2 instance

Open the AWS console and navigate to EC2, Running instances
Find the server instance you want to backup
Right-click the row of the instance and select Image, Create Image
Name the image and click the Create Image button
Save the image Id from the popup

Test your RightScale script without creating unnecessary revisions

There are certain functions in RightScale scripts (RightScripts) that you cannot test without launching a new instance. To test your script:

Edit the Head revision of the RightScale script and save it,
Attach the Head revision of the RightScale script to the Head revision of the RightScale Server Template,
1. Select the Head revision of the server template, on the Scripts tab click the Modify button
2. Click the rev dropdown next to the script’s name and select head
In your Self Service CAT file reference revision 0 (zero) of the Server Template (Head revision),
( “server_template_revision” => “0”, )
Upload the Self Service CAT file to Self Service, but do not publish it,
Launch a server instance from Self Service using the RightScale Self-Service Designer page, not the Catalog.

If the script has some errors, modify the Head revision of the RightScale script and launch a new server instance using the RightScale Self-Service Designer page.

When the script works perfectly:

Commit the RightScale script, creating a new RightScale script revision,
Attach the committed revision of the RightScale script to the Server Template’s Head revision,
Commit the Server Template, creating a new Server Template revision,
Publish the new revision of the Server Template,
Reference the new revision of the Server Template in your Self Service CAT file.

Custom Git Status and Git Pull commands to work with all of your repositories

If you work with many Git repositories it is easy to miss a repository when you commit your changes, and very time consuming to pull from all repositories.

It is easy to automate both processes.

In this example, we will place all files in the ~/Git/devops-scripts folder. If you want to place your files at another location change it in the files below.

\/ \/ \/ \/ \/ \/ \/ \/ \/ \/

In Windows

To find your home directory on a Windows workstation, open a Bash window (Git Bash) and execute

echo ~

/\ /\ /\ /\ /\ /\ /\ /\ /\ /\

Create the Bash script files

Create a file with the list of Git repositories and save it as ~/Git/devops-scripts/git-repositories.txt
You can separate groups of repositories with empty lines.

# Local_Path+Remote_Path+Rename_the_folder_to(optional)
~/Git/MY_REPO1+https://github.com/MY_REPO1.git+RENAME_TO1
~/Git/MY_REPO2+https://github.com/MY_REPO2.git

~/Git/MY_REPO3+https://github.com/MY_REPO3.git

Create a file to check the status of all Git repositories and save it as ~/Git/devops-scripts/git-status.sh

#!/bin/bash

# git-status.sh
# Shows the GIT STATUS of the Git repositories and shows the recommended actions
# The list is in git-repositories.txt

eval input="~/Git/devops-scripts/git-repositories.txt" # Need EVAL, otherwise the Bash loop cannot interpret ~ as the home directory if the path contains ~

RED='\033[0;31m'
NC='\033[0m' # No Color

while IFS= read -r folder
do

 # Get the length of the path
 # On Windows the empty line manifests itself as a 1 character long not empty string
 path_length=${#folder}

 if [[ ( -n "$folder" ) ]] && [[ ( 1 < $path_length ) ]] ; then
 # The path is not empty and the length is greater than 1

 # Ignore the lines starting with #
 if [[ ! $folder == \#* ]] ; then
 # The line is not a comment, process it

 # Split the local and remote path
 eval local_path=$(echo "$folder" | cut -f1 -d+) # Need EVAL, otherwise the IF statement below does not work if the path contains ~
 remote_path=$(echo "$folder" | cut -f2 -d+)

 if [ ! -d "$local_path" ]
 then

 # The local directory doesn't exist.
 echo -e "${RED}$local_path does not exist, PULL to get $remote_path${NC}"

 else

 eval cd $local_path # Need EVAL, otherwise CD does not work if the path contains ~
 pwd | tr -d '\n'

 status=$(git status)

 # --------------------------------------------------------------------------
 # To handle

 # On branch master
 # Your branch is up-to-date with 'origin/master'.
 # Untracked files:
 # (use "git add <file>..." to include in what will be committed)

 if [[ $status == *"Untracked files"* ]]; then
 echo -e " ---- ${RED}New files added, please ADD${NC}" | tr -d '\n' # -e tells echo to enable backslash escapes
 fi

 # --------------------------------------------------------------------------
 # To handle

 # On branch master
 # Your branch is up-to-date with 'origin/master'.
 # Changes not staged for commit:
 # (use "git add <file>..." to update what will be committed)
 # (use "git checkout -- <file>..." to discard changes in working directory)

 if [[ $status == *"Changes not staged for commit"* ]]; then
 echo -e " ---- ${RED}Files changed, please ADD${NC}" | tr -d '\n' # -e tells echo to enable backslash escapes
 fi

 # --------------------------------------------------------------------------
 # To handle

 # On branch master
 # Your branch is up-to-date with 'origin/master'.
 # Changes to be committed:
 # (use "git reset HEAD <file>..." to unstage)

 if [[ $status == *"Changes to be committed"* ]]; then
 echo -e " ---- ${RED}Please COMMIT${NC}" | tr -d '\n' # -e tells echo to enable backslash escapes
 fi

 # --------------------------------------------------------------------------
 # To handle

 # On branch master
 # Your branch is ahead of 'origin/master' by 1 commit.
 # (use "git push" to publish your local commits)
 # nothing to commit, working tree clean

 if [[ $status == *"git push"* ]]; then
 echo -e " ---- ${RED}Please PUSH${NC}" | tr -d '\n' # -e tells echo to enable backslash escapes
 fi

 # --------------------------------------------------------------------------
 # To handle

 # On branch master
 # Your branch and 'origin/master' have diverged,
 # and have 1 and 1 different commits each, respectively.
 # (use "git pull" to merge the remote branch into yours)
 # nothing to commit, working tree clean

 if [[ $status == *"git pull"* ]]; then
 echo -e " ---- ${RED}Repos diverged, PULL to merge${NC}" | tr -d '\n' # -e tells echo to enable backslash escapes
 fi

 # --------------------------------------------------------------------------

 echo

 fi

 fi

 fi
done < "$input"

Create a file to pull from all Git repositories and save it as ~/Git/devops-scripts/git-pull.sh

#!/bin/bash

# git-pull.sh
# Pulls Git repositories and clones the missing ones
# The list is in git-repositories.txt

eval input="~/Git/devops-scripts/git-repositories.txt" # Need EVAL, otherwise the Bash loop cannot interpret ~ as the home directory if the path contains ~

RED='\033[0;31m'
NC='\033[0m' # No Color

while IFS= read -r folder
do

 # Get the length of the path
 # On Windows the empty line manifests itself as a 1 character long not empty string
 path_length=${#folder}

 if [[ ( -n "$folder" ) ]] && [[ ( 1 < $path_length ) ]] ; then
 # The path is not empty and the length is greater than 1

 # Ignore the lines starting with #
 if [[ ! $folder == \#* ]] ; then
 # The line is not a comment, process it

 # Split the local and remote path
 eval local_path=$(echo "$folder" | cut -f1 -d+) # Need EVAL, otherwise the IF statement below does not work if the path contains ~
 remote_path=$(echo "$folder" | cut -f2 -d+)
 rename_folder_to=$(echo "$folder" | cut -f3 -d+)

 # Check if the local path exists
 if [ ! -d "$local_path" ]; then
 # The local directory doesn't exist.
 echo
 echo -e "${RED}$local_path does not exist, cloning $remote_path${NC}"

 # Get the parent directory name
 parent_dir="${local_path%/*}"

 # Create the parent directory
 mkdir -p $parent_dir

 # Go to the parent_dir
 cd $parent_dir

 # Clone the Git repository
 git clone $remote_path

 if [[ ( -n "$rename_folder_to" ) ]] ; then

 # Rename the folder if necessary
 end_of_url="${remote_path##*/}"
 filename="${end_of_url%.*}"

 echo -e "${RED}Renaming $filename to $rename_folder_to${NC}"

 mv $filename $rename_folder_to

 fi


 else
 echo
 eval cd $local_path # Need EVAL, otherwise CD does not work if the path contains ~
 pwd
 git pull

 fi

 fi

 fi

done < "$input"

Enable the script execution

\/ \/ \/ \/ \/ \/ \/ \/ \/ \/

In Mac OS

Open a terminal window in the ~/Git/devops-scripts folder
Run these commands to enable the execution of the Bash files

chmod u+x git-status.sh
chmod u+x git-pull.sh

/\ /\ /\ /\ /\ /\ /\ /\ /\ /\

Create an alias

Aliases allow you to execute commands from any folder in the command line and hide the complexity of the entire command.

\/ \/ \/ \/ \/ \/ \/ \/ \/ \/

In Mac OS

If you use the iTerm terminal window

iTerm reads the configuration settings from the~/.zshrc file. If you use iTerm add the alias definitions to the end of the ~/.zshrc file.

Open the~/.zshrc file and add the lines below to the end of it

If you use the built-in OSX Terminal

The built-in Terminal windows reads the configuration settings from the~/.bash_profile file.

Open the ~/.bash_profile file and add the lines below to it

In Windows

If you use Git Bash

Navigate to your home directory ( cd ~ )
Create the Bash config file if does not exist
```
cp > ~/.bashrc
```
Bash will show the “cp: missing file operand” an error message, but it will create the file that you can edit with any text editor
Add the lines below to the file

/\ /\ /\ /\ /\ /\ /\ /\ /\ /\

alias status='~/Git/devops-scripts/git-status.sh'
alias pull='~/Git/devops-scripts/git-pull.sh'

Refresh the settings in the Bash shell environment

\/ \/ \/ \/ \/ \/ \/ \/ \/ \/

In MacOS

source ~/.bash_profile

In Windows

Close the Bash window and open it again

/\ /\ /\ /\ /\ /\ /\ /\ /\ /\

How to use it

Open a terminal window in any folder

To check if you need to add, commit or push files in a Git repository:

status

If you have added new files to the repository you get:

/Users/YOUR_USER_NAME/Git/devops-scripts —- New files added, please ADD

If you have changed files in a Git repository you get:

/Users/YOUR_USER_NAME/Git/devops-scripts —- Files changed, please ADD

If you have already executed the git add . command, but have not yet committed the changes:

/Users/YOUR_USER_NAME/Git/devops-scripts—- Please COMMIT

If you have already committed the changes, but have not pushed the to the remote repository:

/Users/YOUR_USER_NAME/Git/devops-scripts —- Please PUSH

To pull from every Git repository before you start to work and clone the missing ones

pull

Useful Bash commands

Most of these commands work in all Linux, MacOSX, and Unix shells, but some of them only work in selected shells.

Find files recursively by file name pattern

find . -name '*.MY_EXTENSION'

Find files, but exclude a directory from the search. (The -print is needed at the end of the line)

find . -path ./tmp/windows -prune -o -name '*.MY_EXTENSION' -print

Get the absolute path of files recursively

find `pwd` -name MY_FILENAME

Delete files recursively by file name pattern

Works in zsh but not in sh or bash.
New versions of bash cannot do this by default only if the shell option globstar is enabled:
shopt -s globstar
The globbing pattern is **/*.o but that is not limited to files.

find . -type f -name '*.MY_EXTENSION' -delete

Make a shell script file executable

chmod u+x FILE_NAME.sh

Directory Listing

Sort by date, most recent on top

ls -lt

Sort by date oldest on top

ls -ltr

Sort by date, display only last modification date and file name

ls -lt | awk '{print $7,$8,$9, $10}'

Display the CHMOD numbers in the output of the ls command

ls -l | awk '{k=0;for(i=0;i<=8;i++)k+=((substr($1,i+2,1)~/[rwx]/) \
 *2^(8-i));if(k)printf("%0o ",k);print}'

Generate a public SSH key from the private key

ssh-keygen -y -f PRIVATE_KEY_FILE.pem > PUBLIC_KEY_FILE.pub

Replace all spaces in every filename and directory name recursively with dash

find . -name “* *” -print0 | sort -rz | \
while read -d $’\0′ f; do mv -v “$f” “$(dirname “$f”)/$(basename “${f// /-}”)”; done

To check if a package is installed

# For RHEL 
rpm -qa *MY_SEARCH_TERM*
# For Ubuntu
dpkg —list | grep MY_PACKAGE_NAME

Security and user management

Add a user to a Linux user group

To add your user account to a user group

sudo usermod -aG THE_GROUP_NAME $USER

To add another user to a group

sudo usermod -aG THE_GROUP_NAME THE_USER_NAME

Windows 10 can’t connect to this network

If your Windows 10 computer can successfully connect to some wireless networks, but cannot connect to certain wireless networks, it is possible, that the stored credentials are out of date. To force Windows 10 to ask you for the updated username and password

In the Start Menu select Settings,
On the Windows Settings page select Network & Internet,
On the left side select Wi-Fi,
On the Wi-Fi page select Manage known networks,
Select the network you are not able to connect and click the Forget button,
In the lower right corner of the screen select the Wi-Fi network icon, select the network, and click the Connect button.

How to disable Microsoft OneDrive on Windows 8.1 and Windows 10

To disable OneDrive on Windows 8.1 and Windows 10 navigate to https://support.office.com/en-us/article/Turn-off-or-uninstall-OneDrive-f32a17ce-3336-40fe-9c38-6efb09f944b0 and follow the instructions for your computer’s operating system.

Find the AWS AMI that a RightScale server template is using

When you use RightScale to launch servers in the cloud, you want to use the same base image to test your Chef cookbooks in Test Kitchen.

Packer by Hashicorp is a utility to create custom server images based on cloud images. You supply the image ID and other configuration parameters to create a new custom image.

To find the AMI ID of the base image of the RightScale Server Template

Open the RightScale user interface in your browser,
Select the RightScale account where the server template was created,
In RightScale Cloud Management select Design / ServerTemplates,
Select the server template,
Select Images,
Select the MultiCloud image,
Select Clouds,
Select the image in the region you are working in,
The Resource UID is the AMI ID of the image.

Mailbox unavailable. The server response was: No such user here

When you send email from an application, most SMTP servers require authentication. Your application needs to supply a valid username and password to be able to connect to the server. This prevents strangers to use your SMTP server and send millions of spam emails.

When your application does not supply the correct credentials to connect to the SMTP server, instead of a descriptive message you can get this:

Mailbox unavailable. The server response was: <SEND_TO@SEND_TO.com> No such user here

The error message complains about the addressee, the email address you are sending the email to, instead of the username and password you try to use to access your SMTP server.

Make sure you are using the correct username and password to connect to the SMTP server.

Unable to read data from the transport connection: net_io_connectionclosed.

When your application tries to send an email using a Gmail account you may get the error message:

Unable to read data from the transport connection: net_io_connectionclosed.

One of the causes can be, when “Less secure apps” are not allowed to use the Gmail account.

To enable “Less secure apps” to use the Gmail account

Log into Gmail and navigate to
https://www.google.com/settings/security/lesssecureapps
Select the Turn on radio button