Custom Git Status and Git Pull commands to work with all of your repositories

If you work with many Git repositories it is easy to miss a repository when you commit your changes, and very time consuming to pull from all repositories.

It is easy to automate both processes.

In this example, we will place all files in the ~/Git/devops-scripts folder. If you want to place your files at another location change it in the files below.

\/  \/  \/  \/  \/  \/  \/  \/  \/  \/

In Windows

To find your home directory on a Windows workstation, open a Bash window (Git Bash) and execute

echo ~

/\  /\  /\  /\  /\  /\  /\  /\  /\  /\

Create the Bash script files

  • Create a file with the list of Git repositories and save it as ~/Git/devops-scripts/git-repositories.txt
    You can separate groups of repositories with empty lines.
# Local_Path+Remote_Path+Rename_the_folder_to(optional)
~/Git/MY_REPO1+https://github.com/MY_REPO1.git+RENAME_TO1
~/Git/MY_REPO2+https://github.com/MY_REPO2.git

~/Git/MY_REPO3+https://github.com/MY_REPO3.git
  • Create a file to check the status of all Git repositories and save it as ~/Git/devops-scripts/git-status.sh
#!/bin/bash

# git-status.sh
# Shows the GIT STATUS of the Git repositories and shows the recommended actions
# The list is in git-repositories.txt

eval input="~/Git/devops-scripts/git-repositories.txt" # Need EVAL, otherwise the Bash loop cannot interpret ~ as the home directory if the path contains ~

RED='\033[0;31m'
NC='\033[0m' # No Color

while IFS= read -r folder
do

 # Get the length of the path
 # On Windows the empty line manifests itself as a 1 character long not empty string
 path_length=${#folder}

 if [[ ( -n "$folder" ) ]] && [[ ( 1 < $path_length ) ]] ; then
 # The path is not empty and the length is greater than 1

 # Ignore the lines starting with #
 if [[ ! $folder == \#* ]] ; then
 # The line is not a comment, process it

 # Split the local and remote path
 eval local_path=$(echo "$folder" | cut -f1 -d+) # Need EVAL, otherwise the IF statement below does not work if the path contains ~
 remote_path=$(echo "$folder" | cut -f2 -d+)

 if [ ! -d "$local_path" ]
 then

 # The local directory doesn't exist.
 echo -e "${RED}$local_path does not exist, PULL to get $remote_path${NC}"

 else

 eval cd $local_path # Need EVAL, otherwise CD does not work if the path contains ~
 pwd | tr -d '\n'

 status=$(git status)

 # --------------------------------------------------------------------------
 # To handle

 # On branch master
 # Your branch is up-to-date with 'origin/master'.
 # Untracked files:
 # (use "git add <file>..." to include in what will be committed)

 if [[ $status == *"Untracked files"* ]]; then
 echo -e " ---- ${RED}New files added, please ADD${NC}" | tr -d '\n' # -e tells echo to enable backslash escapes
 fi

 # --------------------------------------------------------------------------
 # To handle

 # On branch master
 # Your branch is up-to-date with 'origin/master'.
 # Changes not staged for commit:
 # (use "git add <file>..." to update what will be committed)
 # (use "git checkout -- <file>..." to discard changes in working directory)

 if [[ $status == *"Changes not staged for commit"* ]]; then
 echo -e " ---- ${RED}Files changed, please ADD${NC}" | tr -d '\n' # -e tells echo to enable backslash escapes
 fi

 # --------------------------------------------------------------------------
 # To handle

 # On branch master
 # Your branch is up-to-date with 'origin/master'.
 # Changes to be committed:
 # (use "git reset HEAD <file>..." to unstage)

 if [[ $status == *"Changes to be committed"* ]]; then
 echo -e " ---- ${RED}Please COMMIT${NC}" | tr -d '\n' # -e tells echo to enable backslash escapes
 fi

 # --------------------------------------------------------------------------
 # To handle

 # On branch master
 # Your branch is ahead of 'origin/master' by 1 commit.
 # (use "git push" to publish your local commits)
 # nothing to commit, working tree clean

 if [[ $status == *"git push"* ]]; then
 echo -e " ---- ${RED}Please PUSH${NC}" | tr -d '\n' # -e tells echo to enable backslash escapes
 fi

 # --------------------------------------------------------------------------
 # To handle

 # On branch master
 # Your branch and 'origin/master' have diverged,
 # and have 1 and 1 different commits each, respectively.
 # (use "git pull" to merge the remote branch into yours)
 # nothing to commit, working tree clean

 if [[ $status == *"git pull"* ]]; then
 echo -e " ---- ${RED}Repos diverged, PULL to merge${NC}" | tr -d '\n' # -e tells echo to enable backslash escapes
 fi

 # --------------------------------------------------------------------------

 echo

 fi

 fi

 fi
done < "$input"
  • Create a file to pull from all Git repositories and save it as ~/Git/devops-scripts/git-pull.sh
#!/bin/bash

# git-pull.sh
# Pulls Git repositories and clones the missing ones
# The list is in git-repositories.txt

eval input="~/Git/devops-scripts/git-repositories.txt" # Need EVAL, otherwise the Bash loop cannot interpret ~ as the home directory if the path contains ~

RED='\033[0;31m'
NC='\033[0m' # No Color

while IFS= read -r folder
do

 # Get the length of the path
 # On Windows the empty line manifests itself as a 1 character long not empty string
 path_length=${#folder}

 if [[ ( -n "$folder" ) ]] && [[ ( 1 < $path_length ) ]] ; then
 # The path is not empty and the length is greater than 1

 # Ignore the lines starting with #
 if [[ ! $folder == \#* ]] ; then
 # The line is not a comment, process it

 # Split the local and remote path
 eval local_path=$(echo "$folder" | cut -f1 -d+) # Need EVAL, otherwise the IF statement below does not work if the path contains ~
 remote_path=$(echo "$folder" | cut -f2 -d+)
 rename_folder_to=$(echo "$folder" | cut -f3 -d+)

 # Check if the local path exists
 if [ ! -d "$local_path" ]; then
 # The local directory doesn't exist.
 echo
 echo -e "${RED}$local_path does not exist, cloning $remote_path${NC}"

 # Get the parent directory name
 parent_dir="${local_path%/*}"

 # Create the parent directory
 mkdir -p $parent_dir

 # Go to the parent_dir
 cd $parent_dir

 # Clone the Git repository
 git clone $remote_path

 if [[ ( -n "$rename_folder_to" ) ]] ; then

 # Rename the folder if necessary
 end_of_url="${remote_path##*/}"
 filename="${end_of_url%.*}"

 echo -e "${RED}Renaming $filename to $rename_folder_to${NC}"

 mv $filename $rename_folder_to

 fi


 else
 echo
 eval cd $local_path # Need EVAL, otherwise CD does not work if the path contains ~
 pwd
 git pull

 fi

 fi

 fi

done < "$input"

Enable the script execution

\/  \/  \/  \/  \/  \/  \/  \/  \/  \/

 In Mac OS

  • Open a terminal window in the ~/Git/devops-scripts folder
  • Run these commands to enable the execution of the Bash files
chmod u+x git-status.sh
chmod u+x git-pull.sh

/\  /\  /\  /\  /\  /\  /\  /\  /\  /\

Create an alias

Aliases allow you to execute commands from any folder in the command line and hide the complexity of the entire command.

\/  \/  \/  \/  \/  \/  \/  \/  \/  \/

In Mac OS

If you use the iTerm terminal window

iTerm reads the configuration settings from the~/.zshrc file. If you use iTerm add the alias definitions to the end of the ~/.zshrc file.

  • Open the~/.zshrc file and add the lines below to the end of it
If you use the built-in OSX Terminal

The built-in Terminal windows reads the configuration settings from the~/.bash_profile file.

  • Open the ~/.bash_profile file and add the lines below to it

In Windows

If you use Git Bash
  • Navigate to your home directory ( cd ~ )
  • Create the Bash config file if does not exist 
    cp > ~/.bashrc

    Bash will show the “cp: missing file operand” an error message, but it will create the file that you can edit with any text editor

  • Add the lines below to the file

/\  /\  /\  /\  /\  /\  /\  /\  /\  /\

 

alias status='~/Git/devops-scripts/git-status.sh'
alias pull='~/Git/devops-scripts/git-pull.sh'

Refresh the settings in the Bash shell environment

\/  \/  \/  \/  \/  \/  \/  \/  \/  \/

In MacOS

source ~/.bash_profile

In Windows

Close the Bash window and open it again

/\  /\  /\  /\  /\  /\  /\  /\  /\  /\

How to use it

Open a terminal window in any folder

To check if you need to add, commit or push files in a Git repository:

status

If you have added new files to the repository you get:

/Users/YOUR_USER_NAME/Git/devops-scripts —- New files added, please ADD

If you have changed files in a Git repository you get:

/Users/YOUR_USER_NAME/Git/devops-scripts —- Files changed, please ADD

If you have already executed the git add . command, but have not yet committed the changes:

/Users/YOUR_USER_NAME/Git/devops-scripts—- Please COMMIT

If you have already committed the changes, but have not pushed the to the remote repository:

/Users/YOUR_USER_NAME/Git/devops-scripts —- Please PUSH

To pull from every Git repository before you start to work and clone the missing ones

pull

 

 

Useful Bash commands

Most of these commands work in all Linux, MacOSX, and Unix shells, but some of them only work in selected shells.

Find files recursively by file name pattern

find . -name '*.MY_EXTENSION'

Find files, but exclude a directory from the search. (The -print is needed at the end of the line)

find . -path ./tmp/windows -prune -o -name '*.MY_EXTENSION' -print

Get the absolute path of files recursively

find `pwd` -name MY_FILENAME

Delete files recursively by file name pattern

Works in zsh but not in sh or bash.
New versions of bash cannot do this by default only if the shell option globstar is enabled:
shopt -s globstar
The globbing pattern is **/*.o but that is not limited to files.

find . -type f -name '*.MY_EXTENSION' -delete

Make a shell script file executable

chmod u+x FILE_NAME.sh

Directory Listing

Sort by date, most recent on top

ls -lt

Sort by date oldest on top

ls -ltr

Sort by date, display only last modification date and file name

ls -lt | awk '{print $7,$8,$9, $10}'

Display the CHMOD numbers in the output of the ls command

ls -l | awk '{k=0;for(i=0;i<=8;i++)k+=((substr($1,i+2,1)~/[rwx]/) \
 *2^(8-i));if(k)printf("%0o ",k);print}'

Generate a public SSH key from the private key

ssh-keygen -y -f PRIVATE_KEY_FILE.pem > PUBLIC_KEY_FILE.pub

Replace all spaces in every filename and directory name recursively with dash

find . -name “* *” -print0 | sort -rz | \
while read -d $’\0′ f; do mv -v “$f” “$(dirname “$f”)/$(basename “${f// /-}”)”; done

To check if a package is installed

# For RHEL 
rpm -qa *MY_SEARCH_TERM*
# For Ubuntu
dpkg —list | grep MY_PACKAGE_NAME

Security and user management

Add a user to a Linux user group

To add your user account to a user group

sudo usermod -aG THE_GROUP_NAME $USER

To add another user to a group

sudo usermod -aG THE_GROUP_NAME THE_USER_NAME

Windows 10 can’t connect to this network

If your Windows 10 computer can successfully connect to some wireless networks, but cannot connect to certain wireless networks, it is possible, that the stored credentials are out of date. To force Windows 10 to ask you for the updated username and password

  1. In the Start Menu select Settings,
     
  2. On the Windows Settings page select Network & Internet,
  3. On the left side select Wi-Fi,
  4. On the Wi-Fi page select Manage known networks,
  5. Select the network you are not able to connect and click the Forget button,
  6. In the lower right corner of the screen select the Wi-Fi network icon, select the network, and click the Connect button.

 

Find the AWS AMI that a RightScale server template is using

When you use RightScale to launch servers in the cloud, you want to use the same base image to test your Chef cookbooks in Test Kitchen.

Packer by Hashicorp is a utility to create custom server images based on cloud images. You supply the image ID and other configuration parameters to create a new custom image.

To find the AMI ID of the base image of the RightScale Server Template

  1. Open the RightScale user interface in your browser,
  2. Select the RightScale account where the server template was created,
  3. In RightScale Cloud Management select  Design / ServerTemplates,
  4. Select the server template,
  5. Select Images,
  6. Select the MultiCloud image,
  7. Select Clouds,
  8. Select the image in the region you are working in,
  9. The Resource UID is the AMI ID of the image.

Mailbox unavailable. The server response was: No such user here

When you send email from an application, most SMTP servers require authentication. Your application needs to supply a valid username and password to be able to connect to the server. This prevents strangers to use your SMTP server and send millions of spam emails.

When your application does not supply the correct credentials to connect to the SMTP server, instead of a descriptive message you can get this:

Mailbox unavailable. The server response was: <SEND_TO@SEND_TO.com> No such user here

The error message complains about the addressee, the email address you are sending the email to, instead of the username and password you try to use to access your SMTP server.

Make sure you are using the correct username and password to connect to the SMTP server.

Unable to read data from the transport connection: net_io_connectionclosed.

When your application tries to send an email using a Gmail account you may get the error message:

Unable to read data from the transport connection: net_io_connectionclosed.

One of the causes can be, when “Less secure apps” are not allowed to use the Gmail account.

To enable “Less secure apps” to use the Gmail account

  1. Log into Gmail and navigate to
    https://www.google.com/settings/security/lesssecureapps
  2. Select the Turn on radio button

 

The .NET MVC model has to have a parameterless constructor

When you create a class to be used as the model in a .NET MVC application, the class has to have a parameterless constructor. It is called during the POST when the user clicks the submit button of the form. If the class does not have a parameterless constructor, we will get the following generic error message. None of the breakpoints can help you to find the cause, and there is no entry in the event log.

The website cannot display the page
HTTP 500

Most likely causes:
•The website is under maintenance.
•The website has a programming error.

What you can try:
Refresh the page.
Go back to the previous page.
More information

Turn off the built in touchpad when a mouse is attached

All laptops today feature a touchpad, that enables you to move the pointer on the screen. It can be very annoying when your thumb accidentally moves the cursor when you type, even if an external mouse is connected to the computer.

Windows 10 provides a checkbox to automatically disable the internal pointing device when you connect a mouse.

  • Open the Control Panel,
  • Select Mouse,
  • On the Device Settings tab check the Disable internal pointing device when external USB pointing device is attached checkbox.

How to verify or generate file checksums

You can verify the checksum of a file with the built-in features of the operating systems

On Macintosh and Linux

Open the terminal window and execute the commands

SHA-256

shasum -a 256 FILE_NAME

On Windows

CRC32, CRC64, SHA256, SHA1, BLAKE2sp

Right-click the file in Windows Explorer and select CRC SHA > * (star)

The popup window will show the values, but unfortunately, there is no way to copy them to the clipboard. To be able to copy the hash value to the clipboard open a Bash window on the Windows computer and execute the Macintosh/Linux commands above.

To install Bash on a Windows workstation see the Git section of Install the DevOps development tools on Windows.