Python ImportError: No module named …

Symptom:

from MY_FOLDER.MY_FILE import MY_CLASS causes an error:
ImportError: No module named MY_FOLDER.MY_FILE

Debugging:

  1. Execute the program in debug mode,
  2. In the Variables window expand Special Variables,
  3. Read the __warningregistry__ value.

Possible cause:

The __init__.py file is missing in the directory of the referenced class.

Solution:

Create an empty __init__.py file in the directory where the class is referenced from.

Python basics

Great Python introductory video for C# developers
https://www.youtube.com/watch?v=jj60geqaP08

__  (double underscore) Starting the attribute name with double underscore attribute makes it private. To expose the private attributes and methods

print(dir(MY_CLASS))

 

Code hierarchy

In Python indentation denotes the hierarchy of the instructions, not curly braces, like in C# or Java.

Directories

When you create a directory in your project, create an empty __init__.py file in the new directory to be able to reference objects from that directory, and avoid

from MY_FOLDER.MY_FILE import MY_CLASS causes an error:
ImportError: No module named MY_FOLDER.MY_FILE

Classes

Predefined methods

__init__(self)  Initializer method (constructor)

__repr__(self)  Representation

__iter__(self)  Method to define hot to iterate through the object

Decorators

@property  Exposes a method as a property of the class

from _04_anonymous.anonobject import AnonObject
ImportError: No module named _04_anonymous.anonobject

{(“Not importing directory ‘/Users/pinterl/Git/devops-scripts/Python_Tutorial/Python/Python_for_C_sharp_developers/_04_anonymous’: missing __init__.py”, <type ‘exceptions.ImportWarning’>, 6): True}

Working with PyCharm

Project management

Create a new project in PyCharm

  1. In the File menu select New Project…,
  2. On the left side select the Python framework you want to use,
  3. Enter the name for your project to the end of the Location path,
  4. If you have selected a framework that asks for more information
    1. Click the More Settings link and enter the additional values,
    2. Click the Create button,
    3. PyCharm generates folders for your project and application,
    4. When PyCharm asks you about compatibility inspection, enable it, to make sure your code is compatible with future versions of Python.
    5. PyCharm will enable Code compatibility inspection. Click OK to save the setting.
    6. PyCharm will still set your project interpreter to 2.6, so set it to 2.7. In the PyCharm menu select Preferences…,
    7. On the left side select Project:…, Project Interpreter,
    8. In the Project Interpreter drop-down select 2.7 and click the OK button.

Delete a project in PyCharm

The PyCharm project explorer does not have a delete option. To delete a PyCharm project

  1. Open the project location in Finder,
  2. Move the project folder to the Trash,
  3. The PyCharm project explorer window will reflect the change.

Shortcuts, tips, and tricks

 Auto generate

Auto generate functions, classes, attributes

  1. Enter the name of the undefined element where you want to use it,
  2. Click the light bulb, or press ⌥⏎ (Alt-Enter) to pop up the Intention Action alert,
  3. Select the action to generate the definition.

Auto generate fields for classes

  1. Enter the __init__ function of the class with the field names,
  2. Place the cursor on the field you want to generate the definition for,
  3. Click the light bulb, or press ⌥⏎ (Alt-Enter) to pop up the Intention Action alert,
  4. Select Add field ‘…’ to class …,
  5. Press Enter to select the option,
  6. Press Enter at the end of the new line to move the cursor to the next line, otherwise, the Intention Action alert does not pop up when you press ⌥⏎ (Alt-Enter)

 

Code completion

  • Press Ctrl-Space to complete the name of the element.
  • Press Ctrl-Space twice to complete the name of any class, even if it is not yet imported. The import statement will be automatically added.

Quick documentation

To get a short documentation on an element under the cursor, press F1. (By default on a Macintosh you need to press fn-F1, because the F1 button is programmed to adjust the brightness.)

For more information

Auto completing code, see https://www.jetbrains.com/help/pycharm/auto-completing-code.html

Keyboard shortcuts
https://www.jetbrains.com/help/pycharm/keyboard-shortcuts-and-mouse-reference.html

Create a Terraform Enterprise environment

If you just start to work with Terraform Enterprise, you need to create a Terraform environment.

Preparation

GitHub account

To access GitHub from Terraform Enterprise, create a GitHub team and account with admin access to the GitHub repository that will store the Terraform scripts.

  1. Create a GitHub team who will have admin access to the Terraform script repository,
  2. Create a GitHub user what Terraform Enterprise will use to access the Terraform script repository,
  3. Add the new GitHub user to your GitHub organization,
  4. Log into GitHub with the new user account credentials and wait for the email verification email from GitHub,
  5. Open the verification email from GitHub and click the button in the email to verify your email address,
  6. Stay logged into GitHub with the new user account and wait for the invitation email from GitHub to join the organization,
  7. Open the invitation email from GitHub and click the button to accept the invitation,
  8. In GitHub add the new GitHub user to the new team,
  9. Create a GitHub repository to store the Terraform scripts,
  10. Add the new team to the Terraform script repository with admin rights.

If the Terraform modules are in a separate GitHub repository, add the new team to that repository with Read rights.

AWS account

Create a new user account that Terraform Enterprise will use to access AWS

  1. Create a new AWS user,
  2. Add user rights
    1. AmazonEC2FullAccess
    2. AmazonS3FullAccess

Connect your Atlas account to GitHub

We will authorize Terraform Enterprise to access the Terraform script GitHub repository. We set this connection up in our personal profile in Terraform enterprise, but when we create new Terraform environments Terraform Enterprise will use this connection to access the GitHub repository the environment will be connected to. Do not use your personal GitHub account to make the connection, because if your personal account loses access to the GitHub repository, the Terraform environment will not be able to connect to it. For this connection, we have already created a new user in GitHub above.

  1. In a web browser log into GitHub with the user account you want Terraform Enterprise use to access the GitHub repository,
  2. In the same web browser open a new tab and navigate to https://atlas.hashicorp.com/settings/connections and log into Atlas with your Hashicorp account,
  3. Click Connect GitHub to Atlas
  4. On the GitHub authorization page click the Authorize hashicorp button,
  5. Later if you want to unlink Atlas from your GitHub account, in the Personal section select Connections, and click the Unlink button.

Create the GitHub repository

Create a repository in GitHub to store the Terraform config files. You can specify subdirectories for each Terraform environment to watch, so one repository can serve the security group creation, and EC2 instance creation for the same developer group.

  1. Create the GitHub repository,
  2. Create a folder that the Terraform Enterprise environment will monitor for changes.

Create a new environment

    1. In a web browser navigate to https://atlas.hashicorp.com/configurations/import,
    2. Log in with your Atlas user account,
    3. In the dropdown list of the New Environment page (click the text of the dropdown, not the arrow) select Link to GitHub.
      Name the environment to include the type of the server you want to launch and the server environment, for example, test_linux_sandbox.
      To save the environment click the Create & Continue button,
    4. On the left side select Variables,
    5. In the Environment section click the Edit button,

  4. If the environment is linked to the Terraform Enterprise Server

    1. The environment has been created. Click the your account settings page to create a personal token. It will allow you to access your account without using your username and password.
    2. Enter a description for the token and click the Generate Token button,
    3. Save the token value at a safe place
    4. To return back to the Runs page select Terraform Enterprise in the upper left corner
    5. Click the name of the environment you want to work with
    6. Configure the environment. On the left side click Settings
      1. Uncheck Plan on artifact uploads
    7. Store your personal access token on your workstation to be able to access Terraform Enterprise. Open a Bash window and execute the export command:

      export ATLAS_TOKEN=[YOUR AUTHENTICATION TOKEN]
    8. Add the terraform backend configuration to the .tf file on your workstation

      terraform {
  backend "atlas" {
    name    = "YOUR_ORGANIZATION/YOUR_ENVIRONMENT"
    address = "https://atlas.hashicorp.com"
  }
}
    9. Navigate to the Terraform configuration folder and initialize it with the backend configuration you have entered to the .tf file. 
      terraform init
    10. Push the Terraform configuration to the Enterprise server

      terraform push -name="YOUR_ORGANIZATION/YOUR_ENVIRONMENT" -atlas-address="https://atlas.hashicorp.com"
    11. On the left side click Configurations to see the list of configs your organization have pushed to the server.
    12. Run a Terraform Plan to see if your code works 
      terraform plan
    13. Execute your Terraform config on your workstation 
      terraform apply
    14. Push the result of the run to the Terraform Enterprise server 
      terraform push -name="YOUR_ORGANIZATION/YOUR_ENVIRONMENT" -atlas-address="https://atlas.hashicorp.com"
    15. To view the result of the run, on the left side click Runs.
    16. To view the created infrastructure states on the left side click States

Create the Terraform scripts

 

More resources

For a tutorial on Multiple Environments in Terraform see https://github.com/hashicorp/multiple-envs

No instances for regex `’, try running `kitchen list’ in Chef Test Kitchen

When I tried to execute the “kitchen list” command in Chef Test Kitchen the following error came up:

No instances for regex `’, try running `kitchen list’

I could not find the reason for the error, so I opened the .kitchen.yml file in the Atom editor, added a space to one of the comments, and saved the file. The error went away.

GitKraken installation and configuration

GitKraken is a Git user interface to manage Git repositories.

Installation

  1. Download the GitKraken installer from https://www.gitkraken.com/download

Configuration

  1. Start the GitKraken application
  2. Login with your GitHub account, or create a new account

    or
  3. Connect to a Git repository
  4. Click the Open a Repository button
  5. Select a repository folder and click the Open button

3D Printer GCODE instructions

GCODE is a standard file type to control 3D printers. The 3D object is usually exported from the CAD program to an STL file, that fully describes the end product. 3D printers build the physical objects layer-by-layer, so we need to slice the object into thin layers.

Turn off the fan after 5 minutes

When the printer completed the job, the fan of the Monoprice Select Mini printer says on, making noise, and wearing out the bearing. To turn the fan off after 5 minutes, add this code to the end of every GCODE file.

G4 P300000 ;wait 5 minutes before turning off the fan
M106 S1    ;turn off fan

To automatically append the instructions to every GCODE file, in Cura add the lines to the end.code section:

  1. In the Cura application select the Start/End-GCode tab,
  2. Select end.gcode,
  3. Enter the lines in the bottom window.

 

Terraform Enterprise Administration

Create a new organization

  1. In your web browser navigate to https://atlas.hashicorp.com/help/organizations,
  2. On the left menu bar under Organizations click Create,
  3. In the middle of the screen click the new organization page link,
  4. Enter the email address and user name for the organization owner and click the Create organization button.

Using Git

Frequently used Git commands

Git runs entirely on your workstation, and a copy of the entire repository is also on your local hard drive. GitHub, BitBucket, and other providers only give you a storage space to allow you to share your repository with others and provide a web user interface to manage it. You can use any provider’s application on your workstation to manage any of your Git repositories. Many developers use SourceTree, the great application written by Atlassian, the owner of BitBucket, to manage Git repositories on their workstations that are shared at GitHub.

Here are the most frequently used Git commands.

Create the local repository

Initialize a new Git repository in the current directory. This command creates the .git sub-directory to store your repository and its configuration file.

git init

Display the local repository status

Display the list of added, deleted and modified files in the local repository

git status

Display your changes in the files

Display the changes in files since the last git add

git diff

Display the changes in the local stage area after you have executed  git add

git diff --staged

Stage your changes in the local repository

Add your changes to the local stage area

git add .

Save the changes to the local repository

Commit your changes from the local stage area to the local repository with a message

git commit -m "My message"

Edit the last commit message

You can edit the last commit message even after the push.

git commit --amend

When you execute git status you get the message

Your branch and ‘origin/master’ have diverged,
and have 1 and 1 different commits each, respectively.
(use “git pull” to merge the remote branch into yours)
nothing to commit, working tree clean

To synchronize your local repository with the remote server, execute

git pull

The merge window pops up. You can leave the default message, or type your explanation. To save your message and close the window

  1. Press the ESC key on your keyboard
  2. Press the keys (including the colon at the beginning) :wq

Send your changes to the remote repository

Push the changes to the remote repository at GitHub, Bitbucket, or others

git push

Get the latest from the remote repository

Pull the latest changes from the remote repository

git pull

Advanced topics

Push an existing repository from the command line

git remote add origin https://github.com/ORGANIZATION/REPOSITORY_NAME.git
git push -u origin master

Clone a private Git repository from the command line

git clone https://my_username:my_password_or_token@github.com/my_user_name/my_repo_name.git

or use an SSH key, see https://help.github.com/en/articles/connecting-to-github-with-ssh

Branching

Forks and Pull Requests

View an old version of the repository

To view the repository in the state of an old commit and return back to the current state use the “checkout” command

Save the state of the working directory

Temporarily save the current state of the working directory including untracked (new) and ignored files

 git stash -a

List the commits of the repository

git log

Search all commits of a file in a branch

First, select the branch to search in, if it is not the current branch

git checkout BRANCH_NAME
git log --all --full-history -- **/MY_FILE.*

View the old version of the repository

git checkout MY_OLD_COMMIT_SHA

To create a new branch for the restored version and switch to it

git switch -c <new-branch-name>

To switch back to the “main” branch to see the latest version of the repository

git switch main

To switch to the new branch for the restored version of the repository

git switch <new-branch-name>

Create a new branch and continue the work

If you realize, this is the last stable version of the application, create a new branch and develop your project from this point forward. This way you will not lose changes you made after this commit, but you will be able to create new commits based on the working version and merge the new branch back to the “master” branch later.

git checkout -b MY_APP_WORKS_AGAIN_BRANCH

Restore the state of the working directory before the “stash” command

git stash pop

If you want to restore the repository to the state before the “git stash” with the “git stash pop” command and you have created new files, to prevent the accidental deletion of those files, Git will display the error message:

MyPath/File already exists, no checkout
Could not restore untracked files from stash

You also get his message when an ignored file has been created, like the “.DS_Store” file on Mac OS.

Remove ignored files

To remove the ignored files from the work tree execute the following. Please note the upper case “X”.

Dry run to see what will be deleted:

git clean -n -f -X

Delete the files:

git clean -f -X

Remove ignored and untracked files

To remove the ignored files, and new (untracked) files and directories from the working tree, clean the repository with the following. Please note the lower case “x”.

Dry run to see what will be deleted:

git clean -n -f -d -x

Delete the files:

git clean -f -d -x

Rollback

If you have accidentally committed a change and want to roll back the changes you can use the “reset” command. It is very dangerous because it can rewrite history, remove commits, delete files in your working directory, so you can lose your work. The “–mixed” is the default option of the “reset” command, so if no option is specified, that will be executed.

If the change has NOT been pushed to the remote repository (GitHub)

Remove a file from the stage

The “add” command adds files to the “stage”. If you have “add”-ed multiple files and do not want to “commit” one of them together with the rest, remove a file from the “stage”, but keep it in your working directory.

git reset HEAD -- <file>
git reset HEAD -- <directoryName>

Remove the last commit

Move the history back before the last “commit” and all “add”s that are associated with it. You will not lose any changes in your working directory. Use this command if you realize you want to make more changes before the next commit.

git reset HEAD~1

Remove the last commit and lose all changes since that

Restore the files in the repository to the state of the prior “commit”. You will lose all changes you made since that. This command moves the HEAD back one commit, so it deletes the last commit from the history.

git reset --hard HEAD~1

Undo the rollback

If the reset was unnecessary you can undo it for a limited time. Git runs the garbage collector every 30 days, and it removes orphaned commits, so you have 0 to 30 days to undo the rollback. If the garbage collector runs a few minutes after the reset, the changes are lost forever.

To see the list of commit SHAs that the garbage collector not yet deleted

git reflog

To undo the reset of a commit while it is still available

git checkout -b aNewBranchName shaYouDestroyed

If the changes have already been pushed to the remote repository

The “reset” command can cause serious problems for others working in the same repository.

To undo a commit

Use the “revert” command to correct mistakes. The “revert” command will create a new commit with the state you want without rewriting the history of the repository.

git revert <bad-commit-sha1-id>
git push origin
 

To restore the repo to the state of a previous commit

To erase history, and restore the repository to the state of a previous commit use the “reset” command. “git push .. -f” will force the push to erase the history on the server, and also overwrite other user’s commits after the specified one, so coordinate with them, and they also need to “reset”their local repositories.

git reset --hard <commit-id>
git push origin main -f

Remove a file form the entire history of the repository

If a file is too big to be uploaded to the remote repository, you may get the error message when you push the repository to the remote:

remote: Resolving deltas: 100%, done.
remote: error: GH001: Large files detected. You may want to try Git Large File Storage – https://git-lfs.github.com.
remote: error: Trace: …
remote: error: See http://git.io/iEPt8g for more information.
remote: error: File … is … MB; this exceeds GitHub’s file size limit of 100.00 MB
To https://github.com/….git
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to ‘https://github.com/….git’

You may also want to remove every trace of a file from the history for security reasons.

git filter-branch command

This command rewrites the history of the repository and removes every trace of a file

git filter-branch -f --index-filter 'git rm --cached --ignore-unmatch "MY_TOO_BIG_FILE_NAME"'

git-filter-repo utility

An alternative to remove files is the Python utility git-filter-repo

Clone the git filter repository from https://github.com/newren/git-filter-repo.git

git clone https://github.com/newren/git-filter-repo.git

Copy the git-filter-repo file to a directory in your path

Usage

Open a terminal window in the repository you want to clean

cd MY_REPOSITORY_TO_CLEAN

The git-filter-repo utility selects files from the repository, so to delete those files and keep the rest of the files in the repository, use the --invert-paths option.

To remove files or directories from the repository
git filter-repo --path README.md --path guides/ --path tools/releases --invert-paths

To use wildcards include the --path-glob option
git filter-repo --path-glob 'src/*/data' --invert-paths

To remove multiple files or directories, save the list in a text file and refer to it with the --paths-fom-file option

git-filter-repo --invert-paths --paths-from-file /tmp/delete-from-git-repo.txt --force

BFG Cleaner

An other alternative to git filter-branch to remove files from the repository is at https://rtyley.github.io/bfg-repo-cleaner/