Docker Swarm overview

Docker Swarm Hierarchy

  • image in the registry
  • container
  • task ( container and the command to run in it )
  • service ( one or multiple instances of the same task, like multiple copies of the same web API )
  • stack ( one or multiple services that belong together, like a front end web application, middle tier, and database server launch scripted in a .yml file )

The difference between the service and the stack is like docker run vs. docker compose, but in a Docker Swarm cluster.

Docker Swarm Services

Global service

Global services will run on every available node once.

Replicated service

The Manager distributes the given number of tasks ( containers and commands to run ) of the replicated services on the nodes based on the desired scale number, that can be one. Once a task is assigned to a node it cannot be moved, it will run on that node until stops or fails.

Docker Swarm Networking

Host network

Uses the host’s network stack without any namespace separation, and sharing all of the host’s interfaces.

Bridge network

Docker-managed Linux bridge on the Docker host. By default, all containers created on the same bridge can talk to each other.

Overlay network

An overlay network that may span over multiple Docker hosts. Uses the gossip protocol to communicate between hosts.

None

The container’s own network stack and namespace, without any interfaces. It stays isolated from every other network, and even its own host’s network.

MACVLAN

Establishes connections between container interfaces and parent host interfaces. They can be used to assign IP addresses that are routable on physical networks to containers.

Docker Swarm Load Balancing

Internal load balancing

Internal load balancing is enabled by default. When a container contacts another container in the same Docker Swarm, the internal load balancer routes the request.

External ingress load balancing

To enable the external ingress load balancing, publish the port of the service with the –publish flag. Every node in the cluster starts to listen on the published port to answer incoming requests. If the service does not run a container on the node that received the request, the Routing Mesh will route the request to the node that runs the container on the Ingress Network.

Routing Mesh. Source: https://success.docker.com/article/ucp-service-discovery

Create a service with an image in a private registry

These instructions will pass the login token from your local client to the Docker Swarm nodes, so those are able to log into the registry and pull the image.

# Save the Docker Registry password in the PASSWORD environment variable 

# Log into the Docker Registry
echo $PASSWORD | docker login -u [user] registry.my_registry.com --password-stdin

# Create the service
docker service create \
  --with-registry-auth \
  --name my_service \  
 registry.my_registry.com/my_namespace/my_image:latest

Install and configure Visual Studio Code

Install Visual Studio Code

See https://code.visualstudio.com/docs/setup/mac

On Windows

Select all four checkboxes for maximum integration

On macOS
Move Visual Studio Code to the Applications folder

The official instructions above start Visual Studio Code in the Downloads folder. Move the file into the Applications folder.

Configure Visual Studio Code

Start Visual Studio Code from the command line

This is a very useful feature

On Windows

Add the Visual Studio Code executable location to the System path:

"C:\Users\MY_USER_NAME\AppData\Local\Programs\Microsoft VS Code\AppData\Local\Programs\Microsoft VS Code"
  • Restart the all terminals for the installation settings to take effect
  • Type code in the terminal to start Visual Studio Code

Enable code execution in PowerShell

When the PowerShell terminal starts in Visual Studio Code it tries to load the profiles. If an error message is displayed, enable code execution. Run the command in a PowerShell as Administrator

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned

Customize Visual Studio Code

The Visual Studio Code configuration settings are stored in a JSON file on your workstation. You can edit the file and after restart the settings take effect, or you can set the values in the user interface one-by-one.

To customize Visual Studio Code by editing the settings file

Open the settings.json file. The double quotes are important, as both paths contain spaces.

On Mac: $HOME/Library/Application\ Support/Code/User/settings.json

On Windows: “%APPDATA%\Code\User\settings.json”

Useful settings
  • Enable double-click to select the entire word even if it contains a dash (hyphen, minus), as many variables and keywords in modern languages. Remove “-” (dash) from the list in
    Terminal -> Integrated: Word Separators
    Editor: Word Separators (editor.wordSeparators)
  • Set the tab size to 2
    Editor: Tab size
  • Insert final new line
    Files: Insert Final Newline

My current configuration settings file looks like this:

{
    "editor.acceptSuggestionOnEnter": "off",
    "workbench.colorTheme": "Visual Studio Dark",
    "files.insertFinalNewline": true,
    "workbench.startupEditor": "newUntitledFile",
    "editor.renderWhitespace": "none",
    "editor.wordSeparators": "`~!@#$%^&*()=+[{]}\\|;:'\",.<>/?",
    "go.formatTool": "goimports",
    "go.useLanguageServer": true,
    "workbench.colorCustomizations" : {
        "activityBar.activeBackground":"#95968888"
    },
    "editor.tabSize": 2,
    "mssql.connections": [
        {
            "server": "{{put-server-name-here}}",
            "database": "{{put-database-name-here}}",
            "user": "{{put-username-here}}",
            "password": ""
        },
    ],
    "workbench.editorAssociations": {
        "*.ipynb": "jupyter-notebook"
    },
    "notebook.cellToolbarLocation": {
        "default": "right",
        "jupyter-notebook": "left"
    },
    "redhat.telemetry.enabled": false,
    "editor.minimap.enabled": false,
    "go.toolsManagement.autoUpdate": true,
    "search.exclude": {
        "**/.terraform": true
    },
    "[yml]": {
        "editor.insertSpaces": true,
        "editor.tabSize": 2,
        "editor.autoIndent": "advanced"
        },
    "[yaml]": {
        "editor.insertSpaces": true,
        "editor.tabSize": 2,
        "editor.autoIndent": "advanced"
        },
        "git.openRepositoryInParentFolders": "always",
        "[python]": {
            "editor.formatOnType": true
        },
        "editor.inlineSuggest.enabled": true,
        "application.shellEnvironmentResolutionTimeout": 60,
        "[typescriptreact]": {
            "editor.defaultFormatter": "vscode.typescript-language-features"
        },
        "aws.telemetry": false,
        "aws.suppressPrompts": {
            "codeWhispererNewWelcomeMessage": true
        },
        "[jsonc]": {
            "editor.defaultFormatter": "vscode.json-language-features"
        },
        "terminal.integrated.experimentalInlineChat": true,
        "prisma.showPrismaDataPlatformNotification": false,
        "mssql.enableRichExperiences": true
    },
    "files.exclude": {
        "**/*.js": { "when": "$(basename).ts" },
        "**/**.js": { "when": "$(basename).tsx" }
    }
}

To customize Visual Studio Code using the UI

  • Open the settings page with Command-, (comma)

or

  • Start Visual Studio Code
  • Open Preferences -> Settings

Only tab should accept the suggestion

To force Visual Studio Code to only insert the suggested word with the Tab key, and configure the Enter key to always insert a new line

  • On the settings tab search for tab
  • Set the Editor: Accept Suggestion On Enter to off

Adds “editor.acceptSuggestionOnEnter”: “off” to the setting.json file

Add trailing newline to every file
  • On the settings tab search for insert final newline
  • Check the Insert Final Newline checkbox

Adds “files.insertFinalNewline”: true, to the settings.json file

Select text with hyphen with double click

To select the entire textwithhyphen with double click

  • On the settings tab search for editor.wordSeparators
  • Delete (hyphen) from the separator characters

Adds “editor.wordSeparators”: “`~!@#$%^&*()=+[{]}\\|;:’\”,.<>/?” to the settings.json file.

Exclude libraries from search

When we search the source code it can take along time for Visual Studio Code to search through the libraries that only support our application. Exclude the known library folders. Most of them are already included, add the .terraform folder to the list.

  • On the settings tab search for search.exclude and click the Add Pattern button
  • Enter **/.terraform into the field and click the OK button

Adds

"search.exclude": {
        "**/.terraform": true
    }

to the settings.json file.

Ignore compiled .js files during search when a .ts file exists with the same name

Add this to the settings.json file

    "files.exclude": {
        "**/*.js": { "when": "$(basename).ts" },
        "**/**.js": { "when": "$(basename).tsx" }
    }
Insert 2 spaces into the .yml and .yaml files when we press the tab key, use tabs in Makefile
  • Open the extensions page with Shift-Command-X
  • Install the EditorConfig for VS Code plugin
  • Save this .editorconfig file in the root of your project files above all projects ( for example in the ~/Git directory)
[Makefile]
indent_style = tab

[*.yml]
indent_style = space
indent_size = 2
  • Restart Visual Studio Code
Another setting for yml files, (maybe not necessary if the EditorConfig plugin is installed)
  1. Start Visual Studio Code
  2. Open the Settings page with Command-,
  3. Enter yml into the search field and press enter, and click the Edit settings for yaml link
  4. Enter into the settings.json file
    ,
    "[yml]": {
    "editor.insertSpaces": true,
    "editor.tabSize": 2,
    "editor.autoIndent": false
    },
    "[yaml]": {
    "editor.insertSpaces": true,
    "editor.tabSize": 2,
    "editor.autoIndent": false
    }

Open a Bash terminal in Visual Studio Code

To open a new Bash terminal in Visual Studio Code, install Git Bash and restart Visual Studio Code for the change to take effect. See Install Git on Windows

To set Git Bash as the default terminal on Windows
  • In VS Code open Settings
  • On the settings page search for default profile
  • In the Features/Terminal section set the value for Windows to Git Bash

Upgrade Bash on macOS

To be able to debug Bash scripts in Visual Studio Code on macOS, we need to upgrade Bash to at least version 4.0. Even the most modern macOS installs, a more than 15 year old, Bash version 3.2.57 from 2007!!!

bash –version

GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin18)
Copyright (C) 2007 Free Software Foundation, Inc.

To upgrade Bash to the latest version follow the instructions by Daniel Weibel at https://itnext.io/upgrading-bash-on-macos-7138bd1066ba

IMPORTANT!!!
If you use iTerm (zshell) keep /bin/zsh as the last item in the /etc/shells file to make it the default shell when you open iTerm.

...
/usr/local/bin/bash
/bin/zsh

Useful extensions

  • EditorConfig for VS Code
  • GitLens — Git supercharged
  • Dev Containers by Microsoft
  • Go ms-vscode.go
  • Terraform mauve.terraform
  • Bash Debug rogalmic.bash-debug

You must specify a region. You can also configure your region by running “aws configure”.

When we execute an AWS CLI command, we need to supply the AWS Region. If it is not specified, we get the error message:

You must specify a region. You can also configure your region by running “aws configure”.

We can add the region with the –region command line argument, or store it in the ~/.aws/config file.

The format of the ~/.aws/credentials file is

[default]
aws_access_key_id = …
aws_secret_access_key = …

[my-account]
aws_access_key_id = …
aws_secret_access_key = …

The format of the ~/.aws/config file is the following. Make sure to add the word “profile” within the square brackets for every profile you specified in the credentials file, except for the “default” one!!!

[default]
region = us-east-1
output = json

[profile my-account]
region = us-east-1
output = json

x509: certificate signed by unknown authority

I have built a Docker container with a Go application that used the Go AWS SDK. When my program tried to access an S3 bucket I got the error message

RequestError: send request failed
caused by: Get https://MY_BUCKET_NAME.s3.amazonaws.com: x509: certificate signed by unknown authority

To solve the problem I had to add the following line to the Dockerfile

On Ubuntu

RUN apt ca-certificates && rm -rf /var/cache/apk/*

On Alpine

RUN apk update && apk add ca-certificates && rm -rf /var/cache/apk/* 

Docker for Windows cannot start: “Hardware assisted virtualization and data execution protection must be enabled in the BIOS”

To start Docker on Windows, Hyper-V and the Hypervisor has to be enabled on Windows.

  1. Start a PowerShell window as administrator
  2. Enable Hyper-V
    dism.exe /Online /Enable-Feature:Microsoft-Hyper-V /All
  3. Enable the Hypervisor
    bcdedit /set hypervisorlaunchtype auto
  4. Restart the computer

List the Active Directory groups the user is a member of

To list the Active Directory groups where the current user is a member, execute in the PowerShell window

(New-Object System.DirectoryServices.DirectorySearcher("(&(objectCategory=User)(samAccountName=$($env:username)))")).FindOne().GetDirectoryEntry().memberOf

To get the Active Directory groups for another user, replace THE_USER_NAME with the Active directory user name of the person.

(New-Object System.DirectoryServices.DirectorySearcher("(&(objectCategory=User)(samAccountName=THE_USER_NAME))")).FindOne().GetDirectoryEntry().memberOf

Debugging Go programs in Atom

To be able to debug Go programs in Atom, make sure the Go IDE is installed. See Install the Go IDE at Install and set up Go

To start debugging select the Debugger tab in the upper right corner of Atom

If you get the error message: The debugger is not attached, and when you click Attach debugger you get the error message: No debuggers installed, look for available debuggers on atom.io/packages

Check if the go-debug Atom package is installed

  • Open the Settings page from the File menu
  • Select Install on the left side
  • Search for go-debug
  • If the package is not installed, click the Install button

If you still get the same error message, download the go-delve package.

Visit https://github.com/derekparker/delve/tree/master/Documentation/installation for more info.

  • In the terminal window execute
    go get -u github.com/go-delve/delve/cmd/dlv
  • Restart Atom for the change to take effect.

 To debug your Go application

  • Click the Debugger tab in the upper right corner, and select the Debug config
  • To set a breakpoint, click the line in the program and press the F9 key. A red circle appears on the margin. 
  • The breakpoint appears on the right side of the screen too
  • Press F5 to run the application in the debugger
  • When the breakpoint is hit the toolbar on the right side shows the Stacktrace and the Variables. Click the arrow to open the section.
  • Click the arrow next to the variable name to show more details