Turn off the GDPR cookie notification in a dotnet core MVC web application

The dotnet core MVC web application template adds the GDPR cookie acceptance message to the header of the page template. When the site is hosted with HTTP the accept button does not clear the message resulting an infinite loop, when the user cannot log in and the site continuously asks for the acceptance.

When the site is hosted in a Docker container it is also simpler to instal the SSL certificate on the load balancer and host the site with HTTP on port 80.

We use cookies. To be able to use this site please click the Accept button.

To remove the feature delete the

<partial name="_CookieConsentPartial" />

line from the Views\Shared\Layout.cshtml file.

Read the database connection string and other secrets from environment variables

It is not good practice to store secrets in the source code repository. The sample C# dotnet core application code stores the database connection string in the appsettings.json file which eventually will be pushed into GitHub. To follow the Twelve-Factor App methodology, store the configuration values in environment variables and load them runtime. This method enables your application to run in a Docker container and in a Kubernetes pod.

If the connection string is not specified, we get the error message

System.ArgumentNullException: ‘Value cannot be null.
Parameter name: connectionString’

To read the connection string from an environment variable

Add the builders to the Startup.cs file

       public Startup(IConfiguration configuration, IHostingEnvironment env)
        {
            Configuration = configuration;
            Env = env;

            // Read the connection string from the "ASPNETCORE_ConnectionStrings:DefaultConnection" environment variable
            // RESTART Visual Studio if you add or edit environment variables for the change to take effect.
            var builder = new ConfigurationBuilder()
            .SetBasePath(env.ContentRootPath)
            .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
            .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true);

            // This one needs to be last for the environment variables to have the highest priority
            builder.AddEnvironmentVariables("ASPNETCORE_");
            Configuration = builder.Build();
        }

Create the ASPNETCORE_ConnectionStrings__DefaultConnection environment variable. The double underscore will be converted to a colon ( : ) . This variable name works on Windows, and also in Linux where the environment variable name cannot contain colons.

Server=MY_SERVER_URL;Port=5432;Database=MY_DATABASE;Username=MY_USER;Password=MY_PASSWORD

IMPORTANT: If you add or change an environment variable, restart Visual Studio for the change to take effect. It is not enough to restart the application.

Mount NFS volume from macOS

When we try to mount an NFS with

docker run --rm -i -v MY_MOUNT:/data alpine

from macOS we get the error message:

docker: Error response from daemon: error while mounting volume ‘/var/lib/docker/volumes/…/_data’: failed to mount local volume: mount :/data:/var/lib/docker/volumes/…/_data, data: addr=…,vers=4: operation not permitted.

or

docker: Error response from daemon: error while mounting volume ‘/var/lib/docker/volumes/…/_data’: failed to mount local volume: mount :/data:/var/lib/docker/volumes/…/_data, data: addr=…,vers=4: connection refused.

To be able to mount the volume from macOS, use the insecure option in /etc/exports

MY_MOUNT 10.0.0.0/8(rw,sync,no_root_squash,no_all_squash,insecure)

ERROR: column “…” does not exist

PostgreSQL converts all column and table names to lowercase in the SQL query.

If the column or table name contains uppercase letters, we need to use double quotes to be able to reference them.

When we get the error message

ERROR: column “username” does not exist

select * from public.”AspNetUsers” where UserName = ‘… ^

HINT: Perhaps you meant to reference the column “AspNetUsers.UserName”. SQL state: 42703

This error message shows, that PostgreSQL converted UserName to username. To be able to reference the column, use double quotes. If you reference the table name too, don’t copy the hint, make sure the table and column names are separate strings in double quotes with a period between them.

select *  from public."AspNetUsers" where "AspNetUsers"."UserName" = 'MY_VALUE';

PostgreSQL management tools

Install postgresql

The postgresql package contains the PostgreSQL utilities: psql, pg_dump

On macOS

 brew install postgresql

Install pgAdmin

pgAdmin is a browser based database management tool for PosgtreSQL databases.

Download

Install

On macOS

  • Double click the downloaded file
  • Accept the license agreement
  • Drag the application into the Applications folder

Start pgAdmin

On macOS

  • Start the pgAdmin application from the Launchpad. The application runs in a browser window.

Install the pgcli command line utility

On macOS

brew install pgcli

Using pgcli

pgcli is a wrapper of postgresql with limited command set to enable testers and support staff to monitor PostgreSQL databases

Start pgcli with

PGPASSWORD=MY_ADMIN_PASSWORD pgcli -h MY_SERVER_URL -U MY_USERNAME -d MY_DATABASE_NAME

To see the help on the pgcli commands

\?

syntax error, unexpected tLABEL in Chef Test Kitchen

During the test of a Chef recipe that contains a template we may encounter the error message

syntax error, unexpected tLABEL
MY_KEY: MY_VALUE

Make sure there is no white space between the keyword variables and the opening parenthesis in the template definition:

template "MY_FILE" do
  source 'MY_SOURCE'
  variables(
    MY_KEY: MY_VALUE
  )
end

How to remove all Docker containers and images from the host

Docker images are stored on the host to launch as fast as possible. The drawback is, Docker can fill the hard drive with unused images. A recent possible bug in Docker caused the accumulation of unused, unreferenced images on the host. To delete all Docker containers and images, use these commands in the terminal

Get the base line

# Get the free disk space
df -h

# List the running containers
docker ps

# List all containers
docker ps -a

# List the Docker images
docker images

Gentle clean up

  • all stopped containers
  • all networks not used by at least one container
  • all dangling images
  • all dangling build cache
docker system prune

All unused images and stopped containers

docker rm $(docker ps -qa); docker rmi -f $(docker images -q);yes | docker system prune

Deep cleaning

This will stop and delete ALL containers and images on the computer !!!

# Stop all running containers
docker stop $(docker ps -q)

# Remove all stopped containers
docker rm $(docker ps -qa)

# Remove all unused Docker images
docker rmi -f $(docker images -q)

# A final clean
docker system prune

Error: Error creating LB Listener: ValidationError: ‘arn:aws:elasticloadbalancing:….b’ must be in ARN format

We create a new AWS Application Load Balancer and get this error message when the second listener is created

Error: Error creating LB Listener: ValidationError: ‘arn:aws:elasticloadbalancing:….’ must be in ARN format

make sure the region and AWS profile are correctly set for the new listener config.

Learn Kubernetes part 3 – Traefik Ingress Controller

In the previous post, Learn Kubernetes part 2 – NGINX Ingress Controller, we have deployed a web application and exposed it via the kubernetes/ingress-nginx ingress controller.

In this exercise we will use Traefik, another popular proxy server. Traefik is widely used for multiple reasons: it is easier to configure, can automatically generate and renew SSL certificates using Let’s Encrypt. We will also use most of the scripts from the prior exercise.

Create the Traefik Role Based Access Control with ClusterRoleBinding. This will allow Traefik to serve all namespaces of the cluster. To restrict the Traefik to one namespace only, use the “namespace-specific RoleBindings”.

traefik-rbac.yaml (Based on https://docs.traefik.io/user-guide/kubernetes/)
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
    - extensions
    resources:
    - ingresses/status
    verbs:
    - update
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system

Deploy Traefik with a deployment. This introduces a few more network hops, but follows the Kubernetes best practices guidelines and provides more flexibility.

traefik-deployment.yaml (Based on https://docs.traefik.io/user-guide/kubernetes/)
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
        - name: admin
          containerPort: 8080
        args:
        - --api
        - --kubernetes
        - --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 8080
      name: admin
  type: NodePort

Create the resources

Launch the scripted resources

kubectl apply -f .

Check the Traefik pods. List the pods in the kube-system namespace, and make sure the “traefik-ingress-controller” is running.

kubectl --namespace=kube-system get pods

Expose the Traefik Web UI with a service

To be able to access the Traefik Web UI we will create a service and an ingress to expose it outside of the cluster.

ui.yaml (Based on https://docs.traefik.io/user-guide/kubernetes/)
---
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  rules:
  - host: traefik-ui.minikube
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-web-ui
          servicePort: web

Create the resources

Launch the scripted resources

kubectl apply -f .

Check the ingress

List the cluster ingresses

kubectl get ingress --all-namespaces
NAMESPACE     NAME             HOSTS                 ADDRESS   PORTS   AGE
kube-system   traefik-web-ui   traefik-ui.minikube             80      53s

Access the Kubernetes cluster via Traefik

To be able to access the traefik-ui.minikube host we need to modify the /etc/hosts file. Add the host to the file with the localhost IP address.

echo "127.0.0.1 traefik-ui.minikube" | sudo tee -a /etc/hosts

==================================

app1-frontend-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app1-frontend-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: app1-frontend-template-label
  template:
    metadata:
      labels:
        app: app1-frontend-template-label
    spec:
      containers:
        - name: app1-frontend-container-label
          image: nginx:1.7.9
          ports:
          - containerPort: 80
app1-frontend-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: app1-frontend-service
spec:
  selector:
    app: app1-frontend-template-label
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 80