Reverse engineer a database with AspNetCore in Visual Studio

For some reason the .NETCore designers did not think, that developers want to follow best practices by separating the data layer from the presentation layer.

The Entity framework out of the box only works if the database is accessed from the main application project.

When we try to reverse engineer a PostgreSQL database from a class library with the command:

cd MY_CLASS_LIBRARY_DIRECTORY
dotnet ef dbcontext scaffold "Host=localhost;Database=MY_DATABASE_NAME;Username=MY_USERNAME;Password=MY_PASSWORD" Npgsql.EntityFrameworkCore.PostgreSQL

we get the error message:

The specified framework version ‘2.1’ could not be parsed
The specified framework ‘Microsoft.NETCore.App’, version ‘2.1’ was not found.
– Check application dependencies and target a framework version installed at:
/usr/local/share/dotnet/
– Installing .NET Core prerequisites might help resolve this problem:
http://go.microsoft.com/fwlink/?LinkID=798306&clcid=0x409
– The .NET Core framework and SDK can be installed from:
https://aka.ms/dotnet-download
– The following versions are installed:
2.0.0 at [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
2.1.2 at [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
2.1.3 at [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

The problem is, that the …runtimeconfig.json files are only automatically generated in the bin/netcoreapp2.1 directory of the main application project, those are missing from all class libraries.

To enable the automatic generation of the …runtimeconfig.json files, add a line to the <PropertyGroup> section of the data layer class library project (.csproj) file.

<PropertyGroup>
...
<GenerateRuntimeConfigurationFiles>True</GenerateRuntimeConfigurationFiles>
...
</PropertyGroup>

 

Use two GitHub accounts on the same computer

Many developers have multiple GitHub accounts. One for personal projects, and another one for their work or business. There is a way to access multiple accounts simultaneously from the same computer. Only one account at a time can be configured using the HTTP connection, but we can configure the rest of the accounts using SSH.

There are many great articles on this topic, but most of them only provide partial instructions on how to set up multiple GitHub accounts on the same computer. In this post, I will document all the necessary steps to set up two accounts.

Usually, you don’t have much control over what GitHub connectivity is used for the company account, it can be dictated by the customer or your workplace, but you can always control what to use for your personal account. With SSH you can specify all aspects of your personal account, so you can leave the default SSH or HTTP connection for the business or company account.

In this post, we will set up two accounts with SSH connection. Even if the business account currently uses HTTP, it makes sense to set up SSH for that account too, in case they will switch to SSH at a later time.

Create two keys

    1. Open a terminal window and start the authentication agent to avoid the “Could not open a connection to your authentication agent” error later
       eval `ssh-agent -s`
    2. Create the personal SSH key
      ssh-keygen -t rsa -C "MY_PERSONAL_EMAIL@ADDRESS"

      when asked for the file name by the “Enter file in which to save the key“prompt, save the key as id_rsa_pers

    3. Add the personal key to the authentication agent
      ssh-add id_rsa_pers
    4. Create the business SSH key
      ssh-keygen -t rsa -C "MY_COMPANY_EMAIL@ADDRESS"

      when asked for the file name by the “Enter file in which to save the key“prompt, save the key as id_rsa_COMPANY_NAME

    5. Add the business key to the authentication agent
      ssh-add id_rsa_COMPANY_NAME

If you want to use existing keys from another location

If you already have the keys at another location or at another user account, copy the ~/.ssh directory to the new home directory. Do the same if you want to use the same keys in sudo mode. In that case switch to sudo mode with

sudo -i

Copy the .ssh directory

cd ~
cp -r /home/OTHER_USER/.ssh .

Set the file permissions

If you have copied the keys from another location, the file permissions may not set correctly. If the key permissions are too open, Linux displays the message:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: UNPROTECTED PRIVATE KEY FILE! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0755 for ‘/home/MY_USER_NAME/.ssh/id_rsa_pers’ are too open.
It is required that your private key files are NOT accessible by others.
This private key will be ignored.
Load key “/home/MY_USER_NAME/.ssh/id_rsa_pers”: bad permissions
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Set the file permissions. Execute the following statements in a terminal window

  1. Navigate to the .ssh directory
    cd ~/.ssh
  2. Set the key file permissions
    sudo chmod 600 ~/.ssh/id_rsa
    sudo chmod 600 ~/.ssh/id_rsa.pub
    sudo chmod 600 ~/.ssh/id_rsa_pers
    sudo chmod 600 ~/.ssh/id_rsa_pers.pub
    sudo chmod 600 ~/.ssh/id_rsa_COMPANY_NAME
    sudo chmod 600 ~/.ssh/id_rsa_COMPANY_NAME.pub
    
  3. Set the nown_host file permission
    sudo chmod 644 ~/.ssh/known_hosts
  4. Set the directory permission
    sudo chmod 755 ~/.ssh
  5. Start the authentication agent to avoid the “Could not open a connection to your authentication agent” error later
     eval `ssh-agent -s`
  6. Register the keys
    ssh-add id_rsa
    ssh-add id_rsa_pers
    ssh-add id_rsa_COMPANY_NAME

Configure SSH to use the two new keys

  1. Create the SSH config file
    touch ~/.ssh/config
  2. Open the ~/.ssh/config file with a text editor
  3. Add the following lines
    #Personal GitHub
    Host github-pers
      HostName github.com
      User MY_PERSONAL_GITHUB_USERNAME
      IdentityFile ~/.ssh/id_rsa_pers
    
    #Company GitHub
    Host github.com
      HostName github.com
      User MY_COMPANY_GITHUB_USERNAME
      IdentityFile ~/.ssh/id_rsa_COMPANY_NAME
    

Upload the keys to your GitHub accounts

  1. Log into the GitHub website with your personal credentials
  2. In the upper right corner click the down arrow next to your icon, and select Settings
  3. On the left side select SSH and GPG keys
  4. Click the New SSH key button
  5. Type a name for the key,  paste the contents of the ~/.ssh/id_rsa_pers.pub public key file into the text area, and click the Add SSH key button
  6. Log into Github with your company credentials
  7. Upload the id_rsa_COMPANY_NAME.pub public key file as explained above.

Create a personal repository

  1. Create a new folder, personal-test, and add a file to it on your computer
  2. Initialize the Git repository in the folder
    git init
    git add .
    git commit -m "Initial commit"
  3. Log into your personal GitHub account
  4. Create a new repository, and name it personal-test
  5. On the confirmation page click the SSH button
  6. Copy the …or push an existing… code to a text editor
  7. Edit the origin address to match the “Host” of the personal section in the ~/.ssh/config file, and execute the commands
    git remote add origin git@github-pers:MY_PERSONAL_GITHUB_USERNAME/personal-test.git
    git push -u origin master

Company repositories

For your company account use the default HTTP or SSH GitHub repository address without any modifications.

Change the AWS account to launch instances with Chef Test Kitchen

The Chef SDK contains Test Kitchen, that can launch server instances to test your Chef cookbooks. Test Kitchen uses the “chef_zero” provisioner to use your workstation as the virtual Chef server.

To switch Test Kitchen to launch instances in another AWS account

  1. In the .kitchen.yml file update the
    1. availability_zone
    2. subnet_id
    3. aws_ssh_key_id (if different in each account)
    4. security_group_ids. Make sure the management ports are open:
      1. for Linux: port 22 for SSH, or
      2. for Windows: port 5985 – 5986 for WinRM and port 3389 for Remote Desktop
    5. ssh_key (if different in each account)
  2. In the default.rb attribute file update the
    1. …[‘aws_profile’]
  3. In the ~/.aws/credentials file update the
    1. [default] section

Copy files between Linux machines

The rsync command allows you to copy files using SSH connection, between your workstation and another Linux machine. You have to be logged into one of the machines, this command cannot copy files between two remote machines.

To copy a file from your local machine to a remote server

rsync -avz -e "ssh -i SSH_KEY_FILE -o 'StrictHostKeyChecking=no'" --rsync-path="sudo rsync" SOURCE_FILE USER_NAME@SERVER_IP:TARGET_DIRECTORY/

To copy a file from a remote server to your local workstation

rsync -avz -e "ssh -i SSH_KEY_FILE -o 'StrictHostKeyChecking=no'" --rsync-path="sudo rsync" USER_NAME@SERVER_IP:SOURCE_FILE TARGET_DIRECTORY/

Using tmux terminal multiplexer

The tmux terminal multiplexer allows us to open multiple terminal windows in the same SSH session and continue the command execution even when we log out of the SSH session. This way we can execute long-running copy commands overnight without keeping the SSH session open.

Install tmux

On CentOS family Linux

yum install tmux

To start tmux and attach to the last session

tmux a

To keep the current session active

To continue the session execution even when you log out of the server, detach from the session before closing the connection. See the commands below.

Commands

To switch to binding mode

ctrl-b

Commands in binding mode

d Detach
% Vertical split
Horizontal split
arrows Move between panes
c Create new pane
p Previous window
n Next window
l Last used window
q Display window numbers
[ Enable scroll with arrow keys or pgup, pgdown. Press Esc or q to return normal mode
x Close the pane (will ask you to press “y” to confirm)

Zooming

cmd-shift + zoom in
cmd-shift – zoom out
cmd-shift 0 (zero) 100% zoom

Shared sessions

tmux -S /tmp/shared_session Start a new shared session
chmod 777 /tmp/shared_session Allow anyone to join your session
tmux -S /tmp/shared_session attach Attach to a shared session

Add SSH key access to a GitHub repository

Generate an SSH key pair

  1. In a terminal execute
ssh-keygen

Generating public/private rsa key pair.
Enter file in which to save the key : /Users/MY_USERNAME/Git/_Keys/MY_PROJECT/MY_PROJECT_rsa_ci
Enter passphrase (empty for no passphrase):
Enter same passphrase again:

Leave the passphrase empty, many systems cannot work with password protected key pairs.

This process will save the key-pair in two files. The private key with no extension, and the public key with the .pub extension.

Create a new service GitHub user account which will have access to the repository

  1. Create a new email address (GitHub requires unique email addresses for every user)
  2. Register a new user which will have access to the repository

Upload the public key to the service GitHub account

    1. Log into the GitHub account and select Settings in the drop-down in the upper right corner
    2. On the left side select SSH and GPG keys
    3. In the upper right corner select the New SSH key button
    4. Copy the above generated public key to your clipboard.
      On a Mac copy the public key to your clipboard with the command

      pbcopy < /Users/MY_USERNAME/Git/_Keys/MY_PROJECT/MY_PROJECT_rsa_ci.pub
    5. Paste the public key into the textbox

Add the service user as a collaborator to the GitHub repository

  1. Log into the GitHub account that has admin access to the repository
  2. Navigate to the repository and select Settings
  3. Select COllaborators & teams
  4. Enter the username into the search box and click Add collaborator

If you use two-factor authentication in Github

If you use two-factor authentication in your GitHub account, and you need automated access to it, create a Personal Access Token and use it instead of your password. GitHub will not ask you to verify your identity.

Generate a Personal Access Token

    1. Log into your GitHub account
    2. On the left side select Developer settings
    3. Select Personal access tokens
    4. Click the Generate new token button
    5. Select the repo checkbox
    6. Copy the token to your clipboard. This is the last time you are able to see the token.
    7. Save the token at a secure location, and use it instead of your password when you need automated access to your GitHub account.

Creating CentOS server images with Packer

The CentOS images are not available on the AWS Quick Start tab.

CentOS publishes official images on the AWS Marketplace, but you need to subscribe to the image to be able to launch it with an automation software, like Terraform.

Find the latest available CentOS image in the AWS Marketplace

  1. Execute this command to display the list of available images
    aws --region us-east-1 ec2 describe-images --owners aws-marketplace --filters Name=product-code,Values=aw0evgkw8e5c1q413zgy5pjce
  2. Select the latest AMI from the list. The images are NOT ordered by date!

Accept the terms

In every account, you want to launch the image, you have to accept the license agreement terms

If you try to launch the image before you subscribe to it, the error message is displayed

Error launching source instance: OptInRequired: In order to use this AWS Marketplace product you need to accept terms and subscribe. To do so please visit http://aws.amazon.com/marketplace/pp?sku=…
amazon-ebs: status code: 401, request id: []

  1. Follow the link in the error message to http://aws.amazon.com/marketplace/pp?sku=…
  2. Click the Continue to Subscribe button
  3. Click the Accept Terms button
  4. The next page should display
  5. Once the new image has been created, share it with all the accounts you use.

SocketError: Error connecting to … Name or service not known

In an enterprise environment, the company usually operates its own DNS servers.

When a Linux instance launches in AWS, the DNS settings only contain the AWS DNS server. If the company DNS server settings are applied with Chef, during the first Chef Client run those settings do not take effect.

When we reference an internal DNS entry during the first Chef Client run we get the error message:

 

SocketError: Error connecting to https://INTERNAL_ADDRESS.net/… – Failed to open TCP connection to INTERNAL_ADDRESS.net:443 (getaddrinfo: Name or service not known)

  1. There are two solutions to handle this:
  2. Use the IP address of the internal server or load balancer in the URL ( switch from HTTPS to HTTP, because the SSL certificate will not work with the IP address),
  3. Set the DNS servers of the Linux server with Packer when you generate the base image.

The specified version of the NuGet package is not found in Artifactory

The “chocolatey_package” Chef resource can install NuGet packages from Artifactory.

Artifactory is inconsistent in case sensitivity when an application is searching for a NuGet package. When we specify the package ID only, the search is not case sensitive. If the package is called “GoogleChrome” and we search for “googlechrome” the NuGet package is found in Artifactory.

chocolatey_package 'googlechrome' do
  source 'devops-chocolatey'
  options "--allow-empty-checksums --ignore-package-exit-codes"
end

When we specify the version of the package, the search becomes case sensitive in Artifactory. The “chocolatey_package” Chef resource automatically converts the package IDs to lowercase, even if we spell the package ID the same as it is in Artifactory, so “GoogleChrome” can never be found in Artifactory when we specify the NuGet package version.

chocolatey_package 'GoogleChrome' do
  version '66.0.3359.18100'
  source 'devops-chocolatey'
  options "--allow-empty-checksums --ignore-package-exit-codes"
end

The following error message is misleading. In this case, the version was in Artifactory, but the spelling of the package ID is not all lower case in Artifactory.

googlechrome not installed. The package was not found with the source(s) listed.

If you specified a particular version and are receiving this message, it is possible that the package name exists but the version does not.
Version: “66.0.3359.18100”
Source(s): “http://artifactory….”

When we internalize the “googlechrome” Chocolatey package, the package ID in the NuGet package will be spelled as “GoogleChrome”. The “internalize” process downloads the NuGet package from the Chocolatey server, downloads the necessary installer files from the source URL specified in the NuGet package, and creates a new NuGet package that contains all necessary files for the software installation. To have access to these packages any time, even when the Chocolatey package is no longer available at Chocolatey.org, we can upload them to Artifactory or other NuGet repository.

To solve the case sensitivity issue, currently, the only solution is to unzip the package, change the spelling of the ID to lower case and re-create the package again.

  1. Install Chocolatey on your Windows workstation. See Install Chocolatey
  2. Open a Windows command prompt as Administrator
  3. Internalize the package
    choco download googlechrome --internalize
  4. Unzip the NuGet package
    1. Right-click the .nupkg file and select 7-Zip, Extract to “…”
    2. Open the .nuspec file in a text editor
    3. Change the ID to all lower case
      <id>googlechrome</id>
    4. Save the .nuspec file
  5. Re-create the NuGet package
    1. Delete the existing _rels folder, the package creation will recreate it with updated information
    2. Right-click the .nuspec file and select Compile Chocolatey Package
  6. Upload the NuGet package to Artifactory.