SSH Usage and best practices

SSH is the most common protocol for command-line remote access. It exists since 1995 and obviously has evolved a lot over time. If you want to access one of our Unix servers and soon Windows, it's probably the tool that you will use.

As we review our internal security posture, we are enforcing settings more in line with current landscape. Please review the following guidelines to ensure your access is not disrupted.

Windows 10 Insider: integrated SSH server

As far as I can remember, Microsoft Windows only had a Telnet server integrated, if we don’t count recent PowerShell Remoting (using Windows Remote Management, or WinRM). Still, it wasn’t SSH. Since 14352, people were finding traces, of what proved to be an integrated Microsoft SSH Server for Windows. Because Microsoft is also contributing to OpenSSH for Windows project, and that it’s getting to a fairly usable stage, I found the inclusion of the service quite interesting.

SSH server on Windows? Are you insane?

No.

First, I’ve been using SSH server on my Windows machines for quite some time now. It has always been a Cygwin-based solution, though. This SSH and SFTP server allowed me to access my files remotely and do some basic work with them, as well as launch some command-line apps. (I wasn’t fan of launching Windows apps through CLI, but it did prove useful at times.

Last year, a bomb was dropped by a PowerShell team in Microsoft, where they announced that they are going to adopt OpenSSH and modify it so that it can run on Windows and use Windows crypto. And, of course, they wanted to push the changes back upstream. This was a major announcement because, up to that point, Microsoft management always refused to allow SSH work, for one reason or another.

As the year went, PowerShell team posted another update with a nice large slice of completed work and first public release. Since then, there have been more releases and precompiled binaries for all releases are available at their Github releases page. Both key-based and password-based (interactive) auth is supported and the terminal works somehow. SFTP also works.

Twist: Microsoft’s own SSH server

But, a Microsoft’s own SSH server has made a quiet appearance in 14352, where two new services popped up in services.msc:

SSH Server Broker (SshBroker) and SSH Server Proxy (SshProxy)

Surprise? Depends on who you ask. It was a surprise for me, for reasons that I’ve outlined above, but splipstream/RoL on Twitter actually pointed out that SSH/SFTP server is present in both Windows 10 Mobile(in developer mode) and Windows 10 IoT Core for quite some time now.

What’s more: There’s a Microsoft guide for using SSH in Windows IoT and unofficial guide for accessing SFTP on Windows 10 Mobile on XDA, because the SSH server was apparently used to activate Android subsystem on Windows, among other things.

But the SSH server in full Windows 10 is a bit different. Mainly because you can play with the settings.

What can it do?

It can provide you with a cmd.exe. But that’s not all, folks!

First: why two services?

• The broker essentially serves as the SSH server and proxy provides this server to the world by binding it to a port 22.
• So maybe it can be used locally in another fashion.

Second: how do I log in?

• With your user name and password
  • Then, you cannot actually run anything, since it gets shot down right after launch. God MS knows why.
  • Because of that, it has (more or less) capabilities of the good old restore command prompt in Windows XP installer.
• With DevToolsUser
  • This is a special user that logs in as the last logged in user
  • I think that some dev tools from MS use this account internally, but so far I have to see a tool that uses it. I would guess it’s used for remote debugging and/or deployment
  • A PIN can be added in the registry, which allows you to log in with that PIN
  • This account gets the rights of the currently logged in user, and allows you to run everything under that user. (UAC still applies, and you can even run desktop apps, as it will, literally, log you in under the last logged-in account - not the one that was last used!)

You can actually take a look for yourself: the registry key is HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Ssh. When you then use Process Monitor from Sysinternals, you can exactly see which keys the services access.

Process Monitor v3.40

• 
• 
  

Download Process Monitor (981 KB)
Run now from Sysinternals Live.

Introduction

Process Monitor is an advanced monitoring tool for Windows that shows real-time file system, Registry and process/thread activity. It combines the features of two legacy Sysinternals utilities, Filemon and Regmon, and adds an extensive list of enhancements including rich and non-destructive filtering, comprehensive event properties such session IDs and user names, reliable process information, full thread stacks with integrated symbol support for each operation, simultaneous logging to a file, and much more. Its uniquely powerful features will make Process Monitor a core utility in your system troubleshooting and malware hunting toolkit.

Overview of Process Monitor Capabilities

Process Monitor includes powerful monitoring and filtering capabilities, including:

• More data captured for operation input and output parameters
• Non-destructive filters allow you to set filters without losing data
• Capture of thread stacks for each operation make it possible in many cases to identify the root cause of an operation
• Reliable capture of process details, including image path, command line, user and session ID
• Configurable and moveable columns for any event property
• Filters can be set for any data field, including fields not configured as columns
• Advanced logging architecture scales to tens of millions of captured events and gigabytes of log data
• Process tree tool shows relationship of all processes referenced in a trace
• Native log format preserves all data for loading in a different Process Monitor instance
• Process tooltip for easy viewing of process image information
• Detail tooltip allows convenient access to formatted data that doesn't fit in the column
• Cancellable search
• Boot time logging of all operations

The best way to become familiar with Process Monitor's features is to read through the help file and then visit each of its menu items and options on a live system.

Screenshots

Third: how do I transfer files?

• You don’t. Actually, you do, through SFTP. Midnight Commander was able to connect. For some reason, WinSCP is throwing an error message.
  • By default, the Microsoft SSH server does not run runs SFTP. I guess it can be enabled somehow, sincethe Windows 10 Mobile also includes SFTP server. There are definitely SFTP registry entries that the SSH broker tries to access. Some of them modify permissions, much like permissions for regular FTP.

Microsoft SFTP server

Is it better than the OpenSSH for Windows?

I think not:

1. it has obviously a very special purpose and it’s clearly not intended for an ordinary user
2. the configuration options are very limited and most of them is inaccessible (you need to modify ownership and permissions on the registry keys). Obviously, you can have a list of possible login options, so you don’t have to allow password login.

On the other hand,

1. the cmd.exe kinda doesn’t work correctly in the OpenSSH server (you’ll terribly miss arrows and Tab, for instance)
2. one interesting thing, and I should maybe add -

!! A word of caution !!

OpenSSH for Windows will not respect the UAC! When you log in under an administrator account, you can even delete system files and the apps will run with the maximum rights that your user theoretically has!

Using OpenSSH for Windows, you can run non-interactive command-line apps and get output from them, and you can also try to run apps with UI. These will run, but without any window or anything shown on the screen! I found it interesting that even my account did not have access to the processes running under my account! And yet, these apps running under sshd had all the privileges any hacker could wish for. Let’s show a couple of shots:

First, procexp without any special rights

Then, procexp run under administrator

Inspecting a regular process under my account (with UAC)

Inspecting a process running under sshd, with a looooot of rights

But, also, because this is OpenSSH, you have a lot more options to protect your account, for example with a keyfile, and disable password logon as an example. (So many articles have been written about the correct settings, but I’m not posting it since I haven’t tried it yet.) So, be careful, but definitely experiment, you’ll never know where it leads you!

Can I run Bash on OpenSSH for Win?

Definitely not under admin :)

But… I have been able to run Bash under the DevToolsUser!

I can run more tests if there’s a demand, just write below!

Cheers!

SSH Client

· Please ensure you are using up to date SSH clients. Same for any development or deployment tools using SSH. Latest Crypto should be supported like Elliptic curves or Chacha20.

For Windows users

o with Putty, current version is 0.70

o mRemoteNG current is v1.75.7011

o Tutty ssh client is not maintained and should NOT be used

o SecureCRT (commercial) is 8.1.4

For Mac users, command line openssh should be fine if latest Apple updates are done correctly (OpenSSH 7.4 on Macos 10.12.6)

· You SHOULD use SSH Protocol 2. v1 is deprecated and vulnerable.

o Open PuTTY. On the left side, go to Connection->SSH. Select “2 only.” On the left side, select Session (at the top). Highlight “Default Settings.” Click Save.

· Tunneling and ports forwarding will be disabled by default and explicit group assignment will be necessary to add them.

SSH key best practices

We recommend you to use ssh key. Password access will probably be disabled in coming year.

· If your key is years old, please regenerate one.

· Always use a passphrase on your ssh key. Only exception is automation keys.

· Use a password manager to store your passphrase

· Depending on type of key, enforce minimum bits. for RSA, it should be 2048 bits at minimum. Do not use DSA (deprecated since Aug 2015)

· Comment should include username (from ActiveDirectory), hostname and date of where/when key is created. Format should be "user@host-date". This is especially important for generic/automated account keys. Security team is monitoring ssh key usage and should be able to identify team owner quickly else access might be disabled.

· Automation keys should be use with a command restriction on the server meaning key can only be used for a specific command like rsync. Ideally, it is also restricted by subnet.

· Try to use different key per sensitivity level if not per host. For example, one usual key and another for finance servers.

· Never share your private key! Normally, it should never move or copy from its origin system. Just create a new one if needed.

On Windows

Use puttygen to generate key. As said above, if using RSA key type, ensure minimum bit size of 2048.

On MacOS/Unix

$ ssh-keygen -t rsa -b 2048 -C $USER@`hostname -s`-`date "+%Y%m%d"`

Generating public/private rsa key pair. 

Enter file in which to save the key (/Users/myusername/.ssh/id_rsa): 

Enter passphrase (empty for no passphrase):  

Enter same passphrase again:  

Your identification has been saved in /Users/myusername/.ssh/id_rsa. 

Your public key has been saved in /Users/myusername/.ssh/id_rsa.pub. 

The key fingerprint is: SHA256:ZVmKSDUJwb9JPxz54LwCkoH+TlYcTBeVP7efr6cOFus myusername@myhostname-20170912 

The key's randomart image is: 

+---[RSA 2048]----+ 

|     .=+=+...    | 

|     +.o.o.+     | 

|   .  +.. =o     | 

|  . .. .oo+ o .  | 

| .   oo.SB +.o . | 

|  . o.. o * .o.  | 

|   .o. .   o+  ..| 

|   o.   . .o . .o| 

|   ..    .  E.++o| 

+----[SHA256]-----+ 

$ ssh-keygen -t ed25519 -C $USER@`hostname -s`-`date "+%Y%m%d"` -f ~/.ssh/id_ed25519-test 

Generating public/private ed25519 key pair. 

Enter passphrase (empty for no passphrase):  

Enter same passphrase again:  

Your identification has been saved in /Users/myusername/.ssh/id_ed25519-test. 

Your public key has been saved in /Users/myusername/.ssh/id_ed25519-test.pub. 

The key fingerprint is: SHA256:bO4cp3dpyb3ISua3V0HEg5viq025r4aXPb3ZjfKXq5s myusername@myhostname-20170912 

The key's randomart image is: 

+--[ED25519 256]--+ 

|              +. | 

|             . + | 

|              + .| 

|       .   . o . | 

|        S . .   .| 

|       o   ..   .|

|        o =+++...| 

|       o O+*X+++=| 

|        +o*O*EB*=| 

+----[SHA256]-----+

You can copy the key to your server with the following command

ssh-copy-id -i ~/.ssh/mykey user@host

Usage

· Use ssh-agent or putty-agent to avoid typing your passphrase too often

$ ssh-agent 

$ ssh-add ~/.ssh/id_rsa 

Enter passphrase for /Users/myusername/.ssh/id_rsa:  

Identity added: /Users/myusername/.ssh/id_rsa (/Users/myusername/.ssh/id_rsa)

$ ssh host

· Use ProxyJump option for Bastion / Jump host. ProxyJump allows jumphost chaining if needed. ProxyJump requires OpenSSH 7.3+ else use ProxyCommand

$ cat ~/.ssh/config 

Host mytarget

   Hostname target_fqdn_or_IP

   User target_user

   ProxyJump jumpuser1@jumphost1

· If using Keepass password manager, you can integrate ssh key and passphrase inside of it with KeeAgent plugin.

· It's recommended to unload ssh keys before putting system on sleep. Same is also valid for Password Managers.

o On Windows, you can use a scheduled task linked to EventID 1074

o On Macos, you can use sleepwatcher and ssh-add -D