Why can't openssl read an ssh private key created by openssh on OSX

Question

Here is a test script I am using to help debug an issue with openssl &/or ssh on OSX Mojave 10.14.5 with brew installed versions of openssl and openssh

> brew info openssh | head -1
stable 8.0p1 (bottled)
> brew info openssl | head -1
stable 1.0.2r (bottled) [keg-only]
> ssh -V
OpenSSH_7.9p1, LibreSSL 2.7.3
> openssl version
LibreSSL 2.6.5
> ! test -f /tmp/foo || rm /tmp/foo && 
  ssh-keygen -f /tmp/foo -t rsa -P "" -N "" && 
  openssl rsa -in /tmp/foo 
Generating public/private rsa key pair.
Your identification has been saved in /tmp/foo.
Your public key has been saved in /tmp/foo.pub.
The key fingerprint is:
SHA256:iZMoPkGh4wkPvMOfV5KSEVFOLc9Dc8zmBvbhdE4d+Rs [email protected]
The key's randomart image is:
+---[RSA 2048]----+
|  .ooo. o   ..o  |
|.. .+. * B o o   |
|=... .* X =   .  |
|+=o o..* * .   E |
| *+o.o+.S       o|
| .ooo o.       . |
|  oo .           |
|   ..            |
|                 |
+----[SHA256]-----+
unable to load Private Key
4643780204:error:09FFF06C:PEM routines:CRYPTO_internal:no start line:/BuildRoot/Library/Caches/com.apple.xbs/Sources/libressl/libressl-22.260.1/libressl-2.6/crypto/pem/pem_lib.c:683:Expecting: ANY PRIVATE KEY

The key is similar to this (no, this is not a key I will use):

-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABFwAAAAdzc2gtcn
NhAAAAAwEAAQAAAQEA3FGfR9sL6p0HWfq0UWUNQmFszkqipVMsM3J23oJjcvnJJIx72X3F
9xbBZQ+6JuUpD6ME32DE3aFGmsrkDa+cE/wD6lj5ey9ORTCFri3QM57sFwQhXl48hvpXF/
zHXpVJv/BdtiQFJwTuEbkcNfHDvyK7WIJjuS1kF+v2dX/PypzhszoZmEB9vK1s9bMEyclb
toXwWnoMW/bRmv484adIM5G+LhRF5nvtWSu0oRRF0KKpogCvfWyckKCcDQH9/DNwtXSmzE
cVUmUdESlUNrD8tJdI9+G/hldym1zjB0q2ai/97y8Y8OG7mxkpJNEYw/oxFc7g1BEenQf0
6GzPWQfIiwAAA9AmUmlFJlJpRQAAAAdzc2gtcnNhAAABAQDcUZ9H2wvqnQdZ+rRRZQ1CYW
zOSqKlUywzcnbegmNy+ckkjHvZfcX3FsFlD7om5SkPowTfYMTdoUaayuQNr5wT/APqWPl7
L05FMIWuLdAznuwXBCFeXjyG+lcX/MdelUm/8F22JAUnBO4RuRw18cO/IrtYgmO5LWQX6/
Z1f8/KnOGzOhmYQH28rWz1swTJyVu2hfBaegxb9tGa/jzhp0gzkb4uFEXme+1ZK7ShFEXQ
oqmiAK99bJyQoJwNAf38M3C1dKbMRxVSZR0RKVQ2sPy0l0j34b+GV3KbXOMHSrZqL/3vLx
jw4bubGSkk0RjD+jEVzuDUER6dB/TobM9ZB8iLAAAAAwEAAQAAAQBzm2zeEqXlHTLfVztJ
PqI/g8nJUcaYw9T8xgJz7a1rhoCyafkO/f1kE4+1jRQcFsF+EAedgzSqK1dWIEKcn9phbi
tLzBZVOlRy3+w1opqOi8TMqwEreH2AQlpzHtQq4GFLk0BJNAt0FxUpPZ38/Hi/keUGo5za
bWQJXWr86u1JHiGA9D6XQU4/KrsJ4UqSvy3QGyZa+ypJII9mV9deK50pgHVlX6S/+8FubC
c9okRiQAhfAdGMYzMgoE4auRZ5rsZkVT4DCOagetuug/Hk9oh1JbkEazEKz3w1SFm5lJVF
/A6/MNPT52RCQtEDAM7MrWYVP2XIaTw89R5ujMPebQNhAAAAgGbNxMyIXrul5FXJvsfHnh
mtfa4DZDIQ4fQ3c79NwhxS7RdSVRO3gzxbmGOQJfFWC73z/zvbMFlpKp17lGz4GWiBm9x8
3OS5ohRL2S5VmLMSMb2zNjyvYr7uK9E0WLt19Imo6rlOkw81mJ90GcSVx/byHXnu5bSjtD
2o5uyKURL/AAAAgQD6xwksyTYXC8BuvmiKxs/RbRUKvsrCNCWccBqndm6vfq8LrNoLD920
d+VXobe2sW/JyaknG82sutUPKhTwEhezKcZ2dqce/yJ/+y5R29fd/AuRFVfZOHPTy6hKKZ
HpJRSDHsZZkA74OYsz7ZLMNnqEa2fh39isO51gl1qOp5gY+wAAAIEA4Ogz75bUdsCUxvmk
iJ3vmROTIxEbNtueflwrBzgG9LG3lLyM4eAAYPz8RAvGV3MySgO5vY+7e0/Nkdn90TsR75
ALpChIWnwSyH2OFP717vu8dFtufIs9mzAPPt+JNj6RnzE3nY23xtDBi1/6xhdbbiTi4+Nd
2Woyp8N1QgvqGbEAAAAYam9uX3Vwb3dyQGdyZXl3ZWRnZTMubGFuAQID
-----END OPENSSH PRIVATE KEY-----

Replacing OPENSSH with RSA in the generated private key has no effect on the success of the exercise.

If I try the same sequence of operations on a debian docker image, the key generated by ssh can successfully read by openssl running on the same platform.

Is my expectation that this would also work on OSX unreasonable? If so, what's gone wrong?

edit: I had a spurious -o option in the example which I have now removed.

Answer 1

The problem was that the default behaviour of ssh-keygen on OSX Mojave now differs from that on Linux. In particular, ssh-keygen will produce OPENSSH private keys by default on OSX but RSA private keys by default on Linux.

The same behaviour can be guaranteed in both environments by adding -m PEM to the ssh-keygen arguments.

Thanks to James K Polk for guiding me in the correct direction and also this answer.