What is ASN.1 and how can I read something that’s encoded with “asn.1”? I’m not going to go into too much detail as there’s other online resources that do this. However, I will link to those resources that do a good job of explaining the details of these standards. Also I’m not an expert, this is just what I have learnt so far (some of which may be wrong).

ASN.1 (Abstract Syntax Notation number One)

I wouldn’t get caught up too much on ASN.1, it’s just an IDL and not an extremely useful thing to know unless you were defining some new key structure (which you probably shouldn’t be doing). As the name suggests this actually isn’t an encoding or even a way to encode but an abstract syntax. The easiest way to relate to this is Apache Thrift or Google Protocol Buffers. These technologies themselves don’t offer anything concrete, but they are the frameworks on which to create new encodings and object types. Two common encoding types of ASN.1 for cryptography are DER and PEM. There are various object types such as public keys, private keys, signatures, curve parameters, etc. To learn more about ASN see Introduction to ASN.1​.

Also I know you could use ASN.1 for other things outside of cryptography, but there are IDLs with much better tooling and support like Apache Thrift or Google Protocol Buffers. It’s sort of like saying you can still use SOAP, but why would you when better things exist like JSON API?

DER (Distinguished Encoding Rules)

ASN.1 defines a bunch of different encoding rules like BER, CER, PER, and DER. However, by far the most common is DER. There’s a pretty good write up about DER on MSDN that goes into detail about DER: Distinguished Encoding Rules so I won’t talk about it too much. However, it’s a pretty simple encoding of Tag, Length, Value (or TLV for short). This is a pretty common encoding scheme across computer science to describe nested data. Just off the top of my head I can think of four other places this scheme is used: Apache Thrift, Google Protocol Buffers, Bluetooth Service Discovery Protocol (SDP), and Bluetooth Low Energy Advertisements. I’m sure there are many others as well.

Again MSDN has some good write ups on DER’s TLV encoding scheme:

• TLV
• Tag Encoding
• Length and Value Encoding

PEM (Privacy-enhanced Electronic Mail)

One thing I’ve noticed when learning about cryptography is that a lot of the standards are based and designed around email. Since a lot of web standards rely on text encoding such as JSON and XML, it makes sense to re-use email standards like PEM designed around text encoding. PEM is nothing more than base64 encoded DER with some extra metadata. The extra metadata is literally a string at the start and end to describe the base64 encoded DER such as -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- for certificates.

Putting it all together and playing in bash (on macOS)

Now that I have gone over the basics, we can play around with some commands to solidify this understanding.

• First I created some handy bash commands in my ~/.bash_profile to make what’s going on a little bit easier to read:

In ~/.bash_profile:

#!/usr/bin/env bash
function remove-first-and-last-line {
  sed -e "1d" -e "$ d"
}

function remove-line-endings {
  tr -d \\n
}

function hex-to-bin {
  xxd -r -p -
}

function bin-to-hex {
  hexdump -v -e '/1 "%02X"'
  #OR
  #xxd -p | remove-line-endings
}

function parse-der {
  openssl asn1parse -inform der
}

function parse-pem {
  openssl asn1parse -inform pem
}

• Next, lets create a folder for playing around:

  cd ~
  mkdir crypto-fun
  cd crypto-fun
  

• Lets generate a private key using the prime256v1 curve.

  openssl ecparam -name prime256v1 -genkey -noout -out key.pem
  

• Now we have a elliptic curve private key in PEM format so lets see what it looks like in ASN.1:

  cat key.pem | parse-pem
  

• Given what I described above lets try to manually convert the PEM file to DER to make sure the output is the same:

  # First dump the file contents to stdout with cat key.pem
  # Next remove the PEM metadata lines: -----BEGIN EC PRIVATE KEY----- and -----END EC PRIVATE KEY----- with remove-first-and-last-line
  # Next remove al the line endings to get one full blob of base64 with remove-line-endings
  # Next decode the base64 into binary with base64 --decode
  # Finally save the output to key.der with > key.der
  cat key.pem | remove-first-and-last-line | remove-line-endings | base64 --decode > key.der
  cat key.der | parse-der
  

• Lets even diff them just to make sure

  diff <(cat key.pem | parse-pem) <(cat key.der | parse-der)
  

Extra Information

Note that the above experiment in bash was only to solidify an understanding of ASN.1, PEM and DER. There are much better commands for parsing specific types of cryptographic objects. For instance, objects like x509 Certificates often contain DER encoding inside of a DER encoding. Thus instead you should use specific commands for parsing those objects. Here’s a couple examples (I also save these as functions in my ``~/.bash_profile`):

Parsing elliptic private keys (like key.pem or key.der we created earlier):

cat key.pem | openssl ec -noout -text

# Must specify the input type for OpenSSL as PEM is the default
cat key.der | openssl ec -noout -text -inform der

Parsing elliptic public keys

# First lets create the corresponding public key to our private
cat key.pem | openssl ec -pubout -out pub.pem

# Parse the PEM public key
cat pub.pem | openssl ec -pubin -noout -text

# Now lets use openssl to convert the encoding
cat pub.pem | openssl ec -pubin -pubout -out pub.der -outform der

# Parse the DER public key
cat pub.der | openssl ec -pubin -noout -text -inform der

There’s very similar commands for parsing all types of objects in openssl basically with this same format. There are some exceptions like public keys where you have to specify it’s a public key:

cat file | openssl <object_type> -noout -text -inform <der/pem>

Unfortunately OpenSSL is not the smart to read the context given by PEM and automatically parse that given structure. However, a plus side to that is that it’s symmetrical for parsing PEM and DER files of the same object type.

Final Notes

Hopefully you can parse any PEM or DER file with OpenSSL. You can even deconstruct it down to its rawest form in bytes and dig through the RFC specifications to map the values. I hope if you read this far that this was informative and not a huge waste of time. If it was, well sorry.