I have been dabbling with OpenSSL commands to achieve what I needed during IT implementation, but I decided to spent some time to overcome the conceptual hurdles around cryptography. In this domain, following other people’s instructions through the project does not produce much learning value when too many concepts cloud around. Let’s take the bull by the horn.
This article is purely conceptual. There are already lots of step-by-step guideline about acquiring a website certificate. The intention is to elucidate the core concepts on IT cryptography, and then connect the dots to form the big picture in cryptography.
First, let’s distinguish three basic concepts: encoding, hashing and encryption. They are in essence all mathematical functions, but one does not need to understand the underlying algorithm in order to understand what they are.
Encoding – Transform data into a format so it is readable by external system. Encoding is about interoperability. It is not about security whatsoever. Example: ASCII, BASE64, UNICODE.
Hashing – Mathematic algorithms to generate digest of content. A digest is usually fixed-size, non-reversible and deterministic. You cannot restore content from its digest (non-reversible, one-way calculation). Two different contents results guarantees different digests (deterministic). Digest as a result of hashing is mostly about data integrity. For example, in file download you can calculate MD5 hash and compare the result against the digest given by the source. Another common use, is to hash all password in database. In that sense, hashing has to do with security. Popular algorithms are MD5 and SHA-256.
Encryption – Mathematic algorithms that only succeeds if correct parameter (key) is provided. The function is deterministic and reversible (two-way). The parameter (key) used for calculation (encryption) and reverse calculation (decryption) can either be the same or different.
Cryptography involves all three concepts, but it mainly addresses issues around encryption. We break down encryption into two categories, symmetric key encryption (aka private key encryption) and asymmetric key encryption (aka public key encryption).
Symmetric Key Encryption: using the same key (shared secret) for encryption and decryption. Both parties need to keep it secret. Popular algorithm is AES.
Asymmetric Key Encryption: involves a pair of public key and private key. Public key is given out to external systems. Private key is kept secret. Popular algorithm is RSA. You can run some experiments on this page, and I’ve made some additional notes (all based on RSA algorithm):
- Public Key and Private Key are NOT interchangeable. The size of an RSA private key is usually much larger than its public key;
- You can encrypt with either key, and decrypt with the other, so long as you specify the key type at the time of encryption or decryption;
- You can generate public key from private key (ssh-keygen -y); but not the other way round;
- Using a wrong key to encrypt or decrypt leads to failure, instead of wrong result.
Obviously symmetric key encryption is the original form of encryption and requires both parties to keep the key secret, permanently. This is not realistic in real life between organizations. This challenge leads to the adoption of asymmetric key where the public key can be published, for external party to encrypt outgoing messages, whereas the private key is kept secret within the owner, only to decrypt incoming messages.
Knowing this distinction, we can introduce two concepts that are built on top of asymmetric key encryption.
Digital Signature: if an entity signs a document digitally. The digital signature is the digest of the document (hash of the content) encrypted with the signer’s private key.
Digital Certificate: contains owner’s public key and digital signature of issuer (digest of owner’s public key encrypted by issuer’s private key). Client (e.g. browser with CA’s public key preloaded) should not trust the owner’s public key until it compares its digest against decryption result of digital signature (using CA’s public key).
The key difference here is that a digital certificate involves third party. Digital signature by itself cannot address impersonation, which is addressed by digital certificate issued by third party (certified authority). This requires digital certificate must follow some standard, and the most prevalent one is:
X.509: a standard format for digital certificates. It contains:
- a public key
- an identity (a hostname, or an organization, or an individual)
- a signature (either signed by CA or self-signed)
Since the advent of certificate, there are tons of global organizations that need to manage keys and certificates. When we create a secure connection now, we only need to get a certificate from an intermediate CA, thanks to existing certificate chain. It involves lots of work for large organizations to maintain keys and certificates, which requires:
Public Key Infrastructure (PKI): the IT infrastructure to create, manage, distribute, use, store and revoke digital certificates and public keys.
One of the important technique to manage keys and certificates is the encoding. Keys and certificates are usually wrapped with different encoding formats based on what need to be done on them. Here is a summary of encoding formats
| Format | Encoding | what is stored | Possible Suffix |
| PEM | DER file (binary content) encoded in Base64. Certificate files typically include clear text statement “BEGIN CERTIFICATE” and “END CERTIFICATE” | single certificate, certificate chains or private keys | .pem, .crt, .cer, .key |
| DER | Binary format in early days; it’s complex and don’t use unless with a specific purpose. | single certificate, certificate chains, or private keys | .der, .cer |
| PKCS#7 | Base64 encoded ASCII file, typically include clear text statement “BEGIN PKCS7” and “END PKCS7” | only certificates or certificate chains; no private keys | .p7b .p7s |
| PKCS#8 | Similar to PEM as base64 encoded format but for storing private key only, can be password protected | Private key | .key |
| PKCS#12 | a binary format, heavily used by Microsoft products | certificate, certificate chains or private keys; public private key pair | .pfx .p12 |
| OpenSSH | used by OpenSSH to store public keys (as specified in RFC4253) | .pub |
Note: above are just encoding formats. Just by file extension you cannot tell whether the file is a key or a X.509 certificate. When you are configuring certificates, you may come across the following file extensions as well:
.key this extension can indicate any kind of key, but usually it is a private key (used along with .crt file)
.csr certificate signing request, including a public key and an identity required by CA. CA needs this file to issue a certificate. CSR could be encoded in Base-64 or DER
.cer or .crt a certificate, usually in X.509 v3 (public key + identity + signature), the encoding could be PEM or DER.
.jks -> java key store file type. It can be either a key store (private key along with certificate) or trust store (certificate) for Java application. Refer to the section for Java applications.
In the next article, we will examine some use case involving the concepts introduced in this post.