dev.lwlx.xyz

GitHub

Twitter

Getting consistent Encryption in Node / PHP / Browser and openSSL


lwlx

29. October 2020

@0x0000005

I recently was tasked to find a solution for encrypting data in different places and to being able to decrypt them all in a browser during runtime.

Surprisingly, it was much more challenging than expected since there was so little documentation around this online. What was available were a few code-only examples, so I was forced to do R & D and try all implementations and compare the in- & outputs.

Given the lack of a decent out-of-the-box solution, I worry that many developers are settling for easy to use, insecure solutions that limit your project’s security and flexibility. Security should be easy to use and accessible.

Clone the repo here to get started with this setup: https://gist.github.com/Lawlez/88e04e3541cc0608c953a118b86bfc1a

Okay, so lets assume we use following input data to test each implementation:

  • key = '5035ae3567f2e69320b083d59a7364cf8d4b14e77d7b798051241ce546b327d9' //must be 256 bits
  • iv = '1d6ef201e0e7a9019ddf8414034325e2' //must be 128 bits
  • inputData = {"TestData":"w17h Spé^cIäl chàær§¢tèrs", "OK":"://seems/fine?x=lol"}

Let's quickly run through and test each implementation:

Using Node JS Crypto module

Node Provides a nice crypto implementation. It's documentation is rather sparse, but this is what most resources suggest by using:

  • crypto.randomBytes()
  • crypto.createCipheriv()
  • crypto.createDecipheriv()
1const crypto = require('crypto')
2
3/**********************************************************************
4*
5*        DECRYPTION MODULE FOR USE INSIDE NODE.JS                     *
6*
7***********************************************************************/
8
9const encryption = (data = 'TestString {} Héllöüä') => {
10
11   const secretPhrase = crypto.randomBytes(16).toString('hex')
12   const salt = crypto.randomBytes(128 / 8).toString('hex')
13   //here we generate the key and give it back as a string, we use 100k iterations
14   //as suggested in best practices
15   //We can use the key multiple times to encrypt multiple things(-30GB), we just cant use
16   //the same initialization vector twice
17   //the key for aes-256 needs to be 256 bits which equals 32 bytes or 32 characters
18   const configKey = crypto.pbkdf2Sync(secretPhrase, salt, 100000, 32, 'sha256').toString('hex').substr(0, 32)
19   //create unique IV for each encryption, the key can be reused. IV needs to always be 16 bytes
20   const IV = crypto.randomBytes(16)
21
22   //create ciphers for each encryption using the shared key and the unuique IV
23   const projectConfigCipher = crypto.createCipheriv('aes-256-cbc', configKey, IV.toString('hex').substr(0,16)
24
25   //encripting the storage location using the prepared cipher
26   const encrypted = Buffer.concat([configStorageCipher.update(
27       'STORAGE', 'utf8'
28   ), configStorageCipher.final()]).toString('hex')
29
30   return encrypted
31
32}

this is NOT the final implementation; check below to see it.

Testing the implementation

We notice that we need to trim the key to 32bytes and the IV to 16 bytes. This is likely because of the conversion from hex to string after the creation of the key.

  • key = '5035ae3567f2e69320b083d59a7364cf' //is now 32 bytes string
  • iv = '1d6ef201e0e7a901' //is now 16 bytes string

This will probably lead to an issue later on since other implementations actually want the longer strings. Maybe we can find a workaround by base64 encoding instead of stringifying the key and iv.

a quick test reveals, yes, we actually can:

1const IV = crypto.randomBytes(16)
2
3console.log(IV)
4// <Buffer c1 1e 98 84 54 eb 85 f6 b3 d0 51 87 d2 62 80 a7>
5
6console.log(IV.toString('base64'))
7// wR6YhFTrhfaz0FGH0mKApw==
8
9console.log(Buffer.from(IV.toString('base64'), 'base64'))
10//<Buffer c1 1e 98 84 54 eb 85 f6 b3 d0 51 87 d2 62 80 a7>

now we could also just create a buffer again from the string to make it 16/32 bytes (ready for usage):

1Buffer.from('5035ae3567f2e69320b083d59a7364cf8d4b14e77d7b798051241ce546b327d9', 'hex')
2//<Buffer 50 35 ae 35 67 f2 e6 93 20 b0 83 d5 9a 73 64 cf 8d 4b 14 e7 7d 7b 79 80 51 24 1c e5 46 b3 27 d9>
3
4Buffer.from('1d6ef201e0e7a9019ddf8414034325e2','hex'))
5//<Buffer 1d 6e f2 01 e0 e7 a9 01 9d df 84 14 03 43 25 e2>

This seems to be a nice solution when you receive the key and iv as an input; if we generate it ourselves, however, it better to just avoid the conversion to string after generation that many people use in examples:

1// type Buffer, 16 bytes
2const IV = crypto.randomBytes(16)
3
4// type Buffer, 32 bytes
5const configKey = crypto.pbkdf2Sync(secretPhrase, salt, 100000, 32, 'sha256')
6

This key and IV pair can be consumed directly by our ciphers, but we would need to convert it to a hex string first to save and forward them.

So what is the output of this function?

The output we received is of type buffer, but when we convert it to a string using toString('hex') we can read the data:

  • OUTPUT = '<Buffer f2 fb 62 b1 7e e9 da 0c 8c bd 56 f2 45 a9 87 60 b4 e2 a6 d0 c5 de f1 50 bc 6d 86 00 f8 5d b4 79>' //is 32 bytes
  • OUTPUT_Stringified = 'f2fb62b17ee9da0c8cbd56f245a98760b4e2a6d0c5def150bc6d8600f85db479' //is now 64 bytes string
  • OUTPUT_Base64 = '8vtisX7p2gyMvVbyRamHYLTiptDF3vFQvG2GAPhdtHk=' //is now 44 bytes string

so using the codes below, we can switch between these three outputs as we like

1//output Buffer
2encrypted = Buffer.concat([encrypted, Cipher.final()]);
3
4//output String
5encrypted = Buffer.concat([encrypted, Cipher.final()]).toString("hex");
6
7//output Base64
8encrypted = Buffer.concat([encrypted, Cipher.final()]).toString("base64");
9
10//revert conversion to base64
11Buffer.from(encrypted.toString("base64"), "base64");

From what we have learned here, I guess the best option is to use the base64 output method since we can easily convert it to a buffer

Using browserify-aes's node crypto-like implementation inside the Browser

Inside the browser, we cannot use Nodes.js built-in modules. Using browserify-aes we can use a node-like crypto implementation, which uses the same syntax as the node implementation. In my use case, I only need to decipher in the browser, this means I don’t have to worry about a truly random key generation or ciphering.

1import crypto from "browserify-aes";
2
3/**********************************************************************
4 *
5 *        DECRYPTION MODULE FOR USE IN BROWSER DURING RUNTIME          *
6 *
7 ***********************************************************************/
8const decrypt = (Base64Hash) => {
9  //we use the base64 hash generated by openssl cli as an input
10  const Base64Hash =
11    "Z8QIo6YuR7DZqmHHV4WqqorUnUZ2n88gMFADMCt2FKUn/ZeYUj1DEBNS2NthignUNR0hw+OOFU7qACKPZbxx8k0Pe0McXNDrOnUtl3dIwdg=";
12  const Key =
13    "5035ae3567f2e69320b083d59a7364cf8d4b14e77d7b798051241ce546b327d9";
14  const IV = "1d6ef201e0e7a9019ddf8414034325e2";
15  const hexToBin = (hex) => {
16    //converts hex strings to binary arr
17    for (var bytes = [], c = 0; c < hex.length; c += 2) {
18      bytes.push(parseInt(hex.substr(c, 2), 16));
19    }
20    return bytes;
21  };
22  //ein neuer cipher wird vorbereitet, mittels aes256, unserem 256 bit KEY und dem config IV
23  const decipher = crypto.createDecipheriv(
24    "aes256",
25    hexToBin(Key),
26    hexToBin(IV)
27  );
28
29  //der hash wird nun decrypted mittels dem zuvor erstellten cipher
30  const decrypted = Buffer.concat([
31    decipher.update(Buffer.from(Base64Hash, "base64")),
32    decipher.final(),
33  ]).toString("utf8");
34
35  return JSON.parse(decrypted);
36};
  • OUTPUT = {TestData: "w17h Spé^cIäl chàær§¢tèrs", OK: "://seems/fine?x=lol"} //yes, thats our original input! :D

ENCRYPTION & DECRYPTION MODULE FOR PHP7+ USING OPENSSL

In PHP 7 we make use of the openssl_encrypt implementation to encrypt an utf8 string and finally encode it with base64_encode. For decryption we also make use of the official openssl implementation openssl_decrypt, before decrypting we need to decode using base64_decode.

1/**********************************************************************
2*
3*        ENCRYPTION & DECRYPTION MODULE FOR PHP7+ USING OPENSSL       *
4*
5***********************************************************************/
6
7class AESEncryption {
8
9   //key length should be 256 bits for aes 256 this means we use a string with 32 bytes
10   public static $key = "5f08e0ec585393a8e2ca8f0a1a0ae752";
11
12    //iv length should be always be 128 bit / 16 bytes
13   public static $iv = "05d387e7f773035a";
14
15    // The AES uses a block size of sixteen octets (128 bits)
16   public static $Method = 'AES-256-CBC';
17
18   /**
19     * use the AES to encrypt plaintext data and return a base 64 string
20    *
21    * $key
22    */
23   public static function encrypt($cleartext,$key = ''){
24
25       $key = empty($key) ? self::$key : $key;
26
27       $encrypted = openssl_encrypt($cleartext, self::$Method, $key, OPENSSL_RAW_DATA, self::$iv);
28
29       return base64_encode($encrypted);
30
31   }
32
33   /**
34     * use the AES to decrypt a base 64 string into plaintext
35    *
36    * $key
37    */
38   public static function decrypt($encrypted,$key = ''){
39
40       $key = empty($key) ? self::$key : $key;
41
42       $encrypted = base64_decode($encrypted);
43
44       $decrypted = openssl_decrypt($encrypted, self::$Method, $key, OPENSSL_RAW_DATA, self::$iv);
45
46       return trim($decrypted);
47   }
48}

while desperately searching for a solution I looked into doing the encryption in PHP instead of openssl, since I scrapped this idea I cannot explain any further. I still keep this PHP 7 example here because its hard to find examples online that don’t use mcrypt.

Using OpenSSL for use in CLI

Inside of a Command Line Interface, we use openssl do en- or decrypt data.

For node/browserify to be able to decrypt it we need to add the -nosalt option, which disables salting the data.

1#########################################################################################
2#                                                                                       #
3#               ENCRYPTION FOR CLI IN / MACOS / LINUX / WINDOWS                         #
4#                                                                                       #
5#########################################################################################
6
7#encrypt with key & IV but no salt
8cat config.json | openssl aes-256-cbc -iv $(cat iv)  -K $(cat key) -A -nosalt -base64
9
10#decrypt with key IV and base64
11echo "encryptedString" | openssl aes-256-cbc -d -iv $(cat iv)  -K $(cat key) -base64 -A

testing the implementation

I created a json file called test.json containing the inputData. so when we run the following command ...

1cat test.json | openssl aes-256-cbc -iv "1d6ef201e0e7a9019ddf8414034325e2"  -K "5035ae3567f2e69320b083d59a7364cf8d4b14e77d7b798051241ce546b327d9" -A -nosalt

We get no warnings and an output like this:

  • OUTPUT = g??.G?٪a?W????ԝFv?? 0P0+v?'???R=CR??a? ?5!??N?"?e?q?M{C??:u-?wH?? //weird looking binary data

as you can see this is not very usefull so we apply the base64 encoding after encryption

1cat test.json | openssl aes-256-cbc -iv "1d6ef201e0e7a9019ddf8414034325e2"  -K "5035ae3567f2e69320b083d59a7364cf8d4b14e77d7b798051241ce546b327d9" -A -nosalt -base64
  • OUTPUT_base64 = Z8QIo6YuR7DZqmHHV4WqqorUnUZ2n88gMFADMCt2FKUn/ZeYUj1DEBNS2NthignUNR0hw+OOFU7qACKPZbxx8k0Pe0McXNDrOnUtl3dIwdg= //now this looks nice

now we can also decrypt the just created data like so

1echo $encryptedData | openssl aes-256-cbc -d -iv "1d6ef201e0e7a9019ddf8414034325e2" -K "5035ae3567f2e69320b083d59a7364cf8d4b14e77d7b798051241ce546b327d9" -A -base64

This yields us this output

  • OUTPUT = {"TestData":"w17h Spé^cIäl chàær§¢tèrs", "OK":"://seems/fine?x=lol"} //yes, thats our original input! :)

final solution

Using NodeJS during initial build

While building our app this code is responsible for:

  • creating a unique key on every build
  • creating a unique iv for every object to be encrypted
  • outputting an encrypted base64 encoded string of data

the output needs to be consumed by either:

  • the application during runtime (browserify implementation)
  • openssl in case of deployment
1const crypto = require("crypto");
2
3/**********************************************************************
4 *
5 *        DECRYPTION MODULE FOR USE INSIDE NODE.JS                     *
6 *
7 ***********************************************************************/
8
9const encryption = (
10  data = { TestData: "w17h Spé^cIäl chàær§¢tèrs", OK: "://seems/fine?x=lol" }
11) => {
12  const secretPhrase = crypto.randomBytes(16).toString("hex");
13  const salt = crypto.randomBytes(128 / 8).toString("hex");
14  //here we generate the key and give it back as a string, we use 100k iterations
15  //as suggested in best practices
16  //We can use the key multiple times to encrypt multiple things(-30GB), we just cant use
17  //the same initialization vector twice
18  //the key for aes-256 needs to be 256 bits which equals 32 bytes or 32 characters it is currently of type Buffer
19  const configKey = crypto.pbkdf2Sync(secretPhrase, salt, 100000, 32, "sha256");
20  //create unique IV for each encryption, the key can be reused. IV needs to always be 16 bytes. it is currently of type buffer
21  const IV = crypto.randomBytes(16);
22
23  //create ciphers for each encryption using the shared key and the unuique IV
24  const projectConfigCipher = crypto.createCipheriv(
25    "aes-256-cbc",
26    configKey,
27    IV
28  );
29  //when using hex strings as IV/keys you can convert it into a buffer to make it work:
30  //Buffer.from('1d6ef201e0e7a9019ddf8414034325e2','hex')
31
32  //encrypting the storage location using the prepared cipher
33  // our input is an object, so we first stringify it and set the input encoding to utf8, for our output we need base64 encoding
34  let encrypted = projectConfigCipher.update(
35    JSON.stringify(data),
36    "utf8",
37    "base64"
38  );
39  // finalize the encryption also with base64 output encoding
40  encrypted += projectConfigCipher.final("base64");
41
42  /***************************************************************
43   * To be able to decrypt later, we need to save the IV and key somewhere.
44   * it is recommended to store the iv together with the encrypted
45   * data, but you should store the key separately. we save those values
46   *  as hex-encoded strings - so the can later be converted into binary again
47   ****************************************************************/
48  const saveKey = key.toString("hex");
49  const saveIV = IV.toString("hex");
50  // the above strings can be directly interpreted by openssl
51  // the above key can be converted to a buffer in node: Buffer.from(saveKey, 'hex')
52  // the above key can be converted to binary using hexToBin() in the browser
53
54  return encrypted;
55};
  • OUTPUT_DATA = Z8QIo6YuR7DZqmHHV4WqqorUnUZ2n88gMFADMCt2FKUSgCV12rE4RpgPdjXMJJB2vNJZ+00LvE9nkn77fW0pf8c/tzW5MxQpzqV3A+HvniM= //look what a nice base64 string
  • OUTPUT_KEY = '5035ae3567f2e69320b083d59a7364cf8d4b14e77d7b798051241ce546b327d9'
  • OUTPUT_IV = '1d6ef201e0e7a9019ddf8414034325e2'

Our output data looks good, so lets test what openssl can do with it!

Using Node.crypto's output as input in openssl

from node, we get the data shown above as files named key and iv In my use case we only need the key and IV to encrypt a new config. I still did include a decryption example as well here.

Things to keep in mind:

  • output encoding must be base64
  • -nosalt option needs to be enabled
  • -A option needs to be enabled
1#########################################################################################
2#                                                                                       #
3#               ENCRYPTION FOR CLI IN / MACOS / LINUX / WINDOWS                         #
4#                                                                                       #
5#########################################################################################
6
7#encrypt with key & IV but no salt
8#config.json contains the testdata defined above
9cat config.json | openssl aes-256-cbc -iv $(cat iv)  -K $(cat key) -A -nosalt -base64
10
11#decrypt with key IV and base64
12# we use the output base64 string from node here
13echo "Z8QIo6YuR7DZqmHHV4WqqorUnUZ2n88gMFADMCt2FKUSgCV12rE4RpgPdjXMJJB2vNJZ+00LvE9nkn77fW0pf8c/tzW5MxQpzqV3A+HvniM=" | openssl aes-256-cbc -d -iv $(cat iv)  -K $(cat key) -base64 -A

Encryption output:

  • OUTPUT_encrypt = Z8QIo6YuR7DZqmHHV4WqqorUnUZ2n88gMFADMCt2FKUn/ZeYUj1DEBNS2NthignUNR0hw+OOFU7qACKPZbxx8k0Pe0McXNDrOnUtl3dIwdg= //looking good..

Decryption output:

  • OUTPUT_decrypt = {"TestData":"w17h Spé^cIäl chàær§¢tèrs","OK":"://seems/fine?x=lol"} //nice, thats our original data!

Since our Output looks good and even the decryption worked fine, lets test what our browserify implementation can do with this.

Using browserify to decrypt node or openssl input

This code expects the following input:

  • base64 encoded string to decrypt
  • iv in the form of a hex-encoded string
  • key in the form of a hex-encoded string

I get the keys from process.env in this example. You could also receive them via input or even a file.

This code needs to be able to produce consistent output when receiving input from either node or openssl.

We had to create a custom function hexToBin() to convert a hex string into a binary array to be consumed by our cipher.

1import crypto from "browserify-aes";
2
3/**********************************************************************
4 *
5 *        DECRYPTION MODULE FOR USE IN BROWSER DURING RUNTIME          *
6 *
7 ***********************************************************************/
8const decrypt = (Base64Hash) => {
9  const Key = process.env.APP_KEY; //hex encoded string
10  const IV = process.env.APP_IV; //hex encoded string
11
12  const hexToBin = (hex) => {
13    //converts hex strings to binary arr
14    for (var bytes = [], c = 0; c < hex.length; c += 2) {
15      bytes.push(parseInt(hex.substr(c, 2), 16));
16    }
17    return bytes;
18  };
19
20  //ein neuer cipher wird vorbereitet, mittels aes256, unserem 256 bit KEY und dem config IV
21  const decipher = crypto.createDecipheriv(
22    "aes256",
23    hexToBin(Key),
24    hexToBin(IV)
25  );
26
27  //der hash wird nun decrypted mittels dem zuvor erstellten cipher
28  const decrypted = Buffer.concat([
29    decipher.update(Buffer.from(Base64Hash, "base64")),
30    decipher.final(),
31  ]).toString("utf8");
32
33  return JSON.parse(decrypted);
34};
  • OUTPUT_openssl = {TestData: "w17h Spé^cIäl chàær§¢tèrs", OK: "://seems/fine?x=lol"} //yes, thats our original input! :D
  • OUTPUT_nodejs = {TestData: "w17h Spé^cIäl chàær§¢tèrs", OK: "://seems/fine?x=lol"} //and we even handled the node version! sick!

That's it! We did it yay!

Feel free to comment and discuss on my gist: This project on gist

© lwlx. 2021

Version 0.6.1