Puppet: (Server 8) Using binary data in stored in hiera results in "Error 500 on SERVER"

Question

Our site is migrating from puppet6 to puppet8, and we've encountered an issue with storing binary data in hiera.

The original (binary) data file was encrypted with eyaml:

$ eyaml encrypt -f binary.data > binary.data.eyaml

and the resulted eyaml file was included in a Hiera yaml file as:

test::func::data: >
    ENC[PKCS7,.....
    ...]

When using this code:

test::func{ 'hello':
     data    =>  lookup('binary::data')
}

with:

define test::func(
  Binary $data
) {
}

results in an error:

Test::Func[hello]: parameter 'data' expects a Binary value, got String

however using type "String" (which works on puppet6)

define test::func(
  String $data
) {
}

results in:

Error: Could not retrieve catalog from remote server: Error 500 on
SERVER: Server Error: Failed to serialize Puppet::Resource::Catalog
for 'puppetserver': Could not render to
Puppet::Network::Format[rich_data_json]: source sequence is
illegal/malformed utf-8

This is a behavior change moving from 6 to 8.

Any suggestions as to how to get around this issue?

Answer 1

The YAML hiera backend only supports basic types, and Binary type value is created with a base64 encoded string.

function Binary.new(
  String $base64_str,
  Optional[Base64Format] $format
)

1. Configure hiera to cast specific values with the required type using lookup_options.
2. Encode the data with base64.
3. Encrypt the base64 value.

Based on your use case, in the "common" control-repo hiera file add the following to ensure the lookup value is returned as Binary:

lookup_options:
  binary::data:
    convert_to: Binary

Then base64 encode the binary data and encrypt it:

base64 -w0 < binary.data | eyaml encrypt -l binary::data --stdin

The output should look something like:

binary::data: ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBADAFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEA2/1rNGJbO10rQvPYq2aiLCzeIXVdx/ZAzD+mFJ/IM8i4QYgiW0EwVPpvqwJUiy9iyJ3yd3lD8PJLS6c3/22ta5LVOR50eKRUPGSQuZ7fYGfdHpCwTH/kWz2omJrr+sm2ChW2YrUrTVninz6BhBjCBzSWlK9HJmejX4uuZdTu6YZn5LU123hyy9kW1mmtDCCjNfH3dM8K3EQ38ffv55hG4cIOHGOIuxV9J0LS9AAtuOB4sKMmh5u0/ZO6HyOUz5OniniKqZZsRMI3UGYpOqA0A9HxAcmFsySBSVsSwncYfZR3cEKFblfHqGorPJAccNj78XPrWOi5AxjcyIleQwqmPzA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBBk3e5h2VXAjGs5veW3o4EkgBD9bpPwo/f9bPGKyL9Ufvv9]

OR

binary::data: >
  ENC[PKCS7,MIIBeQYJKoZIhvcNAQcDoIIBajCCAWYCAQAxggEhMIIBHQIBAD
  AFMAACAQEwDQYJKoZIhvcNAQEBBQAEggEA2/1rNGJbO10rQvPYq2aiLCzeIX
  Vdx/ZAzD+mFJ/IM8i4QYgiW0EwVPpvqwJUiy9iyJ3yd3lD8PJLS6c3/22ta5
  LVOR50eKRUPGSQuZ7fYGfdHpCwTH/kWz2omJrr+sm2ChW2YrUrTVninz6BhB
  jCBzSWlK9HJmejX4uuZdTu6YZn5LU123hyy9kW1mmtDCCjNfH3dM8K3EQ38f
  fv55hG4cIOHGOIuxV9J0LS9AAtuOB4sKMmh5u0/ZO6HyOUz5OniniKqZZsRM
  I3UGYpOqA0A9HxAcmFsySBSVsSwncYfZR3cEKFblfHqGorPJAccNj78XPrWO
  i5AxjcyIleQwqmPzA8BgkqhkiG9w0BBwEwHQYJYIZIAWUDBAEqBBBk3e5h2V
  XAjGs5veW3o4EkgBD9bpPwo/f9bPGKyL9Ufvv9]

Copy and paste one of those into your hiera yaml file.

Note: if you interpolate a Binary variable's value in puppet code as a string, it will return the base64 value (e.g. notify { "${binary_var}": })