M2Crypto bad performance to decrypt and verify big email

Question

The input : big multipart signed and encrypted email (~10MB) done with openssl.

Decrypting the file seems to be fast enough.

Getting the decrypted information to verify them is MORE than long. It seems there is some problem in M2Crypto library. If you replace the smime_load_pkcs7_bio call by a file writing the p7s + reading it with smime_load_pkcs7 call it is MUCH faster. But I want to avoid a write/read on disk (as it is my bottleneck by now).

Question : Does anyone have some workaround or solution to this performance problem ?

The python 2.7 code :

from M2Crypto import SMIME, X509, BIO, m2

# read signed and encrypted file
with open("toto.p7m", "r") as p7mFile:
  p7mBio = BIO.File(p7mFile)
  p7m = SMIME.PKCS7(m2.pkcs7_read_bio_der(p7mBio._ptr()), 1)

s = SMIME.SMIME()
# Decrypt
s.load_key('cnt.key', 'cnt.crt', callback = lambda x : 'cnt_password.info')
p7s = s.decrypt(p7m)
print("Decryption ok (and fast).")
# Verify
p7s_bio = BIO.MemoryBuffer(p7s)
p7, data = SMIME.smime_load_pkcs7_bio(p7s_bio)
# MUCH FASTER !!!
#p7, data = SMIME.smime_load_pkcs7('toto.p7s') 
print("Wow this long to load something that is in memory!")
sk =  p7.get0_signers(X509.X509_Stack())
if 0 == len(sk) :
  print("ERROR : No signers.")

s.set_x509_stack(sk)

st = X509.X509_Store()
st.load_info('ca.crt')
s.set_x509_store(st)

v = s.verify(p7, data)
if v:
  print("Client signature verified.")
else:
  print("ERROR : Signature verification FAILED.")

Generate keys/certs

# generate control authority (key+cert)
openssl genrsa -out ca.key 2048 -passout file:ca_password.info
openssl req -x509 -new -nodes -key ca.key -passin file:ca_password.info -days 7300 -sha256 -extensions v3_ca -out ca.crt -subj "/C=XX/ST=Xxxxxx/L=XXXX/O=XXXXX/OU=XXXX/CN=XXX Xxxx XX"
openssl x509 -noout -text -in ca.crt

# generate client key + cert for CNT
openssl genrsa -out cnt.key 2048 -passout file:cnt_password.info
openssl req -new -key cnt.key -out cnt.csr -subj "/C=XX/ST=Xxxxxx/L=XXXX/O=XXXXX/OU=XXXX/CN=CNT xxxx"

echo "authorityKeyIdentifier = keyid,issuer" > cnt.ext
echo "basicConstraints = CA:FALSE" >> cnt.ext
echo "keyUsage = digitalSignature, keyEncipherment" >> cnt.ext
echo "subjectKeyIdentifier = hash" >> cnt.ext

openssl x509 -req -in cnt.csr -passin file:cnt_password.info -CA ca.crt -CAkey ca.key -CAcreateserial -out cnt.crt -days 1024 -extfile cnt.ext
openssl x509 -noout -text -in cnt.crt


# generate client key + cert for ET
openssl genrsa -out et.key 2048 -passout file:et_password.info
openssl req -new -key et.key -out et.csr -subj "/C=XX/ST=Xxxxxx/L=XXXX/O=XXXXX/OU=XXXX/CN=ET xxxx"

echo "authorityKeyIdentifier = keyid,issuer" > et.ext
echo "basicConstraints = CA:FALSE" >> et.ext
echo "keyUsage = digitalSignature, keyEncipherment" >> et.ext
echo "subjectKeyIdentifier = hash" >> et.ext

openssl x509 -req -in et.csr -passin file:et_password.info -CA ca.crt -CAkey ca.key -CAcreateserial -out et.crt -days 1024 -extfile et.ext
openssl x509 -noout -text -in et.crt

Generate test data to go into the email

dd if=/dev/urandom of=sample1.jpg bs=1K count=743
dd if=/dev/urandom of=sample2.jpg bs=1K count=3009
dd if=/dev/urandom of=sample3.xml bs=1K count=5
dd if=/dev/urandom of=sample4.mp4 bs=1K count=2864

Generate the email

The email is generated by some PHP code with the Mail/mime.php library.

<?php
date_default_timezone_set('Europe/Paris');
require_once 'Mail/mime.php';

function add_fichier_2_mail(&$mime, $filename) {
  $finfo = finfo_open(FILEINFO_MIME_TYPE);
  $mime_type=finfo_file($finfo,$filename);
  finfo_close($finfo);
  $mime->addAttachment($filename, $mime_type);
}
$crlf = "\n";
$mime_mif = new Mail_mime($crlf);
add_fichier_2_mail($mime_mif, "sample1.jpg");
add_fichier_2_mail($mime_mif, "sample2.jpg");
add_fichier_2_mail($mime_mif, "sample3.xml");
add_fichier_2_mail($mime_mif, "sample4.mp4");
$body = $mime_mif->get();
$hdrs=array();
$entete = $mime_mif->headers($hdrs);
unset($mime_mif);
$msg='';
foreach ($entete as $key=>$value) {
  $msg.=$key.': '.$value.$crlf;
}
$msg.=$crlf.$crlf.$body;
file_put_contents("email_clear.eml",$msg);
?>

Sign / Encrypt the email

The email is signed by ET private key and then encrypted by CNT public certificate. This gives the following commands :

openssl smime -sign -binary -nodetach -certfile et.crt -signer et.crt -inkey et.key -in email_clear.eml -out email_signed.p7s
openssl smime -encrypt -outform DER -binary -des3 -in email_signed.p7s -out email_crypted_signed.p7m cnt.crt

Answer 1

I was able to confirm this on my machine, but not able to figure out why it's happening. The problem seems to be in the BIO.MemoryBuffer class...

As I commented, the workaround that immediately jumped out to me was to create a RAM disk and use smime_load_pkcs7 against that.

Creating a ram disk on Linux

mkdir -p /mnt/tmpfs
mount -o size=16G -t tmpfs none /mnt/tmpfs

This has two benefits: (1) you avoid the faulty code in M2Crypto and (2) you can avoid having to write to or read from the disk.