I'm currently trying to get a microcontroller running FreeRTOS with WolfSSL working with an x86 server using OpenSSL.

I'm encrypting messages server-side with the OpenSSL EVP Cipher functions, using EVP_aes-256_ctr() as the type:

EVP_CIPHER_CTX_new()
EVP_EncryptInit_ex(ctx, EVP_aes_256_ctr(), NULL, TEST_KEY, TEST_IV)
// EVP_CIPHER_CTX_set_padding(ctx, 0) /* tried adding this for wolfSSL compat */
EVP_EncryptUpdate()
EVP_EncryptFinal_ex()

Now this currently works with OpenSSL on the other side running decryption. However, I'm trying to get decryption working with WolfSSL and I've been having problems. I've currently tried a few approaches (mostly based on the FreeRTOS example code), including:

// attempt with bare AesCtrEncrypt:
byte cipher[AES_BLOCK_SIZE * 8]; // this is ~twice as large as the messages
Aes aes;
wc_AesSetKey(&aes, TEST_KEY, AES_BLOCK_SIZE, TEST_IV, AES_ENCRYPTION);
wc_AesCtrEncrypt(&aes, cipher, msg, out_len);

// attempt with wolfssl EVP Ciphers:
EVP_CIPHER_CTX ctx;
byte plain [AES_BLOCK_SIZE * 12];

EVP_CIPHER_CTX_init(&ctx);
EVP_CipherInit(&ctx, EVP_aes_256_ctr(), TEST_KEY, TEST_IV, 0/*decrypt*/)
EVP_Cipher(&ctx, plain, (byte*)msg, len)

But neither of these two main approaches have worked for decrypting the encrypted bytes the server is sending over. Based on the docs I have no idea what to look for - the page for AesCbcEncrypt specifically mentions not using padding for OpenSSL interoperability, but the page for AesCtrEncrypt makes no mention.

- cbc page: https://www.wolfssl.com/doxygen/group__ … 68dd4b218b
- ctr page: https://www.wolfssl.com/doxygen/group__ … a000fe43a1

Thanks in advance!