Topic: Implementing rs256 in an SGX enclave
Hi all,
I'm trying to implement a rs256 signature (for JWTs) using WolfSSL 4.8.0 compiled for an SGX enclave environment. Broadly, the code I implemented will read an array containing a PEM-encoded RSA-4096 private key and convert it to a DER (using `wc_KeyPemToDer`), then decode that into an `RsaKey` object (using `wc_RsaPrivateKeyDecode`), initializes the RNG and a signature buffer, and finally calls `wc_SignatureGenerate`.
The call to `wc_SignatureGenerate` is failing with an error code of -112, which is `MP_EXPTMOD_E`. Recompiling WolfSSL with debug and -O0 and -g, I believe the error originates from `fp_exptmod` where it is checking for "modulus of zero and prevent overflows":
if (fp_iszero(P) || (P->used > (FP_SIZE/2))) {
I believe it is the second test that is failing `(P->used > (FP_SIZE/2))`. In my case, `P->used` is currently set to 64 and `FP_SIZE/2` seems to be equal to 36 (if I am reading the disassembly correctly).
I generated the key I'm testing with using: `openssl req -new -newkey rsa:4096 -nodes -keyout xxx.key -out xxx.csr`, converted to a C include header with xxd. Also happy to post my minimized test code.
Any advice as to debugging this would be greatly appreciated!
Thanks for reading!