For now, I found that I can disable GCM by undefining HAVE_AESGCM in settings.h. This allows the connection to be established successfully using CBC mode instead of GCM mode. Although this allows me to continue development, it feels like a temporary solution that still needs proper resolution.

What I don't know is what is the ramification removing GCM support? What cipher suites are required to be supported, or are typically supported by other TLS client/servers? Based on the fact that GCM was chosen when both CBC and GCM are enabled, I interpret that to mean that GCM is the better choice, but I don't really know.

incremental progress...
I noticed when IV was generated randomly, in AesAuthSetIv(), nonce would be equal to aes->reg, so clearing reg before copying it would clobber the IV that was about to be set.

so in AesAuthSetIv() in ti-aes.c..
I changed

XMEMSET(aes->reg, 0, AES_BLOCK_SIZE)

to...

if(len < 12)
    XMEMSET(aes->reg+len, 0, AES_BLOCK_SIZE-len);

Note that the IF was necessary, otherwise it would fault when IV was larger than 12.. I don't know what the correct thing to do is when IV is larger than 12.. im guessing that simply truncating it isn't the right answer, though.

Regardless, with this change, the GMAC test passes. But the AES-GCM test fails in the portion of test.c that says  /* test with IV != 12 bytes */

For clarity, enabling "hardware encryption" for this platform consists of adding these preprocessor defines to the build:
WOLFSSL_TI_HASH
WOLFSSL_TI_CRYPT
TARGET_IS_SNOWFLAKE_RA2
WOLFSSL_TIRTOS

I am trying to use wolfSSL with a TI TM4C129E, for which there is a port provided with wolfSSL to enable the hardware encryption of the chip. My end goal is to act as an HTTPS server, but for now I am just using example code from TI to connect as client. This fails, with a TLS alert "bac_record_mac." I guess the server is rejecting the MAC that the client sends during or shortly after the Client Hello.

I managed (with some effort) to compile and run "wolfcrypt/test/test.c", which succeeds for all tests when hardware encryption is not enabled, and fails for GMAC and AES-GCM when hardware encryption is enabled. I can tell that this test code is not executed for this platform very frequently, if ever, because it did not work at all until I modified the code in "port/ti-*". Most of the modifications were related to querying the keysize after it was set, copying a hash, etc. When it fails on the GMAC and GCM tests though, it's because it got a different answer from the expected result of the test vectors.

What is the status of support for this hardware? Has it been tested recently? I've been reading up on GCM mode from some NIST documents, but I am still confused about what the difference between GMAC and GCM actually is. Is GMAC just a specific use of GCM?