Thanks Chris. I may be doing something else wrong then. Here are some more details.

When I use 2048 bit DH parameters (example 1 below), I get everything working as I expect even when using 4096 bit RSA (i.e. I think the FP_MAX_BITS thing is correctly set).

However when I use 4096 bit DH parameters (example 2 below), I get "error = -228, malformed buffer input error" when calling SSL_connect() on the client and "error = 6, peer sent close notify alert" when calling SSL_accept on the server (I assume the server is just choking because the client saw something it didn't like and gave up).

This is the call I use to load the DH parameters:

return CyaSSL_CTX_SetTmpDH_buffer
      (ctx, (byte*) dhp, strlen(dhp), SSL_FILETYPE_PEM) == SSL_SUCCESS ? APP_OK : APP_ERR;

Where dhp is either the string from example 1 or example 2. Example 1 works fine, Example 2 gives me error -228 on the client side of the connection and error 6 on the server.

Any thoughts? I'm using DHE-RSA-AES256-GCM-SHA384 with 4096 bit RSA in case that affects anything, and aside from getting errors when I use DH parameters over 2048 bits everything else works exactly as documented/expected. The DH parameters are generated using the openssl command line tool so I am assuming that they are not actually malformed, but I guess I could be wrong?

Thanks for your time,
Demetri

Example 1:
-----BEGIN DH PARAMETERS-----
MIIBCAKCAQEAqqvsxDHhR8uBPQdV0zKZSEMK/L1qe2kdqzmrbxMCFD22gzXy1Dy7
NDZ0FFpxxBkzS8YDvjzvrnT1fQsGe5WcWc10SoFkijKwgOq/9eBJoHLlOWUKxa5J
2Gkx/p0W9hgxTlmy0dpakgl5qenlxvg3CV/RQ9PIR7H6YSec3FeDd1O04fAiqDxu
/Cczg/Tx1MR4gbg1207L21c8UfKXvESqKG9U+krP0vGHPkFilzViOXeeGTcyC+5l
U+2tT796l8PfKTxwSBkI3z1CCzixUS4LnGK3bumC9+zwxEwduid6fd8XxcdD4ccS
Qf8qo3ibOX+2RoVEoUJmMf4mZ0AEBo6NWwIBAg==
-----END DH PARAMETERS-----

Example 2:
-----BEGIN DH PARAMETERS-----
MIICCAKCAgEAoi4K45/SRhfrlSWOXdEYdvRQoUR3QyNQVnh7idW68xP6hXhPFmlP
Gszm1rlY1lwtGkojNU1yLcQxvqeGYBd3CVtzwOzRwnMn+Y/gmXgRfPpGVy8Fm94+
rioJCRqZZ25UP09rQCgEJyrZnEExNvVIClPhrj2a8wDbqQW21JDX/3TCRxcqqYyz
mMxreeacw+Gw+6xrkYbVFhxytT/EiW3t+Dm5is9III2i3fSAM70sEl9I+M2uqMIh
v0PRiIF/l+/uvRoN/7jJLQOxCqDGpDYYozde3dxpdGXPQBClx60j6rPoyT1/6R8M
pW8pzWfJpywBCJwRH8LKgOibINVshtVEHfV9zFIo3+E/7AEYhZje3E9XICZc/NIj
KAeyyMKyvJx5UMRqeiohRRN+gDz46e/ZoWIELbYcH2mBUkVJVbJ2oF2H2whS3fk3
u5sCZ39Xp+sE5qWTkPd1UvPrb/zVfY4ADKuN11INDNiQT4f3uW4+qkgONjTAtQD2
qI4b2zA4zRZ4wso3ssbo/WJB7V6TPGcAqqz5MatD+tQMjzZiabJaV7gZctygyZYm
+7WnxJDRMZ+TqYxAmK9sG7RWIRnj6c76XLB1IYQcer75pbZ02MmAoaPXXTMCQ/Ul
ewY3kIpctt80YJAtfbIer9bRm6jN7jI/8hpjjv6wOqVPNR+MTOEgoDsCAQI=
-----END DH PARAMETERS-----