Static Buffer Allocation Option
Documentation and User’s Guide
May 2016 (revised June 2017)
1.0 INTRO
This document covers the API and build of wolfSSL with the staticmemory feature. Enabling the feature allows for use of static memory when performing a TLS/DTLS connection. To avoid calling system free’s on static memory and avoid memory not getting returned to a heap as available, the user should never call free on values created from wolfSSL functions in static memory mode. If a case is encountered where calling free is necessary on objects created by and returned from a wolfSSL function call, please contact us at wolfSSL for handling the free.
2.0 INSTALLATION
To enable the use of static memory with wolfSSL connections the configure flag staticmemory is used. An example of building wolfSSL with it enabled is as follows.
./configure --enable-staticmemory
Use of the static memory feature requires that WOLFSSL_SMALL_STACK not be turned on and that USE_FAST_MATH be defined. By default autotools turns on fastmath on 64bit systems. For non 64bit systems use --enable-fastmath, on some 32bit systems clang will be needed or C_EXTRA_FLAGS=-DTMF_NO_ASM for building with fastmath. For builds not using autotools, settings.h has a compile time error message displayed if these conditions are not met.
3.0 Design Overview
Static memory is assigned to a CTX structure that then passes a heap hint to all structures created by the CTX. In the following diagram the arrow indicates passing a pointer to the heap, “…” shows is all structs and not specific to the ones listed.
This allows for a connection established using a particular CTX to use the heap assigned to it. Functions being called with not using a structure that has a heap hint will not use the static memory set aside.
Memory is divided into to two separate types; IO memory, and general use. Depending on which flag is used when loading in memory will determine which type it is. By default it is stored as general memory. Assigning memory as IO memory using the flag WOLFMEM_IO_POOL will keep it set aside for when trying to malloc memory for use with sending and receiving information across the connection. This is to eliminate competition with none IO processes. Each IO pool of memory is close to 17k, this is to account for the max TLS packet size being 16k with some overhead.
There are some current function restrictions due to heap hint being passed in. These restrictions are derived from the design of passing heap hint from structure to structure since the functions have no connection with the base CTX but use dynamic memory. Many of these functions are in the OpenSSL compatibility layer. All of these can be ran with the default --enable-staticmemory but in cases will be making calls to malloc instead of using the static memory heap hint.
Functions for creating CTX:
wolfSSL_CTX_load_static_memory() should be used to create a static ctx instead of wolfSSL_CTX_new(). wolfSSL_CTX_new() can be used but will use dynamic memory for the creation of ctx and for the creation of WOLFSSL_METHOD struct. To avoid using dynamic memory wolfSSL_CTX_load_static_memory() can take a ctx set as NULL and a static method function (ending in _ex) to use only memory from the static buffer.
Functions that will not malloc in static mode and return null: these functions have the option of taking in a buffer or creating a buffer. If a buffer is not passed in as an argument then when using static memory a NULL value is returned.
char* wolfSSL_X509_NAME_oneline
byte* wolfSSL_X509_get_device_type
byte* wolfSSL_X509_get_hw_type
byte* wolfSSL_X509_get_hw_serial_number
Functions not supported to use static memory: these functions call malloc without any connection to a ctx heap hint. Many are in the OpenSSL compatibility layer and have the parameters and behavior they do to allow compatibility.
WOLFSSL_X509* wolfSSL_X509_d2i
WOLFSSL_X509* wolfSSL_X509_d2i_fp
WOLFSSL_X509* wolfSSL_X509_load_certificate_file
WOLFSSL_X509_STORE* wolfSSL_X509_STORE_new
WOLFSSL_X509_STORE_CTX* wolfSSL_X509_STORE_CTX_new
WOLFSSL_BIGNUM* wolfSSL_BN_new
void wolfSSL_BN_free
char *wolfSSL_BN_bn2dec
char *wolfSSL_BN_bn2hex
WOLFSSL_DH* wolfSSL_DH_new
WOLFSSL_DSA* wolfSSL_DSA_new
WOLFSSL_RSA* wolfSSL_RSA_new
int wolfSSL_PEM_write_mem_ECPrivateKey
int wolfSSL_PEM_write_mem_DSAPrivateKey
int wolfSSL_PEM_write_mem_RSAPrivateKey
WOLFSSL_EC_KEY *wolfSSL_EC_KEY_new
WOLFSSL_EC_GROUP *wolfSSL_EC_GROUP_new_by_curve_name
WOLFSSL_ECDSA_SIG *wolfSSL_ECDSA_SIG_new
int wolfSSL_PEM_write_mem_ECPrivateKey
int wolfSSL_PEM_write_mem_DSAPrivateKey
WOLFSSL_X509* wolfSSL_get_chain_X509
WOLFSSL_BIO* wolfSSL_BIO_new_socket
WOLFSSL_BIO* wolfSSL_BIO_new
WOLFSSL_BIO* wolfSSL_BIO_new_mem_buf
These 2 have fixed static buffer sizes by default and if a file is larger it is then dynamic: with these two functions there is partial functionality with static buffers. A buffer is placed on the stack by default and if not large enough then the buffer is increased using dynamic memory. In the case of static memory a MEMORY_E value is returned for files too large with these functions rather than creating new dynamic memory.
int wolfSSL_PemCertToDer
int wolfSSL_PemPubKeyToDer
If absolutely no mallocs is sought after then WOLFSSL_NO_MALLOC can be defined. This macro will have wolfSSL not compile in the use of malloc when a heap hint is not used. It will also cause make check to fail with unit tests and api tests that do not have heap hints. With WOLFSSL_NO_MALLOC examples/server/server is setup to function and can be ran to show a connection with absolutely no application mallocs. (note: using something like iprofiler will show system mallocs such as fopen for when certificate files are opened)
4.0 wolfSSL API
wolfSSL_CTX_load_static_memory
Synopsis:
#include <wolfssl/ssl.h>
int wolfSSL_CTX_load_static_memory(WOLFSSL_CTX** ctx, wolfSSL_method_func method, unsigned char* buf, unsigned int sz, int flag, int max);
Description:
This function is used to set aside static memory for a CTX. Memory set aside is then used for the CTX’s lifetime and for any SSL objects created from the CTX. By passing in a NULL ctx pointer and a wolfSSL_method_func function the creation of the CTX itself will also use static memory. wolfSSL_method_func has the function signature of WOLFSSL_METHOD* (*wolfSSL_method_func)(void* heap);.
Passing in 0 for max makes it behave as if not set and no max concurrent use restrictions is in place.
The flag value passed in determines how the memory is used and behavior while operating. Available flags are the following.
0 - default general memory
WOLFMEM_IO_POOL - used for input/output buffer when sending receiving messages.
Overrides general memory, so all memory in buffer passed in is used for IO.
WOLFMEM_IO_FIXED - same as WOLFMEM_IO_POOL but each SSL now keeps two
buffers to themselves for their lifetime.
WOLFMEM_TRACK_STATS - each SSL keeps track of memory stats while running.
Return Values:
If successful, SSL_SUCCESS will be returned.
All unsuccessful return values will be less than 0 or equal to SSL_FAILURE.
Parameters:
ctx - address of pointer to a WOLFSSL_CTX structure.
method - function to create protocol. (should be NULL if ctx is not also NULL)
buf - memory to use for all operations.
sz - size of memory buffer being passed in.
flag - type of memory.
max- max concurrent operations.
Example:
WOLFSSL_CTX* ctx;
WOLFSSL* ssl;
int ret;
unsigned char memory[MAX];
int memorySz = MAX;
unsigned char IO[MAX];
int IOSz = MAX;
int flag = WOLFMEM_IO_FIXED | WOLFMEM_TRACK_STATS;
...
// create ctx also using static memory, start with general memory to use
ctx = NULL:
ret = wolfSSL_CTX_load_static_memory(&ctx, wolfSSLv23_server_method_ex, memory, memorySz, 0, MAX_CONCURRENT_HANDSHAKES);
if (ret != SSL_SUCCESS) {
// handle error case
}
// load in memory for use with IO
ret = wolfSSL_CTX_load_static_memory(&ctx, NULL, IO, IOSz, flag, MAX_CONCURRENT_IO);
if (ret != SSL_SUCCESS) {
// handle error case
}
...
See Also:
wolfSSL_CTX_new
wolfSSL_CTX_is_static_memory
wolfSSL_is_static_memory
wolfSSL_CTX_is_static_memory
Synopsis:
#include <wolfssl/ssl.h>
int wolfSSL_CTX_is_static_memory(WOLFSSL_CTX* ctx, WOLFSSL_MEM_STATS* mem_stats);
Description:
This function does not change any of the connections behavior and is used only for gathering information about the static memory usage.
Return Values:
A value of 1 is returned if using static memory for the CTX is true.
0 is returned if not using static memory.
Parameters:
ctx - a pointer to a WOLFSSL_CTX structure, created using wolfSSL_CTX_new().
mem_stats - structure to hold information about static memory usage.
Example:
WOLFSSL_CTX* ctx;
int ret;
WOLFSSL_MEM_STATS mem_stats;
...
//get information about static memory with CTX
ret = wolfSSL_CTX_is_static_memory(ctx, &mem_stats);
if (ret == 1) {
// handle case of is using static memory
// print out or inspect elements of mem_stats
}
if (ret == 0) {
//handle case of ctx not using static memory
}
…
See Also:
wolfSSL_CTX_new
wolfSSL_CTX_load_static_memory
wolfSSL_is_static_memory
wolfSSL_is_static_memory
Synopsis:
#include <wolfssl/ssl.h>
int wolfSSL_is_static_memory(WOLFSSL* ssl, WOLFSSL_MEM_CONN_STATS* mem_stats);
Description:
wolfSSL_is_static_memory is used to gather information about a SSL’s static memory usage. The return value indicates if static memory is being used and WOLFSSL_MEM_CONN_STATS will be filled out if and only if the flag WOLFMEM_TRACK_STATS was passed to the parent CTX when loading in static memory.
Return Values:
A value of 1 is returned if using static memory for the CTX is true.
0 is returned if not using static memory.
Parameters:
ssl - a pointer to a WOLFSSL structure, created using wolfSSL_new().
mem_stats - structure to contain static memory usage.
Example:
WOLFSSL* ssl;
int ret;
WOLFSSL_MEM_CONN_STATS mem_stats;
...
ret = wolfSSL_is_static_memory(ssl, mem_stats);
if (ret == 1) {
// handle case when is static memory
// investigate elements in mem_stats if WOLFMEM_TRACK_STATS flag
}
...
See Also:
wolfSSL_new
wolfSSL_CTX_is_static_memory
wolfSSLv3_server_method_ex
wolfSSLv3_client_method_ex
wolfTLSv1_server_method_ex
wolfTLSv1_client_method_ex
wolfTLSv1_1_server_method_ex
wolfTLSv1_1_client_method_ex
wolfTLSv1_2_server_method_ex
wolfTLSv1_2_client_method_ex
wolfSSLv23_server_method_ex
wolfSSLv23_client_method_ex
Synopsis:
#include <wolfssl/ssl.h>
WOLFSSL_METHOD* (*wolfSSL_method_func)(void* heap)
Description:
These functions have the same behavior as their counterparts (functions with not having _ex) except that they do not create WOLFSSL_METHOD* using dynamic memory. The functions will use the heap hint passed in to create a new WOLFSSL_METHOD struct.
Return Values:
A value of WOLFSSL_METHOD pointer if success.
NULL is returned in error cases.
Parameters:
heap - a pointer to a heap hint for creating WOLFSSL_METHOD struct.
Example:
WOLFSSL_CTX* ctx;
int ret;
...
ctx = NULL:
ret = wolfSSL_CTX_load_static_memory(&ctx, wolfSSLv23_server_method_ex, memory, memorySz, 0, MAX_CONCURRENT_HANDSHAKES);
if (ret != SSL_SUCCESS) {
// handle error case
}
...
See Also:
wolfSSL_new
wolfSSL_CTX_new
wolfSSL_CTX_free
4.1 wolfCrypt API
wolfSSL_MemoryPaddingSz
Synopsis:
#include <wolfssl/wolfcrypt/memory.h>
int wolfSSL_MemoryPaddingSz(void);
Description:
This function calculates the amount of padding needed with each bucket and is for informational purposes. The padding value is used for holding a memory management structure.
Return Values:
Returns padding size needed with each memory bucket.
Parameters:
None
Example:
...
printf(“with current setup %d padding is used\n”, wolfSSL_MemoryPaddingSz());
See Also:
wolfSSL_CTX_new
wolfSSL_CTX_is_static_memory
wolfSSL_is_static_memory
wc_LoadStaticMemory
wolfSSL_StaticBufferSz
Synopsis:
#include <wolfssl/wolfcrypt/memory.h>
int wolfSSL_StaticBufferSz(byte* buf, word32 sz, int flag);
Description:
This function calculates the optimum buffer size given the preset memory buckets. The value returned is rounded down.
Return Values:
Returns optimum buffer size to use for no trailing memory space unused in the buffer.
Parameters:
buf - address of the buffer, needed for calculating memory alignment requirements.
sz - size in bytes of the buffer
flag - type of memory desired
Example:
int ret;
unsigned char memory[MAX];
int memorySz = MAX;
int flag = WOLFMEM_IO_FIXED | WOLFMEM_TRACK_STATS;
...
ret = wolfSSL_StaticBufferSz(memory, memorySz, flag);
//trim memory buffer based on ret
...
See Also:
wolfSSL_CTX_new
wolfSSL_CTX_is_static_memory
wolfSSL_is_static_memory
wc_LoadStaticMemory
wc_LoadStaticMemory
Synopsis:
#include <wolfssl/wolfcrypt/memory.h>
int wc_LoadStaticMemory(WOLFSSL_HEAP_HINT* hint, unsigned char* buf, unsigned int sz, int flag, int max);
Description:
This function is used to set aside static memory for wolfCrypt use. Memory can be used by passing the created heap hint into functions. An example of this is when calling wc_InitRng_ex. The flag value passed in determines how the memory is used and behavior while operating, in general wolfCrypt operations will use memory from a WOLFMEM_GENERAL pool. Available flags are the following.
WOLFMEM_GENERAL - default general memory
WOLFMEM_IO_POOL - used for input/output buffer when sending receiving messages.
Overrides general memory, so all memory in buffer passed in is used for IO.
WOLFMEM_IO_FIXED - same as WOLFMEM_IO_POOL but each SSL now keeps two
buffers to themselves for their lifetime.
WOLFMEM_TRACK_STATS - each SSL keeps track of memory stats while running.
Return Values:
If successful, 0 will be returned.
All unsuccessful return values will be less than 0.
Parameters:
hint- WOLFSSL_HEAP_HINT structure to use
buf - memory to use for all operations.
sz - size of memory buffer being passed in.
flag - type of memory.
max- max concurrent operations (handshakes, IO). Currently not as applicable with wolfCrypt operations but retained for the instance of when used with TLS connections.
Example:
WOLFSSL_HEAP_HINT hint;
int ret;
unsigned char memory[MAX];
int memorySz = MAX;
int flag = WOLFMEM_GENERAL | WOLFMEM_TRACK_STATS;
...
// load in memory for use
ret = wc_LoadStaticMemory(&hint, memory, memorySz, flag, 0);
if (ret != SSL_SUCCESS) {
// handle error case
}
…
ret = wc_InitRng_ex(&rng, hint, 0);
// check ret value
See Also:
5.0 Tuning
To help in the case of limited resources and to reduce the overhead of using memory buckets, tuning can be done with setting the macros WOLFMEM_BUCKETS and WOLFMEM_DIST. For example configuring with the following reduced the peak usage of the cipher suite ECDHE-ECDSA-AES256-SHA384:
./configure --enable-staticmemory --enable-psk C_EXTRA_FLAGS="-DALT_ECC_SIZE -DWOLFSSL_DEBUG_MEMORY -DWOLFMEM_BUCKETS=64,256,384,432,512,1632,2976,3456,16128 -DWOLFMEM_DIST=16,8,6,4,6,3,2,1,1" --disable-extended-master --enable-debug
The way this would be interpreted is 1 bucket of 16128 bytes each pass, 1 of 3456 bytes, 2 of 2976 bytes and so on, matching the distribution number to the bucket size. WOLFMEM_BUCKETS sizes being a list with the smallest on the left and largest on the right in order.
In the case that memory alignment needs to be adjusted it can be set by using the macro WOLFSSL_STATIC_ALIGN. By default this is set to be 16 byte aligned but the following gives an example of setting to 8 byte aligned.
./configure --enable-staticmemory C_EXTRA_FLAGS=-DWOLFSSL_STATIC_ALIGN=8
5.1 Tuning IO Size
This accounts for if controlling both server and client side and using the --enable-maxfragment feature. When using a set TLS packet size the static IO buffer can be reduced with setting the macro WOLFMEM_IO_SZ. The default size for WOLFMEM_IO_SZ is 16,992 bytes to account for the largest packet size possibly seen when connecting to uncontrolled end points but if reducing the packet size in a controlled connection this size then becomes wasteful. An example of setting this with using autotools would be “./configure --enable-staticmemory C_EXTRA_FLAGS=-DWOLFMEM_IO_SZ=660”. For more information about setting the fragment size with TLS connections see documentation on wolfSSL_CTX_UseMaxFragment and wolfSSL_UseMaxFragment.