wolfSSL implements support for both client side and server side DTLS. The server side requires extra attention when it comes to Denial-of-Service (DoS) attacks. One way to mitigate DoS on DTLS servers is to operate statelessly until a cookie exchange is completed with the peer. The cookie exchange is implemented in all versions of DTLS. DTLS 1.2 uses a special HelloVerifyRequest message while DTLS 1.3 uses the TLS 1.3 HelloRetryRequest with a cookie extension. The general principle of the cookie exchange is shown in the following figures.
Client Server
------ ------
ClientHello ------>
<----- HelloVerifyRequest
(contains cookie)
ClientHello ------>
(with cookie)
[Rest of handshake]
Figure 1: DTLS 1.2 cookie exchange (https://www.rfc-editor.org/rfc/rfc6347#section-4.2.1)
Client Server
------ ------
ClientHello ------>
<----- HelloRetryRequest
+ cookie
ClientHello ------>
+ cookie
[Rest of handshake]
Figure 2: DTLS 1.3 cookie exchange (https://www.rfc-editor.org/rfc/rfc9147.html#section-5.1)
The trick is to parse the initial ClientHello without maintaining state. In wolfSSL release 5.6.0, we implemented parsing the initial ClientHello without maintaining state at all (https://github.com/wolfSSL/wolfssl/pull/5910). Previously, wolfSSL would reset the object when requesting a cookie exchange but this can be unreliable on errors or when new features are implemented. Now we have a dedicated routine to parse the ClientHello statelessly.
wolfSSL also has a callback available when a peer has been verified. To set this callback, use the int wolfDTLS_SetChGoodCb(WOLFSSL* ssl, ClientHelloGoodCb cb, void* user_ctx) API.
If you have any questions or run into any issues, contact us at facts@wolfssl.com, or call us at +1 425 245 8247.