• Registered: 2020-01-21
  • Posts: 7

Topic: [SOLVED] Asynchronous Reads...

Looking through the code I see no interface for asynchronous reads.
IE asynchronously pushing data into the SSL connection.
(see this earlier unresolved thread on this forum
:https://www.wolfssl.com/forums/topic393-using-wolfssl-embedded-ssl-with-an-asynchronous-socket.html)


Using your interface as is will the following work:

It seems to imply that the asynchronous accept/connect calls will fail if the read returns would block, but to keep calling the accept or the connect until it does succeed.

It also seems to suggest that the wolfSSL_peek will cause a read too occur...

So my strategy would be the following:

If I get asynchronous notification of new data to be read from tcp into the SSL connection:

If the connection is still trying to connect or accept, call the appropriate connect/accept until that function succeeds.
If the connection has finished the accept/connect negotiation then call peak instead.

This should work and can be multiplexed across multiple SSL connections inside a single SSL worker thread.

Does this sound like the correct strategy?

Share

  • From: Bozeman
  • Registered: 2014-05-09
  • Posts: 777

Re: [SOLVED] Asynchronous Reads...

Hi pbreed,

I think you're digging a little too deep. Assuming you are implementing a server given the desire to manage non-blocking reads on multiple connections am I right?

There is a unique port for each client connection with TLS and a unique WOLFSSL_OBJECT per connection. If the worker thread calls read on port X with WOLFSSL_OBJECT X and there is nothing to be read yet wolfSSL will just return a 0 or the number of bytes it was able to read to the application level, the application could then move on and call read on port Y with WOLFSSL_OBJECT Y to see if that one has data yet (or in the case of a previous partial read, finish what was reading whatever was missed the first time) and so on, once all active ports are checked loop back to the beginning and start over with any connections that didn't finish in the last iteration of the loop.

We have a non-blocking I/O example here you could start with as a base for your project: https://github.com/wolfSSL/wolfssl-exam … blocking.c

Can you tell us a bit about what you are working on and what is driving the effort?

- K

Kaleb J. Himes's Website

Share

3 Reply by pbreed (edited by pbreed 2020-01-31 17:04:35)

  • Registered: 2020-01-21
  • Posts: 7

Re: [SOLVED] Asynchronous Reads...

I'm CTO of Netburner. (Also original author of NetBurners written from scratch SSL/TLS)
We are porting to use WolfSSL...
(Why are we porting? because we found that keeping up with the evolution of TLS standards and ciphers etc.. was more work than the internal economic value it was creating. So we tried to solve this by porting to XXXXSSL.
Then almost immediately hit the wall with features we needed not supported by XXXX....
So we are trying again to use an external library (wolfSSL in this case) with the goal of not having a full time engineer work on nothing but TLS.



The reason for my question is almost perfectly encapsulated in the forum comments I posted in my initial request...
https://www.wolfssl.com/forums/topic393 … ocket.html


Basically we a TCP system that can generate notification of new data, and write buffer changing from block to unblocked...

I also have customer written code that operates on an FD  that needs all of the select (read,write,error) functionality...

I need to connect the two realms without creating a very expensive task for each SSL connection....


So TCP tells me there is new data available to read on the TCP socket associated with the WolfSSL connection...
I give that data to the wolfSSL system and as a result of that data I might or might not have generated decoded data for the customer/user code to read. So until that TCP data is processed I don't know if I can tell the customer/user FD if there is data avail , an  error or whatever...

One grossly inefficient way to do this would be to start a task/thread for each connection...
I'm looking for how I can multiplex each of these TCP notifications saying read data is available in a single SSL managment thread to marshal data from TCP throught the WolfSSL to/from the customer/user connections.

Both the XXXXSSL and NetBurners internally written TLS  code we are porting away from had that capability nativity.

I also need to extend the concept from just the read channel to write and accept and connect so the whole system can run asynchronously.

If I can't make this mode of operation work then I will have to seek other solutions, as the alternative is to tell 10K customers they must rewrite their code...

I've looked at the example pointed to in your previous response  and it seems like it only manages a single connection asyncronously, and does not do an asynchronous accept....

I've spent close to a week digging in your code and I think the solution I outlined above will work...
We are in the middle of experimenting in this space....

I do have one specific question I don't exactly understand...

lets say I call wolfSSL_write  with  data to write...so much data its going to need to be in multiple TLS messages...

There are three possible outcomes...
1)wolfSSL_write is able to encrypt the all of the requested write data into a block and ship ALL of that data out via TCP.

2)wolfSSL_write is able to encrypt the all of the requested write data into a block and ship SOME  of that data out via TCP.
with subsequent writes to the TCP system returning would block...

3)wolfSSL_write is able to encrypt some of the requested write data into a block and ship SOME  of that data out via TCP.
with subsequent writes to the TCP system returning would block...

 
So operating in an asynchronous manner in case 1, I'd expect the wolfSSL_write function to return the number of bytes written, matching the number requested...

In 2,3 I'd expect an error of
SSL_ERROR_WANT_WRITE

, but 2,3 are DIFFERNT!

How do I tell the difference between case 2 and case 3....
When I later get notified that the TCP connection has cleared its write backlog....
how do I recover for case 2 and case 3...

This is especially problematic when the write request will span multiple TLS encrypted data blocks...

Share

  • Registered: 2020-01-21
  • Posts: 7

Re: [SOLVED] Asynchronous Reads...

Can I change the behavior in wolfSSL_write by limiting the writes to some maximum that guarantees it fits in one encrypted block?

If so then I can manage outside the wolfSSL code to block/unblock writes into the system if TCP can't accept a full encrypted block of data...

Share

  • From: Bozeman
  • Registered: 2014-05-09
  • Posts: 777

Re: [SOLVED] Asynchronous Reads...

@pbreed,

Let me know if I miss anything in the below:

So in your case 2 and 3...

Case 2 should return a WOLFSSL_SUCCESS because all the data was encrypted and handed off to the TCP stack. Once it's handed off to the TCP stack wolfSSL no longer has any control of it or knowledge of if only a partial send was achieved. wolfSSL is wholly disconnected from the transport medium, this is why we are able to provide custom I/O callbacks that work over any mode of transport and we are not limited to TCP only.

Case 3 is the only case above that should return a WANT_WRITE because only a partial encrypt was achieved and not all of the data was handed off to the TCP stack.

Can I change the behavior in wolfSSL_write by limiting the writes to some maximum that guarantees it fits in one encrypted block?

Yes absolutely. If you need the write callback to behave differently you could either modify the default one that expects TCP or you could write your own "send" function that says "if the sz passed in is larger than 16k (the limit of a TLS packet) only try to encrypt and send 16k at most, return the amount encrypted and written successfully. (which will be turned into a WANT_WRITE when it bubbles out).

Here is some basic documentation for the return codes and behavior that are expected from the I/O callbacks:

int myReceive(WOLFSSL *ssl, char *buf, int sz, void *ctx)
{
    // ssl = the current SSL object, cast to void if unused
    // buf = the buffer to receive the message, always used
    // sz = the size in bytes to receive, always used
    // ctx = a custom user context, can be anything, a structure, char buf, variable, cast to the correct type and use as needed, cast to void if unused.

    // RULE1: Only return the amount received.
    // RULE2: In the case of a failed receive return one of the following errors as appropriate, returning 0 will
    //        trigger an automatic re-receive attempt without returning control to the calling application.
    //        WOLFSSL_CBIO_ERR_GENERAL    = -1,     /* general unexpected err */           
    //        WOLFSSL_CBIO_ERR_WANT_READ  = -2,     /* need to call read  again */         
    //        WOLFSSL_CBIO_ERR_WANT_WRITE = -2,     /* need to call write again */         
    //        WOLFSSL_CBIO_ERR_CONN_RST   = -3,     /* connection reset */                 
    //        WOLFSSL_CBIO_ERR_ISR        = -4,     /* interrupt */                        
    //        WOLFSSL_CBIO_ERR_CONN_CLOSE = -5,     /* connection closed or epipe */       
    //        WOLFSSL_CBIO_ERR_TIMEOUT    = -6      /* socket timeout */ 
    // RULE3: In the case of a partial receive, only return the amount read, call wolfSSL_read again
    //        with the exact same parameters (including sz), the state machine will internally keep
    //        track of received vs remainder and will handle the remainder appropriately.
}

int mySend(WOLFSSL *ssl, char *buf, int sz, void *ctx)
{
    // ssl = the current SSL object, cast to void if unused
    // buf = the message to send, always used
    // sz = the size in bytes to send, always used
    // ctx = a custom user context, can be anything, a structure, char buf, variable, cast to the correct type and use as needed, cast to void if unused.

    // RULE1: Only return the amount sent.
    // RULE2: In the case of a failed send return one of the following errors as appropriate, returning 0 will
    //        trigger an automatic re-send attempt without returning control to the calling application.
    //        WOLFSSL_CBIO_ERR_GENERAL    = -1,     /* general unexpected err */           
    //        WOLFSSL_CBIO_ERR_WANT_READ  = -2,     /* need to call read  again */         
    //        WOLFSSL_CBIO_ERR_WANT_WRITE = -2,     /* need to call write again */         
    //        WOLFSSL_CBIO_ERR_CONN_RST   = -3,     /* connection reset */                 
    //        WOLFSSL_CBIO_ERR_ISR        = -4,     /* interrupt */                        
    //        WOLFSSL_CBIO_ERR_CONN_CLOSE = -5,     /* connection closed or epipe */       
    //        WOLFSSL_CBIO_ERR_TIMEOUT    = -6      /* socket timeout */ 
    // RULE3: In the case of a partial send, only return the amount written, call wolfSSL_write again
    //        with the exact same parameters (including sz), the state machine will internally keep
    //        track of send vs remainder and will handle the remainder appropriately.
}

    // Register your callbacks in place of the defaults:
    wolfSSL_CTX_SetIORecv(ctx, mySend);
    wolfSSL_CTX_SetIOSend(ctx, myReceive);

Let me know if this helps.

Warm Regards,

K

Kaleb J. Himes's Website

Share