1 Topic by SheldonCooper (edited by SheldonCooper 2013-05-26 23:44:42)

  • Registered: 2011-05-31
  • Posts: 24

Topic: Sniffer Memory issue revamped -[Probably Solved - please look]

Hey again,
I have redesigned my experiment and got the following results:
1. I added counters for the number of sessions and packet buffers created - by the end of the run (after enough time without traffic) - all sessions and packets were accounted for being freed
2. it appears that the following is true:
I have skipped the creating of the session->sslServer and session->sslClient, and returned just before CheckPreRecord in sniffer.c/ssl_decodePacket.
3. also, this happens without ProcessMessage - which rules out a pretty large section of code to inspect.

that way, it appears that the overall memory consumption is much lower, but more importantly, returns back to base level after done transmitting traffic (a large recording, so i can see actual memory shifts...).

this turns my suspicion towards SSL_new and SSL_free - is it possible that XMALLOC and XFREE aren't performing "traditional" malloc/free operations, but some sort of pooling, I was trying to understand from the code but failed to reach certainty.

any help will be greatly appreciated.

thanks,
Dan

Share

  • Registered: 2011-05-31
  • Posts: 24

Re: Sniffer Memory issue revamped -[Probably Solved - please look]

Albeit the clear lack of interest  wink , I will update on my current results:
I performed the following:
in ssl.c/wolfSSL_new(WOLFSSL_CTX* ctx) I change:

ssl=(WOLFSSL*)XMALLOC(size(WOLFSSL),ctx->heap,DYNAMIC_TYPE_SSL)

with

ssl=(WOLFSSL*)malloc(sizeof(WOLFSSL))

this - for some reason, appears to have solved the issue.

anyone cares to shed some light on XMALLOC's operation?

i'm buffled...

thanks,
Dan

Share

  • From: Bozeman, MT
  • Registered: 2010-10-20
  • Posts: 650

Re: Sniffer Memory issue revamped -[Probably Solved - please look]

Hi Dan,

By default, wolfSSL maps XMALLOC(), XFREE(), and XREALLOC() to wolfSSL_Malloc(), wolfSSL_Free(), and wolfSSL_Realloc() respectively, in wolfssl/ctaocrypt/types.h:

/* default C runtime, can install different routines at runtime via cbs */  
#include <wolfssl/ctaocrypt/memory.h>                                        
#define XMALLOC(s, h, t)     ((void)h, (void)t, wolfSSL_Malloc((s)))         
#define XFREE(p, h, t)       {void* xp = (p); if((xp)) wolfSSL_Free((xp));}  
#define XREALLOC(p, n, h, t) wolfSSL_Realloc((p), (n))

The way this is set up allows the user application to register their own memory functions at runtime via the memory callbacks that wolfSSL provides, but defaults to just using malloc(), free(), and realloc() if no custom memory functions have been registered.  For example, wolfSSL_Malloc(), which is used by default, is located in ctaocrypt/src/memory.c:

void* wolfSSL_Malloc(size_t size)                                                
{                                                                               
    void* res = 0;                                                              
                                                                               
    if (malloc_function)                                                        
        res = malloc_function(size);                                            
    else                                                                        
        res = malloc(size);                                                     
        #ifdef WOLFSSL_MALLOC_CHECK                                              
            if(res == NULL)                                                     
                err_sys("wolfSSL_malloc")  ;                                 
        #endif                                                                  
                                                                                
    return res;                                                                 
}

Best Regards,
Chris

https://www.wolfssl.com

chrisc's Website

Share

  • Registered: 2011-05-31
  • Posts: 24

Re: Sniffer Memory issue revamped -[Probably Solved - please look]

thanks for your reply. If not flawed, my experiment shows that replacing the define for XMALLOC(s,h,t) with malloc(s) and respectively for XFREE with free(p) resulted in steady memory consumption as opposed to suspected memory leakage. I agree this is strange. I will conduct it again and let you know for sure. in the mean time - what is the meaning of ctx->heap when passed to XMALLOC?


one more thing about my method: I run traffic through the application (a lot of traffic I recorded over months from my website), and watch the TOP on my machine. I define memory leakage when linux's TOP does not go down to "base levle" after i finish transmitting traffic and RemoveStaleSessions is called (I added a way for me to call this method externally). I also exposed counters for the number of sessions and packet buffers allocated and freed - so this is certainly not the issue (all are eventually zeroed).

do you find this method reasonable? can you, from your experience, rely on TOP in such a manner?

thanks in advance,
Dan

Share

  • john
  • Moderator
  • Offline
  • From: Seattle, WA
  • Registered: 2011-11-14
  • Posts: 148

Re: Sniffer Memory issue revamped -[Probably Solved - please look]

XMALLOC()'s three parameters are a size, a heap pointer, and a memory type. With the default setup, the heap pointer (ctx->heap) and the type are not passed to the wolfSSL_Malloc() function. If you want to manage separate heaps, and allocate different types of memory out of different pools or heaps, you supply your own allocator that calls different allocators depending on the type. (We have someone who is using that on an embedded processor to allocate the send and receive buffers from a separate heap for large buffers like that.)

I've used top like that on another project years ago for the same reason. Right now I run the example server, example client, or the build test through Valgrind to check for leaks.

Are you passing FIN packets through to ssl_DecodePacket()? The sniffer-sessions should be freed when FIN is received and processed.  Part of freeing the sniffer-session is freeing the sslServer and sslClient objects.

john's Website

Share