I tried to compile the 2.8.5a with --enable-debug but it fails:
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
autoreconf: configure.ac: tracing
autoreconf: running: libtoolize --copy --force
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, `build-aux'.
libtoolize: copying file `build-aux/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.
libtoolize: copying file `m4/libtool.m4'
libtoolize: copying file `m4/ltoptions.m4'
libtoolize: copying file `m4/ltsugar.m4'
libtoolize: copying file `m4/ltversion.m4'
libtoolize: copying file `m4/lt~obsolete.m4'
autoreconf: running: /usr/bin/autoconf --force
autoreconf: running: /usr/bin/autoheader --force
autoreconf: running: automake --add-missing --copy --force-missing
autoreconf: Leaving directory `.'
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether UID '1000' is supported by ustar format... yes
checking whether GID '1000' is supported by ustar format... yes
checking how to create a ustar tar archive... gnutar
checking how to print strings... printf
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for a sed that does not truncate output... /bin/sed
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /usr/bin/ld -m elf_x86_64
checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... yes
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether the -Werror option is usable... yes
checking for simple visibility declarations... yes
checking whether make supports nested variables... (cached) yes
checking for gethostbyname... yes
checking for getaddrinfo... yes
checking for gettimeofday... yes
checking for inet_ntoa... yes
checking for memset... yes
checking for socket... yes
checking arpa/inet.h usability... yes
checking arpa/inet.h presence... yes
checking for arpa/inet.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking netdb.h usability... yes
checking netdb.h presence... yes
checking for netdb.h... yes
checking netinet/in.h usability... yes
checking netinet/in.h presence... yes
checking for netinet/in.h... yes
checking stddef.h usability... yes
checking stddef.h presence... yes
checking for stddef.h... yes
checking sys/ioctl.h usability... yes
checking sys/ioctl.h presence... yes
checking for sys/ioctl.h... yes
checking sys/socket.h usability... yes
checking sys/socket.h presence... yes
checking for sys/socket.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking errno.h usability... yes
checking errno.h presence... yes
checking for errno.h... yes
checking for socket in -lnetwork... no
checking size of long long... 8
checking size of long... 8
checking whether byte ordering is bigendian... no
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking whether gcc and cc understand -c and -o together... yes
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ accepts -g... (cached) yes
checking dependency style of g++... (cached) gcc3
checking for size_t... yes
checking for uint8_t... yes
checking dependency style of gcc... gcc3
checking for cos in -lm... yes
checking for thread local storage (TLS) class... __thread
checking whether the linker accepts ... yes
configure: adding automake macro support
configure: creating aminclude.am
./configure: line 18099: AX_SAVE_FLAGS: command not found
checking mcheck.h usability... yes
checking mcheck.h presence... yes
checking for mcheck.h... yes
checking for main in -lmcheck... yes
./configure: line 18155: AX_RESTORE_FLAGS: command not found
checking whether the linker accepts -lmcheck... yes
./configure: line 18240: printf: --: invalid option
printf: usage: printf [-v var] format [arguments]
./configure: line 18247: printf: -D: invalid option
printf: usage: printf [-v var] format [arguments]
checking for debug... yes
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking for PTHREAD_PRIO_INHERIT... yes
checking pcap/pcap.h usability... yes
checking pcap/pcap.h presence... yes
checking for pcap/pcap.h... yes
checking for libz... yes
checking for library containing gethostbyname... none required
checking for library containing socket... none required
checking whether C compiler accepts ... yes
checking for vcs system... git
checking for vcs checkout... yes
checking whether the linker accepts -Werror... yes
checking whether the linker accepts -z relro -z now... yes
checking whether the linker accepts -pie... yes
checking whether C compiler accepts -Werror... yes
checking whether C compiler accepts -g... yes
checking whether C compiler accepts -ggdb... yes
checking whether C compiler accepts -O0... yes
checking whether C compiler accepts -Wno-pragmas... yes
checking whether C compiler accepts -Wall... yes
checking whether C compiler accepts -Wno-strict-aliasing... yes
checking whether C compiler accepts -Wextra... yes
checking whether C compiler accepts -Wunknown-pragmas... yes
checking whether C compiler accepts -Wthis-test-should-fail... no
checking whether C compiler accepts --param=ssp-buffer-size=1... yes
checking whether C compiler accepts -Waddress... yes
checking whether C compiler accepts -Warray-bounds... yes
checking whether C compiler accepts -Wbad-function-cast... yes
checking whether C compiler accepts -Wchar-subscripts... yes
checking whether C compiler accepts -Wcomment... yes
checking whether C compiler accepts -Wfloat-equal... yes
checking whether C compiler accepts -Wformat-security... yes
checking whether C compiler accepts -Wformat=2... yes
checking whether C compiler accepts -Wmaybe-uninitialized... yes
checking whether C compiler accepts -Wmissing-field-initializers... yes
checking whether C compiler accepts -Wmissing-noreturn... yes
checking whether C compiler accepts -Wmissing-prototypes... yes
checking whether C compiler accepts -Wnested-externs... yes
checking whether C compiler accepts -Wnormalized=id... yes
checking whether C compiler accepts -Woverride-init... yes
checking whether C compiler accepts -Wpointer-arith... yes
checking whether C compiler accepts -Wpointer-sign... yes
checking whether C compiler accepts -Wredundant-decls... yes
checking whether C compiler accepts -Wshadow... yes
checking whether C compiler accepts -Wshorten-64-to-32... no
checking whether C compiler accepts -Wsign-compare... yes
checking whether C compiler accepts -Wstrict-overflow=1... yes
checking whether C compiler accepts -Wstrict-prototypes... no
checking whether C compiler accepts -Wswitch-enum... yes
checking whether C compiler accepts -Wundef... yes
checking whether C compiler accepts -Wunused... yes
checking whether C compiler accepts -Wunused-result... yes
checking whether C compiler accepts -Wunused-variable... yes
checking whether C compiler accepts -Wwrite-strings... yes
checking whether C compiler accepts -fwrapv... yes
creating cyassl-config - generic 2.8.5 for -lcyassl -lz
checking the number of available CPUs... 4
configure: added jobserver support to make for 5 jobs
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating stamp-h
config.status: creating Makefile
config.status: creating cyassl/version.h
config.status: creating cyassl/options.h
config.status: creating support/cyassl.pc
config.status: creating rpm/spec
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
---
Running make clean...
---
Generating user options header...
option w/o begin -D is -g, not saving to cyassl/options.h
option w/o begin -D is -pthread, not saving to cyassl/options.h
option w/o begin -D is -maes, not saving to cyassl/options.h
option w/o begin -D is -msse4, not saving to cyassl/options.h
option w/o begin -D is -Wall, not saving to cyassl/options.h
option w/o begin -D is -Wno-unused, not saving to cyassl/options.h
---
Configuration summary for wolfssl version 2.8.5
* Installation prefix: /usr/local
* System type: unknown-linux-gnu
* Host CPU: x86_64
* C Compiler:
* C Flags: -Werror -g -ggdb -O0 -Wno-pragmas -Wall -Wno-strict-aliasing -Wextra -Wunknown-pragmas --param=ssp-buffer-size=1 -Waddress -Warray-bounds -Wbad-function-cast -Wchar-subscripts -Wcomment -Wfloat-equal -Wformat-security -Wformat=2 -Wmaybe-uninitialized -Wmissing-field-initializers -Wmissing-noreturn -Wmissing-prototypes -Wnested-externs -Wnormalized=id -Woverride-init -Wpointer-arith -Wpointer-sign -Wredundant-decls -Wshadow -Wsign-compare -Wstrict-overflow=1 -Wswitch-enum -Wundef -Wunused -Wunused-result -Wunused-variable -Wwrite-strings -fwrapv
* C++ Compiler:
* C++ Flags: -g -O2
* CPP Flags: -fvisibility=hidden -DHAVE_LIBZ
* LIB Flags: -pie -z relro -z now -Werror
* Debug enabled: yes
* Warnings as failure: yes
* make -j: 5
* VCS checkout: yes
Features
* Single threaded: no
* Filesystem: yes
* OpenSSL Extra API: no
* fastmath: yes
* sniffer: yes
* snifftest: yes
* ARC4: yes
* AES: yes
* AES-NI: yes
* AES-GCM: yes
* AES-CCM: yes
* DES3: yes
* Camellia: yes
* NULL Cipher: yes
* MD5: yes
* RIPEMD: yes
* SHA: yes
* SHA-512: yes
* BLAKE2: yes
* keygen: no
* certgen: no
* certreq: no
* HC-128: yes
* RABBIT: yes
* PWDBASED: no
* HKDF: yes
* MD4: yes
* PSK: yes
* LEANPSK: no
* RSA: yes
* DSA: yes
* DH: yes
* ECC: yes
* FPECC: yes
* ECC_ENCRYPT: yes
* ASN: yes
* CODING: yes
* MEMORY: yes
* ERROR_STRINGS: yes
* DTLS: yes
* Old TLS Versions: yes
* OCSP: no
* CRL: no
* CRL-MONITOR: no
* Persistent session cache: no
* Persistent cert cache: no
* Atomic User Record Layer: no
* Public Key Callbacks: no
* NTRU: no
* SNI: no
* Maximum Fragment Length: no
* Truncated HMAC: no
* All TLS Extensions: no
* PKCS#7 no
* valgrind unit tests: no
* LIBZ: yes
* Examples: no
---
make -j5 all-am
make[1]: Entering directory `/home/luca/Work/phoenix/neteye/workspace/cyassl-git'
CCAS ctaocrypt/src/aes_asm.lo
CC src/src_libcyassl_la-internal.lo
CC src/src_libcyassl_la-io.lo
CC src/src_libcyassl_la-keys.lo
CC src/src_libcyassl_la-ssl.lo
CC src/src_libcyassl_la-tls.lo
CC ctaocrypt/src/src_libcyassl_la-hmac.lo
CC ctaocrypt/src/src_libcyassl_la-random.lo
CC ctaocrypt/src/src_libcyassl_la-sha256.lo
CC ctaocrypt/src/src_libcyassl_la-logging.lo
CC ctaocrypt/src/src_libcyassl_la-port.lo
CC ctaocrypt/src/src_libcyassl_la-error.lo
CC ctaocrypt/src/src_libcyassl_la-memory.lo
CC ctaocrypt/src/src_libcyassl_la-rsa.lo
CC ctaocrypt/src/src_libcyassl_la-dh.lo
CC ctaocrypt/src/src_libcyassl_la-asn.lo
CC ctaocrypt/src/src_libcyassl_la-coding.lo
CC ctaocrypt/src/src_libcyassl_la-aes.lo
CC ctaocrypt/src/src_libcyassl_la-des3.lo
CC ctaocrypt/src/src_libcyassl_la-sha.lo
CC ctaocrypt/src/src_libcyassl_la-arc4.lo
CC ctaocrypt/src/src_libcyassl_la-md4.lo
CC ctaocrypt/src/src_libcyassl_la-md5.lo
CC ctaocrypt/src/src_libcyassl_la-dsa.lo
CC ctaocrypt/src/src_libcyassl_la-camellia.lo
CC ctaocrypt/src/src_libcyassl_la-md2.lo
CC ctaocrypt/src/src_libcyassl_la-ripemd.lo
CC ctaocrypt/src/src_libcyassl_la-sha512.lo
CC ctaocrypt/src/src_libcyassl_la-blake2b.lo
CC src/src_libcyassl_la-sniffer.lo
CC ctaocrypt/src/src_libcyassl_la-hc128.lo
CC ctaocrypt/src/src_libcyassl_la-rabbit.lo
CC ctaocrypt/src/src_libcyassl_la-tfm.lo
CC ctaocrypt/src/src_libcyassl_la-ecc.lo
CC ctaocrypt/src/src_libcyassl_la-compress.lo
CC ctaocrypt/benchmark/benchmark.o
CC ctaocrypt/test/test.o
CC sslSniffer/sslSnifferTest/snifftest.o
CCLD src/libcyassl.la
/usr/bin/ld: final link failed: Memory exhausted
collect2: error: ld returned 1 exit status
make[1]: *** [src/libcyassl.la] Error 1
make[1]: Leaving directory `/home/luca/Work/phoenix/neteye/workspace/cyassl-git'
make: *** [all] Error 2
without --enable-debug the build works