• Registered: 2013-02-27
  • Posts: 5

Topic: failed testsuite results

Hello,

When I run the basic build test suite, using the command

./testsuite/testsuite

all tests pass. However, when I run the more comprehensive test suite, using the command

make test

I get a failed test. Below is the output of the test results for diagnostic purposes. These results were generated on a Windows 7 Pro PC, with AMD FX(tm)-8150 Eight-Core Processor and 64-bit OS. I'm using wolfSSL 2.5.0, and ran the test running CygWin.

P.S. What's the technical definition of "shizzle"?  smile 


$ make test
make -j2  check-am
make[1]: Entering directory `/cygdrive/c/Users/myUserName/Downloads/cyassl-2.5.0/cyassl-2.5.0'
make -j2  testsuite/testsuite.exe  tests/unit.exe
make[2]: Entering directory `/cygdrive/c/Users/myUserName/Downloads/cyassl-2.5.0/cyassl-2.5.0'
make[2]: warning: -jN forced in submake: disabling jobserver mode.
make[2]: `tests/unit.exe' is up to date.
make[2]: Leaving directory `/cygdrive/c/Users/myUserName/Downloads/cyassl-2.5.0/cyassl-2.5.0'
make -j2  check-TESTS
make[2]: Entering directory `/cygdrive/c/Users/myUserName/Downloads/cyassl-2.5.0/cyassl-2.5.0'
make[2]: warning: -jN forced in submake: disabling jobserver mode.
MD5      test passed!
MD4      test passed!
SHA      test passed!
SHA-256  test passed!
HMAC-MD5 test passed!
HMAC-SHA test passed!
HMAC-SHA256 test passed!
ARC4     test passed!
DES      test passed!
DES3     test passed!
AES      test passed!
RANDOM   test passed!
RSA      test passed!
DH       test passed!
DSA      test passed!
PWDBASED test passed!
Client message: hello cyassl!
Server response: I hear you fa shizzle!
sending server shutdown command: quit!
client sent quit command: shutting down!
6cd8940c5e7229f9357cc15b202b593befbbc8ea  input
6cd8940c5e7229f9357cc15b202b593befbbc8ea  output

All tests passed!
PASS: testsuite/testsuite.exe
starting unit tests...
 Begin API Tests
   CyaSSL_Init(): passed
   CyaSSLv3_server_method(): passed
   CyaSSLv3_client_method(): passed
   CyaTLSv1_server_method(): passed
   CyaTLSv1_client_method(): passed
   CyaTLSv1_1_server_method(): passed
   CyaTLSv1_1_client_method(): passed
   CyaTLSv1_2_server_method(): passed
   CyaTLSv1_2_client_method(): passed
   CyaSSLv23_client_method(): passed
   CyaSSL_CTX_new(NULL): passed
   CyaSSL_CTX_new(method): passed
   CyaSSL_CTX_use_certificate_file(NULL, NULL, 9999): passed
   CyaSSL_CTX_use_certificate_file(ctx, bogusFile, SSL_FILETYPE_PEM): passed
   CyaSSL_CTX_use_certificate_file(ctx, svrCert, 9999): passed
   CyaSSL_CTX_use_certificate_file(ctx, svrCert, SSL_FILETYPE_PEM): passed
   CyaSSL_CTX_use_PrivateKey_file(NULL, NULL, 9999): passed
   CyaSSL_CTX_use_PrivateKey_file(ctx, bogusFile, SSL_FILETYPE_PEM): passed
   CyaSSL_CTX_use_PrivateKey_file(ctx, svrKey, 9999): passed
   CyaSSL_CTX_use_PrivateKey_file(ctx, svrKey, SSL_FILETYPE_PEM): passed
   CyaSSL_CTX_load_verify_locations(NULL, NULL, NULL): passed
   CyaSSL_CTX_load_verify_locations(ctx, NULL, NULL): passed
   CyaSSL_CTX_load_verify_locations(ctx, NULL, NULL): passed
   CyaSSL_CTX_load_verify_locations(ctx, caCert, bogusFile): passed
   CyaSSL_CTX_load_verify_locations(ctx, caCert, 0): passed
   CyaSSL_new(NULL) server: passed
   CyaSSL_new(ctx_nocert) server: passed
   CyaSSL_new(ctx) server: passed
   CyaSSL_new(NULL) client: passed
   CyaSSL_new(ctx_nocert) client: passed
   CyaSSL_new(ctx) client: passed
Client message: hello cyassl!
Server response: I hear you fa shizzle!
   CyaSSL_Cleanup(): passed
 End API Tests
 Begin HASH Tests
   MD4      test passed!
   MD5      test passed!
   SHA      test passed!
   SHA-256  test passed!
   HMAC-MD5 test passed!
   HMAC-SHA test passed!
   HMAC-SHA256 test passed!
 End HASH Tests
 Begin Cipher Suite Tests
starting default cipher suite tests
notice: using default file tests/test.conf
# server SSLv3 RC4-SHA
# client SSLv3 RC4-SHA
trying server command line[1]: SuiteTest -v 0 -l RC4-SHA
trying client command line[1]: SuiteTest -v 0 -l RC4-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server SSLv3 RC4-MD5
# client SSLv3 RC4-MD5
trying server command line[2]: SuiteTest -v 0 -l RC4-MD5
trying client command line[2]: SuiteTest -v 0 -l RC4-MD5
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server SSLv3 DES-CBC3-SHA
# client SSLv3 DES-CBC3-SHA
trying server command line[3]: SuiteTest -v 0 -l DES-CBC3-SHA
trying client command line[3]: SuiteTest -v 0 -l DES-CBC3-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1 RC4-SHA
# client TLSv1 RC4-SHA
trying server command line[4]: SuiteTest -v 1 -l RC4-SHA
trying client command line[4]: SuiteTest -v 1 -l RC4-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1 RC4-MD5
# client TLSv1 RC4-MD5
trying server command line[5]: SuiteTest -v 1 -l RC4-MD5
trying client command line[5]: SuiteTest -v 1 -l RC4-MD5
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1 DES-CBC3-SHA
# client TLSv1 DES-CBC3-SHA
trying server command line[6]: SuiteTest -v 1 -l DES-CBC3-SHA
trying client command line[6]: SuiteTest -v 1 -l DES-CBC3-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1 AES128-SHA
# client TLSv1 AES128-SHA
trying server command line[7]: SuiteTest -v 1 -l AES128-SHA
trying client command line[7]: SuiteTest -v 1 -l AES128-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1 AES256-SHA
# client TLSv1 AES256-SHA
trying server command line[8]: SuiteTest -v 1 -l AES256-SHA
trying client command line[8]: SuiteTest -v 1 -l AES256-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1 AES128-SHA256
# client TLSv1 AES128-SHA256
trying server command line[9]: SuiteTest -v 1 -l AES128-SHA256
trying client command line[9]: SuiteTest -v 1 -l AES128-SHA256
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1 AES256-SHA256
# client TLSv1 AES256-SHA256
trying server command line[10]: SuiteTest -v 1 -l AES256-SHA256
trying client command line[10]: SuiteTest -v 1 -l AES256-SHA256
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.1 RC4-SHA
# client TLSv1.1 RC4-SHA
trying server command line[11]: SuiteTest -v 2 -l RC4-SHA
trying client command line[11]: SuiteTest -v 2 -l RC4-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.1 RC4-MD5
# client TLSv1.1 RC4-MD5
trying server command line[12]: SuiteTest -v 2 -l RC4-MD5
trying client command line[12]: SuiteTest -v 2 -l RC4-MD5
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.1 DES-CBC3-SHA
# client TLSv1.1 DES-CBC3-SHA
trying server command line[13]: SuiteTest -v 2 -l DES-CBC3-SHA
trying client command line[13]: SuiteTest -v 2 -l DES-CBC3-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.1 AES128-SHA
# client TLSv1.1 AES128-SHA
trying server command line[14]: SuiteTest -v 2 -l AES128-SHA
trying client command line[14]: SuiteTest -v 2 -l AES128-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.1 AES256-SHA
# client TLSv1.1 AES256-SHA
trying server command line[15]: SuiteTest -v 2 -l AES256-SHA
trying client command line[15]: SuiteTest -v 2 -l AES256-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.1 AES128-SHA256
# client TLSv1.1 AES128-SHA256
trying server command line[16]: SuiteTest -v 2 -l AES128-SHA256
trying client command line[16]: SuiteTest -v 2 -l AES128-SHA256
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.1 AES256-SHA256
# client TLSv1.1 AES256-SHA256
trying server command line[17]: SuiteTest -v 2 -l AES256-SHA256
trying client command line[17]: SuiteTest -v 2 -l AES256-SHA256
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.2 RC4-SHA
# client TLSv1.2 RC4-SHA
trying server command line[18]: SuiteTest -v 3 -l RC4-SHA
trying client command line[18]: SuiteTest -v 3 -l RC4-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.2 RC4-MD5
# client TLSv1.2 RC4-MD5
trying server command line[19]: SuiteTest -v 3 -l RC4-MD5
trying client command line[19]: SuiteTest -v 3 -l RC4-MD5
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.2 DES-CBC3-SHA
# client TLSv1.2 DES-CBC3-SHA
trying server command line[20]: SuiteTest -v 3 -l DES-CBC3-SHA
trying client command line[20]: SuiteTest -v 3 -l DES-CBC3-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.2 AES128-SHA
# client TLSv1.2 AES128-SHA
trying server command line[21]: SuiteTest -v 3 -l AES128-SHA
trying client command line[21]: SuiteTest -v 3 -l AES128-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.2 AES256-SHA
# client TLSv1.2 AES256-SHA
trying server command line[22]: SuiteTest -v 3 -l AES256-SHA
trying client command line[22]: SuiteTest -v 3 -l AES256-SHA
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.2 AES128-SHA256
# client TLSv1.2 AES128-SHA256
trying server command line[23]: SuiteTest -v 3 -l AES128-SHA256
trying client command line[23]: SuiteTest -v 3 -l AES128-SHA256
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server TLSv1.2 AES256-SHA256
# client TLSv1.2 AES256-SHA256
trying server command line[24]: SuiteTest -v 3 -l AES256-SHA256
trying client command line[24]: SuiteTest -v 3 -l AES256-SHA256
Client message: hello cyassl!
Server response: I hear you fa shizzle!
# server SSLv3 RC4-SHA NON-BLOCKING
# client SSLv3 RC4-SHA NON-BLOCKING
trying server command line[25]: SuiteTest -v 0 -l RC4-SHA -N
trying client command line[25]: SuiteTest -v 0 -l RC4-SHA -N
... client would read block
... server would read block
... client would read block
yassl error: CyaSSL_read failed
FAIL: tests/unit.exe
=====================================
1 of 2 tests failed
Please report to http://www.yassl.com
=====================================
Makefile:2027: recipe for target `check-TESTS' failed
make[2]: *** [check-TESTS] Error 1
make[2]: Leaving directory `/cygdrive/c/Users/myUserName/Downloads/cyassl-2.5.0/cyassl-2.5.0'
Makefile:2276: recipe for target `check-am' failed
make[1]: *** [check-am] Error 2
make[1]: Leaving directory `/cygdrive/c/Users/myUserName/Downloads/cyassl-2.5.0/cyassl-2.5.0'
Makefile:2279: recipe for target `check' failed
make: *** [check] Error 2

Share

  • john
  • Moderator
  • Offline
  • From: Seattle, WA
  • Registered: 2011-11-14
  • Posts: 148

Re: failed testsuite results

Sorry for the delay in getting back to you. We were at RSA last week and some things got lost between the cracks.

Apparently, there is a case where Cygwin's recv() command will return ECONNABORTED, where usually we see some other error code. It looks like it is happening when we're using non-blocking sockets for a particular set of test cases. I'm adding some handling for that error code. We'll be treating it in our EmbedRecv callback like it was closed. (The fix is in our wolfSSL embedded SSL GitHub repository.)

The word "shizzle" is slang for "sure", as in agreement. "I hear you for sure." The term was popularized by the hip-hop singer Snoop Dogg.

john's Website

Share

  • Registered: 2013-02-27
  • Posts: 5

Re: failed testsuite results

Hi John,

Thanks for the reply and info. No worries whatsoever about the delay! I will download the new code from GitHub, fo shizzle.

mmhorii

Share