Hi,
I am trying to replace the mbedTLS network layer with wolfSSL in the AWS IoT C SDK.
The simple subscribe_publish_sample demo that comes with the SDK works fine from a Linux host when using mbedTLS.
However, with my replacement of it with wolfSSL, I am getting a failure with error = 42, which seems to be that the server rejected the client certificate...?
The TCP connection to the AWS server is established, then the socket associated with the wolfSSL session, and the server's rootCA, client's certificate and private key added to the session (I removed the error handling code):
const int cert_container_format = SSL_FILETYPE_PEM;
WOLFSSL_METHOD * const method = wolfTLSv1_2_client_method();
hentry = gethostbyname(host_url);
memcpy(&sock_addr.sin_addr.s_addr, *hentry->h_addr_list, hentry->h_length);
sock_addr.sin_family = hentry->h_addrtype;
socket_fd = socket(sock_addr.sin_family, SOCK_STREAM, 0);
sock_addr.sin_port = htons(host_port);
connect(socket_fd, (struct sockaddr *)&sock_addr, sizeof(sock_addr));
wolfSSL_Init();
ssl_ctx = wolfSSL_CTX_new(method);
session = wolfSSL_new(ssl_ctx);
result = wolfSSL_CTX_load_verify_locations(ssl_ctx, root_ca_filepath, NULL);
result = wolfSSL_CTX_use_PrivateKey_file(ssl_ctx, dev_prvkey_filepath,
cert_container_format);
result = wolfSSL_CTX_use_certificate_file(ssl_ctx, dev_cert_filepath, cert_container_format);
wolfSSL_set_fd(session, socket_fd);
result = wolfSSL_connect(session);
Here is the full dump with debugging enabled:
AWS IoT SDK Version 2.1.1-
DEBUG: main L#166 rootCA /home/ben/git/aws-iot-device-sdk-embedded-C/samples/linux/subscribe_publish_sample/../../../certs/aws-rootca.pem
DEBUG: main L#167 clientCRT /home/ben/git/aws-iot-device-sdk-embedded-C/samples/linux/subscribe_publish_sample/../../../certs/aws-cert.pem
DEBUG: main L#168 clientKey /home/ben/git/aws-iot-device-sdk-embedded-C/samples/linux/subscribe_publish_sample/../../../certs/aws-prvkey.pem
Connecting...
URL: a3i4lmgkxatoyr.iot.us-west-2.amazonaws.com
name: dualstack.iotmoonraker-u-elb-1w8qnw1336zq-1186348092.us-west-2.elb.amazonaws.com, len: 4, type: 2
alternate names:
a3i4lmgkxatoyr.iot.us-west-2.amazonaws.com
iotmoonraker.us-west-2.prod.iot.us-west-2.amazonaws.com
addresses:
52.10.19.111
52.41.23.91
34.210.178.78
35.165.44.84
52.25.57.203
35.160.71.83
wolfSSL Entering wolfSSL_Init
wolfSSL Entering wolfCrypt_Init
wolfSSL Entering WOLFSSL_CTX_new_ex
wolfSSL Entering wolfSSL_CertManagerNew
wolfSSL Leaving WOLFSSL_CTX_new, return 0
wolfSSL Entering SSL_new
wolfSSL Leaving SSL_new, return 0
DEBUG: iot_tls_connect L#271 root CA file: /home/ben/git/aws-iot-device-sdk-embedded-C/samples/linux/subscribe_publish_sample/../../../certs/aws-rootca.pem
wolfSSL Entering wolfSSL_CTX_load_verify_locations
Getting dynamic buffer
Processing CA PEM file
wolfSSL Entering PemToDer
Adding a CA
wolfSSL Entering GetExplicitVersion
wolfSSL Entering GetSerialNumber
Got Cert Header
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
Got Algo ID
Getting Cert Name
Getting Cert Name
Got Subject Name
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
Got Key
Parsed Past Key
wolfSSL Entering DecodeCertExtensions
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeBasicCaConstraint
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeKeyUsage
wolfSSL Entering GetObjectId()
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeSubjKeyId
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
Parsed new CA
Freeing Parsed CA
Freeing der CA
OK Freeing der CA
wolfSSL Leaving AddCA, return 0
Processed a CA
Processed at least one valid CA. Other stuff OK
DEBUG: iot_tls_connect L#280 dev private key file: /home/ben/git/aws-iot-device-sdk-embedded-C/samples/linux/subscribe_publish_sample/../../../certs/aws-prvkey.pem
wolfSSL Entering wolfSSL_CTX_use_PrivateKey_file
Getting dynamic buffer
wolfSSL Entering PemToDer
DEBUG: iot_tls_connect L#290 dev certificate file: /home/ben/git/aws-iot-device-sdk-embedded-C/samples/linux/subscribe_publish_sample/../../../certs/aws-cert.pem
wolfSSL Entering wolfSSL_CTX_use_certificate_file
Getting dynamic buffer
wolfSSL Entering PemToDer
Checking cert signature type
wolfSSL Entering GetExplicitVersion
wolfSSL Entering GetSerialNumber
Got Cert Header
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
Got Algo ID
Getting Cert Name
Getting Cert Name
Got Subject Name
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
Got Key
Not ECDSA cert signature
wolfSSL Entering SSL_set_fd
wolfSSL Entering SSL_set_read_fd
wolfSSL Leaving SSL_set_read_fd, return 1
wolfSSL Entering SSL_set_write_fd
wolfSSL Leaving SSL_set_write_fd, return 1
wolfSSL Entering SSL_connect()
Adding signature algorithms extension
growing output buffer
Signature Algorithms extension to write
Elliptic Curves extension to write
Shrinking output buffer
connect state: CLIENT_HELLO_SENT
growing input buffer
received record layer msg
wolfSSL Entering DoHandShakeMsg()
wolfSSL Entering DoHandShakeMsgType
processing server hello
wolfSSL Entering VerifyClientSuite
wolfSSL Leaving DoHandShakeMsgType(), return 0
wolfSSL Leaving DoHandShakeMsg(), return 0
More messages in record
received record layer msg
wolfSSL Entering DoHandShakeMsg()
wolfSSL Entering DoHandShakeMsgType
processing certificate
wolfSSL Entering ProcessPeerCerts
Loading peer's cert chain
Put another cert into chain
Put another cert into chain
wolfSSL Entering GetExplicitVersion
wolfSSL Entering GetSerialNumber
Got Cert Header
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
Got Algo ID
Getting Cert Name
Getting Cert Name
Got Subject Name
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
wolfSSL Entering GetObjectId()
Got Key
Parsed Past Key
wolfSSL Entering DecodeCertExtensions
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeAuthInfo
wolfSSL Entering GetObjectId()
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeBasicCaConstraint
wolfSSL Entering GetObjectId()
Certificate Policy extension not supported yet.
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeCrlDist
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeKeyUsage
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeAltNames
Unsupported name type, skipping
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeSubjKeyId
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeAuthKeyId
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
About to verify certificate signature
wolfSSL Entering ConfirmSignature
wolfSSL Leaving ConfirmSignature, return 0
Adding CA from chain
Adding a CA
wolfSSL Entering GetExplicitVersion
wolfSSL Entering GetSerialNumber
Got Cert Header
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
Got Algo ID
Getting Cert Name
Getting Cert Name
Got Subject Name
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
wolfSSL Entering GetObjectId()
Got Key
Parsed Past Key
wolfSSL Entering DecodeCertExtensions
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeAuthInfo
wolfSSL Entering GetObjectId()
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeBasicCaConstraint
wolfSSL Entering GetObjectId()
Certificate Policy extension not supported yet.
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeCrlDist
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeKeyUsage
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeAltNames
Unsupported name type, skipping
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeSubjKeyId
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeAuthKeyId
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
Parsed new CA
Freeing Parsed CA
Freeing der CA
OK Freeing der CA
wolfSSL Leaving AddCA, return 0
Verifying Peer's cert
wolfSSL Entering GetExplicitVersion
wolfSSL Entering GetSerialNumber
Got Cert Header
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
Got Algo ID
Getting Cert Name
Getting Cert Name
Got Subject Name
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
wolfSSL Entering GetObjectId()
Got Key
Parsed Past Key
wolfSSL Entering DecodeCertExtensions
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeAltNames
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeBasicCaConstraint
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeKeyUsage
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeExtKeyUsage
wolfSSL Entering GetObjectId()
wolfSSL Entering GetObjectId()
wolfSSL Entering GetObjectId()
Certificate Policy extension not supported yet.
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeAuthKeyId
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeCrlDist
wolfSSL Entering GetObjectId()
wolfSSL Entering DecodeAuthInfo
wolfSSL Entering GetObjectId()
wolfSSL Entering GetObjectId()
wolfSSL Entering GetAlgoId
wolfSSL Entering GetObjectId()
About to verify certificate signature
wolfSSL Entering ConfirmSignature
wolfSSL Leaving ConfirmSignature, return 0
Verified Peer's cert
wolfSSL Leaving ProcessPeerCerts, return 0
wolfSSL Leaving DoHandShakeMsgType(), return 0
wolfSSL Leaving DoHandShakeMsg(), return 0
More messages in record
received record layer msg
wolfSSL Entering DoHandShakeMsg()
wolfSSL Entering DoHandShakeMsgType
processing server key exchange
wolfSSL Entering DoServerKeyExchange
wolfSSL Entering EccVerify
wolfSSL Leaving EccVerify, return 0
wolfSSL Leaving DoServerKeyExchange, return 0
wolfSSL Leaving DoHandShakeMsgType(), return 0
wolfSSL Leaving DoHandShakeMsg(), return 0
More messages in record
received record layer msg
wolfSSL Entering DoHandShakeMsg()
wolfSSL Entering DoHandShakeMsgType
processing certificate request
wolfSSL Leaving DoHandShakeMsgType(), return 0
wolfSSL Leaving DoHandShakeMsg(), return 0
More messages in record
received record layer msg
wolfSSL Entering DoHandShakeMsg()
wolfSSL Entering DoHandShakeMsgType
processing server hello done
wolfSSL Leaving DoHandShakeMsgType(), return 0
wolfSSL Leaving DoHandShakeMsg(), return 0
connect state: HELLO_AGAIN
connect state: HELLO_AGAIN_REPLY
connect state: FIRST_REPLY_DONE
growing output buffer
Shrinking output buffer
sent: certificate
connect state: FIRST_REPLY_FIRST
wolfSSL Entering SendClientKeyExchange
wolfSSL Entering EccMakeKey
wolfSSL Leaving EccMakeKey, return 0
wolfSSL Entering EccSharedSecret
wolfSSL Leaving EccSharedSecret, return 0
growing output buffer
Shrinking output buffer
wolfSSL Leaving SendClientKeyExchange, return 0
sent: client key exchange
connect state: FIRST_REPLY_SECOND
wolfSSL Entering SendCertificateVerify
sent: certificate verify
connect state: FIRST_REPLY_THIRD
growing output buffer
Shrinking output buffer
sent: change cipher spec
connect state: FIRST_REPLY_FOURTH
growing output buffer
wolfSSL Entering BuildMessage
wolfSSL Leaving BuildMessage, return 0
Shrinking output buffer
sent: finished
connect state: FINISHED_DONE
received record layer msg
got ALERT!
Got alert
wolfSSL error occurred, error = 42
wolfSSL error occurred, error = -313
ERROR: iot_tls_connect L#326 wolfSSL Entering SSL_get_error
wolfSSL Leaving SSL_get_error, return -313
failure creating SSL connection to server [-313]
wolfSSL Entering SSL_free
CTX ref count not 0 yet, no free
Shrinking input buffer
wolfSSL Leaving SSL_free, return 0
wolfSSL Entering SSL_CTX_free
CTX ref count down to 0, doing full free
wolfSSL Entering wolfSSL_CertManagerFree
wolfSSL Leaving SSL_CTX_free, return 0
wolfSSL Entering wolfSSL_Cleanup
wolfSSL Entering wolfCrypt_Cleanup
wolfSSL Entering SSL_shutdown()
wolfSSL Entering wolfSSL_Cleanup
ERROR: main L#197 Error(-4) connecting to a3i4lmgkxatoyr.iot.us-west-2.amazonaws.com:8883
And here is the wireshark capture while running the demo (there is a bit of noise since I am accessing the linux box over ssh; the lines of interest are the ones that communicate with 52.10.19.111):
324 43.301028126 147.34.2.16 → 134.86.9.42 DNS 546 Standard query response 0x6223 A a3i4lmgkxatoyr.iot.us-west-2.amazonaws.com CNAME iotmoonraker.us-west-2.prod.iot.us-west-2.amazonaws.com CNAME dualstack.iotmoonraker-u-elb-1w8qnw1336zq-1186348092.us-west-2.elb.amazonaws.com A 52.10.19.111 A 52.25.57.203 A 35.165.44.84 A 35.160.71.83 A 34.210.178.78 A 52.41.23.91 NS ns-560.awsdns-06.net NS ns-1475.awsdns-56.org NS ns-1
769.awsdns-29.co.uk NS ns-332.awsdns-41.com A 205.251.198.233 AAAA 2600:9000:5306:e900::1 A 205.251.193.76 AAAA 2600:9000:5301:4c00::1
325 43.301247236 134.86.9.42 → 52.10.19.111 TCP 74 40282 → 8883 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 TSval=1333683979 TSecr=0 WS=128
326 43.301402675 134.86.9.42 → 172.30.14.242 SSH 346 Server: Encrypted packet (len=292)
327 43.301516535 134.86.9.42 → 172.30.14.242 SSH 362 Server: Encrypted packet (len=308)
328 43.412499273 52.10.19.111 → 134.86.9.42 TCP 66 8883 → 40282 [SYN, ACK] Seq=0 Ack=1 Win=5840 Len=0 MSS=1360 SACK_PERM=1 WS=256
329 43.412535130 134.86.9.42 → 52.10.19.111 TCP 54 40282 → 8883 [ACK] Seq=1 Ack=1 Win=29312 Len=0
330 43.412809416 134.86.9.42 → 172.30.14.242 SSH 362 Server: Encrypted packet (len=308)
331 43.413122845 134.86.9.42 → 172.30.14.242 SSH 474 Server: Encrypted packet (len=420)
332 43.413231625 134.86.9.42 → 172.30.14.242 SSH 346 Server: Encrypted packet (len=292)
333 43.413355748 134.86.9.42 → 172.30.14.242 SSH 430 Server: Encrypted packet (len=376)
334 43.413557094 134.86.9.42 → 172.30.14.242 SSH 1066 Server: Encrypted packet (len=1012)
335 43.413863013 134.86.9.42 → 172.30.14.242 SSH 346 Server: Encrypted packet (len=292)
336 43.413988506 134.86.9.42 → 52.10.19.111 SSL 196 Client Hello
337 43.413992496 134.86.9.42 → 172.30.14.242 SSH 602 Server: Encrypted packet (len=548)
338 43.414175329 134.86.9.42 → 172.30.14.242 SSH 686 Server: Encrypted packet (len=632)
339 43.414370228 52.10.19.111 → 134.86.9.42 TCP 60 8883 → 40282 [ACK] Seq=1 Ack=143 Win=5888 Len=0
340 43.447421429 172.30.14.242 → 134.86.9.42 TCP 60 59328 → 22 [ACK] Seq=897 Ack=77873 Win=256 Len=0
341 43.533620599 52.10.19.111 → 134.86.9.42 TCP 1414 [TCP segment of a reassembled PDU]
342 43.533659332 134.86.9.42 → 52.10.19.111 TCP 54 40282 → 8883 [ACK] Seq=143 Ack=1361 Win=32128 Len=0
343 43.533830301 52.10.19.111 → 134.86.9.42 TLSv1.2 1283 Server Hello, Certificate, Server Key Exchange, Certificate Request, Server Hello Done
344 43.533876234 134.86.9.42 → 52.10.19.111 TCP 54 40282 → 8883 [ACK] Seq=143 Ack=2590 Win=35072 Len=0
345 43.533879884 134.86.9.42 → 172.30.14.242 SSH 122 Server: Encrypted packet (len=68)
346 43.534159660 134.86.9.42 → 172.30.14.242 SSH 578 Server: Encrypted packet (len=524)
347 43.534870988 134.86.9.42 → 172.30.14.242 SSH 602 Server: Encrypted packet (len=548)
348 43.535092364 134.86.9.42 → 172.30.14.242 SSH 858 Server: Encrypted packet (len=804)
349 43.545353568 134.86.9.42 → 52.10.19.111 TLSv1.2 66 Certificate
350 43.545651974 52.10.19.111 → 134.86.9.42 TCP 60 8883 → 40282 [ACK] Seq=2590 Ack=155 Win=5888 Len=0
351 43.552906179 134.86.9.42 → 52.10.19.111 TLSv1.2 129 Client Key Exchange
352 43.553226294 52.10.19.111 → 134.86.9.42 TCP 60 8883 → 40282 [ACK] Seq=2590 Ack=230 Win=5888 Len=0
353 43.553237461 134.86.9.42 → 52.10.19.111 TLSv1.2 60 Change Cipher Spec
354 43.553645295 52.10.19.111 → 134.86.9.42 TCP 60 8883 → 40282 [ACK] Seq=2590 Ack=236 Win=5888 Len=0
355 43.553661141 134.86.9.42 → 52.10.19.111 TLSv1.2 99 Hello Request, Hello Request
356 43.554058924 52.10.19.111 → 134.86.9.42 TCP 60 8883 → 40282 [ACK] Seq=2590 Ack=281 Win=5888 Len=0
357 43.560697730 172.30.14.242 → 134.86.9.42 TCP 60 59328 → 22 [ACK] Seq=897 Ack=78601 Win=253 Len=0
358 43.560714514 134.86.9.42 → 172.30.14.242 SSH 1294 Server: Encrypted packet (len=1240)
359 43.560717288 134.86.9.42 → 172.30.14.242 SSH 1294 Server: Encrypted packet (len=1240)
360 43.560718987 172.30.14.242 → 134.86.9.42 TCP 60 59328 → 22 [ACK] Seq=897 Ack=79269 Win=256 Len=0
361 43.560902106 172.30.14.242 → 134.86.9.42 TCP 60 59328 → 22 [ACK] Seq=897 Ack=80573 Win=256 Len=0
362 43.560917321 134.86.9.42 → 172.30.14.242 SSH 1294 Server: Encrypted packet (len=1240)
363 43.560920124 134.86.9.42 → 172.30.14.242 SSH 1294 Server: Encrypted packet (len=1240)
364 43.560921717 172.30.14.242 → 134.86.9.42 TCP 60 59328 → 22 [ACK] Seq=897 Ack=81753 Win=252 Len=0
365 43.561116273 134.86.9.42 → 172.30.14.242 SSH 1294 Server: Encrypted packet (len=1240)
366 43.561120211 134.86.9.42 → 172.30.14.242 SSH 758 Server: Encrypted packet (len=704)
367 43.659283864 52.10.19.111 → 134.86.9.42 TLSv1.2 61 Alert (Level: Fatal, Description: Bad Certificate)
368 43.659391034 134.86.9.42 → 52.10.19.111 TCP 54 40282 → 8883 [FIN, ACK] Seq=281 Ack=2597 Win=35072 Len=0
369 43.659472966 52.10.19.111 → 134.86.9.42 TCP 60 8883 → 40282 [FIN, ACK] Seq=2597 Ack=281 Win=5888 Len=0
370 43.659473048 134.86.9.42 → 172.30.14.242 SSH 346 Server: Encrypted packet (len=292)
371 43.659485750 134.86.9.42 → 52.10.19.111 TCP 54 40282 → 8883 [ACK] Seq=282 Ack=2598 Win=35072 Len=0
372 43.659701144 52.10.19.111 → 134.86.9.42 TCP 60 8883 → 40282 [ACK] Seq=2598 Ack=282 Win=5888 Len=0
373 43.659736873 134.86.9.42 → 172.30.14.242 SSH 826 Server: Encrypted packet (len=772)
374 43.660747207 134.86.9.42 → 172.30.14.242 SSH 282 Server: Encrypted packet (len=228)
375 43.681111175 172.30.14.242 → 134.86.9.42 TCP 60 59328 → 22 [ACK] Seq=897 Ack=82345 Win=256 Len=0
376 43.681302227 172.30.14.242 → 134.86.9.42 TCP 60 59328 → 22 [ACK] Seq=897 Ack=83697 Win=256 Len=0
377 43.705798708 172.30.14.242 → 134.86.9.42 TCP 60 59328 → 22 [ACK] Seq=897 Ack=86177 Win=256 Len=0
378 43.705811590 172.30.14.242 → 134.86.9.42 TCP 60 59328 → 22 [ACK] Seq=897 Ack=88657 Win=256 Len=0
379 43.705814480 172.30.14.242 → 134.86.9.42 TCP 60 59328 → 22 [ACK] Seq=897 Ack=90601 Win=256 Len=0
380 43.806210879 172.30.14.242 → 134.86.9.42 TCP 60 59328 → 22 [ACK] Seq=897 Ack=91665 Win=252 Len=0
381 43.814093995 134.86.9.42 → 172.30.14.242 SSH 658 Server: Encrypted packet (len=604)
382 43.847200047 172.30.14.242 → 134.86.9.42 TCP 60 59328 → 22 [ACK] Seq=897 Ack=91893 Win=256 Len=0
The exact same certificates/keys work with when using mbedTLS.
I'm obviously missing something here, might be obvious to someone too, but I'm not seeing it.
Thanks.