Here is a tcpdump capture of an unsuccessful session:
http://www.cloudshark.org/captures/b0c01d8de085
It seems that on RTMP_Connect0 the socket gets an ACK timeout and tries to retransmit two more times.
Then on RTMP_Connect1 receives a response of the two retransmitted packets in the form of a reset packet, hence fails.
Maybe forcing a SO_KEEPALIVE flag on RTMP_Connect0 to avoid retransmission solves it.