More info on the crash I've found:
After deeper debugging, I've found that the crash is happening in calloc:
Code:
RTMPPacket_Alloc(RTMPPacket *p, int nSize)
{
//Crash in calloc
char *ptr = calloc(1, nSize + RTMP_MAX_HEADER_SIZE);
if (!ptr)
return FALSE;
p->m_body = ptr + RTMP_MAX_HEADER_SIZE;
p->m_nBytesRead = 0;
return TRUE;
}
That memory corruption pointed me to run with valgrind, that showed the culprit:
Code:
==22473== Invalid write of size 1
==22473== at 0x4E421DC: HandleInvoke (rtmp.c:3333)
==22473== by 0x4E4368D: RTMP_ClientPacket (rtmp.c:1361)
==22473== by 0x4E43849: RTMP_ConnectStream (rtmp.c:1152)
==22473== by 0x403BFA: main (rtmpdump.c:1374)
==22473== Address 0x61e1c67 is 0 bytes after a block of size 39 alloc'd
==22473== at 0x4C2C04B: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==22473== by 0x4E42189: HandleInvoke (rtmp.c:3328)
==22473== by 0x4E4368D: RTMP_ClientPacket (rtmp.c:1361)
==22473== by 0x4E43849: RTMP_ConnectStream (rtmp.c:1152)
==22473== by 0x403BFA: main (rtmpdump.c:1374)
Full valgrind output
http://pastebin.com/9bAXB9W9
So it seems that the memory corruption exists due to AMF_DecodeInt24 returning incorrect size:
Code:
packet->m_nBodySize = AMF_DecodeInt24(header + 3);
and then allocating the memory, the crash occurs.
Code:
if (!RTMPPacket_Alloc(packet, packet->m_nBodySize))
Maybe KSV can enlighten me here, but I think the proof is that allocating more memory avoids the crash:
Code:
if (!RTMPPacket_Alloc(packet, 2*packet->m_nBodySize))