Talos Vulnerability Report

SUMMARY

A heap based buffer overflow vulnerability present in KeyView PDF filter as used by Domino can lead to remote arbitrary code execution.

TESTED VERSIONS

• KeyView 10.16 as used by IBM Domino 9.0.1

PRODUCT URLs

http://www-03.ibm.com/software/products/en/ibmdomino

DETAILS

While parsing an ID description of a PDF trailer, no checks are being made regarding its length. The ID, which is usually an ASCII representation of hexadecimal value, is converted into it’s byte array equivalent without size checks leading to a straight forward heap overflow which can be abused to overwrite sensitive data structures or heap metadata and subvert the process execution flow to arbitrary code execution.

Shortened test case triggering the vulnerability can be summarized as follows:

%PDF-1.3
%âãÏÓ
trailer
<<
/Root 4 0 R 
/ID[<AAAAAAAAAAAAA...(a large number of hex characters)...41414141
>>
startxref
4139
%%EOF

When initializing the parser, a large chunk of memory is allocated in PDFAllocateContext function of pdfsr.so library (132648 bytes in size to be precise). An offset into this large context memory chunk is used as a start of the buffer that ends up being overflown. The overflow happens in the following code:

.text:B79DC7DA loc_B79DC7DA:
.text:B79DC7DA mov     [ebp+var_15], al 						[1]
.text:B79DC7DD lea     eax, [edx+1] 						[2]
.text:B79DC7E0 mov     [ebp+var_F0], eax 					 	[3]
.text:B79DC7E6 movzx   eax, byte ptr [edx+1] 					
.text:B79DC7EA mov     [ebp+var_14], al 						[4]	
.text:B79DC7ED lea     eax, [edx+2]
.text:B79DC7F0 mov     [ebp+var_F0], eax
.text:B79DC7F6 mov     [ebp+var_13], 0
.text:B79DC7FA mov     dword ptr [esp+0Ch], 0 ; group
.text:B79DC802 mov     dword ptr [esp+8], 10h ; base
.text:B79DC80A mov     dword ptr [esp+4], 0 ; endptr
.text:B79DC812 lea     eax, [ebp+var_15]
.text:B79DC815 mov     [esp], eax      ; nptr
.text:B79DC818 call    ___strtol_internal 					[5]
.text:B79DC81D mov     edx, [ebp+counter]
.text:B79DC823 mov     [edx+edi+11D8h], al 					[6]
.text:B79DC82A mov     edx, [ebp+var_F0]						
.text:B79DC830 cmp     esi, edx							[7]
.text:B79DC832 jbe     loc_B79DCB16

.text:B79DC838 add     [ebp+counter], 1
.text:B79DC83F movzx   eax, byte ptr [edx]
.text:B79DC842 cmp     al, '>'							[8]
.text:B79DC844 jnz     short loc_B79 

At the beginning of the above code, at [1], al contains the first character of the ID hex string which is saved in var_15, at [2],[3] and [4] the second character is saved in var_14 which, along with var_15, makes up a string that is passed to strtol at [5]. Function strtol is called with base set to 0x10, converting from hex string into bytes. At [6] the returned value is written to the buffer. This is where the out of bounds write occurs when the counters are past the boundary of the buffer. At [7] the current string counter is checked against the file end, esi points to the end of file. If the check at [7] passes, another check is made at [8] which is looking for a terminating > character. No buffer length checks are being made.

This buffer overflow can be abused to overwrite other parts of the context structure or even, further up the heap, the heap metadata thus leading to process crash and possible arbitrary code execution.

The supplied test case is crafted in such way that the buffer overflow will overwrite a pointer on the heap which is later dereferenced during heap cleanup. The process crashes while calling free on an invalid pointer. Further control over data overwritten in the context structure can lead to arbitrary code execution.

Detection of PDF files specifically crafted to trigger this vulnerability can be based on the abnormally large length of the ID value in the PDF trailer.

The vulnerability can be triggered with the supplied test case in the filter standalone KeyView binary shipped with IBM Domino, or by sending it as an attachment with an email to a Domino mail server.

TIMELINE

2016-02-09 - Vendor Notification
2016-06-08 – Public Disclosure