Talos Vulnerability Report

DESCRIPTION

An integer overflow leading to two distinct issues can be triggered by a specially crafted PDF file.

TESTED VERSIONS

Oracle Outside In IX sdk 8.5.1

PRODUCT URLs

http://www.oracle.com/technetwork/middleware/content-management/oit-all-085236.html

DETAILS

While parsing a PDF file with specific /Size element, a memory allocation operation can fail, returning a NULL pointer due to integer overflow, which is unchecked and leads to a crash during a memset() call. A carefuly selected size value can also lead to further memory corruption.

The supplied testcase can be abbrevated to the following:

%PDF-1.3  
xref
trailer
<</Size 444444444444444444499999999999>>
startxref
4
%%EOF

A huge /Size value leads to failed memory allocation in the following basic block:

.text:B74ECE59 mov     edi, eax 					[1]
.text:B74ECE5B shl     edi, 4 						[2]
.text:B74ECE5E mov     [esp+6BCh+s], edi 
.text:B74ECE61 call    _SYSNativeAlloc              [3]
.text:B74ECE66 mov     edx, [esp+6BCh+arg_10] 		
.text:B74ECE6D mov     [edx+1D6Ch], eax				[4]
.text:B74ECE73 test    eax, eax
.text:B74ECE75 jz      loc_B7

At [1], the value in eax comes straight from the 32bit rounded value from the /Size element. At [2], it is multiplied by four therefore invalidating the integer overflow check that was done previously. A malloc wrapper is called at [3] and the returned pointer (NULL in this case) is saved at [4]. Even though the pointer is checked against NULL at the end, in a subsequent basic block it is still used as a destination for memset:

.text:B74ECE7B loc_B74ECE7B:
.text:B74ECE7B mov     ecx, [esp+6BCh+arg_10]
.text:B74ECE82 mov     eax, [ecx+1D6Ch]
.text:B74ECE88 mov     [esp+6BCh+n], edi ; n 		[1]
.text:B74ECE8C mov     [esp+6BCh+c], 0 ; c
.text:B74ECE94 mov     [esp+6BCh+s], eax ; s 		[2]
.text:B74ECE97 call    _memset 						[3]

The same size derived in the previous basic block is used at [1] as a size parameter for memset. At [2], saved pointer is retrieved and is NULL in this case. The application crashes at [3] due to invalid pointer.

If a size value is choosen carefuly, it can lead to an integer overflow at [2] in the first basic block such that a small value is passed to SYSNativeAlloc at [3]. In this case, the subsequent memset call would pass without issue. The problem arises when, due to rounding, heap allocator returns a pointer to a bigger heap chunk than requested. In this case, the memset call will initiallize only the originaly requested size, leaving the rest of the buffer uninitialized to zero. Later on in the code, this buffer is treated as a pointer array with checks for NULL pointers, but the uninitialized portion of the buffer may have non-NULL values leading to further issues.

As an example, if the size value is specified to be 0x10000001 it will pass the check before allocation in the first basic block above, but when shifted by 4, it becomes 0x10, making a small allocation. Depending on an underlying allocator, the actuall size of the allocated chunk would be bigger. In case of Linux, in this case, the returned chunk will be 24 bytes long and subsequent memset will onyl initialize the first 16 bytes.

Afterwards, the code reaches the following loop inside VwStreamClose:

.text:B74D17BB mov     esi, eax						[1]
.text:B74D17BD xor     edi, edi 					[2]
.text:B74D17BF jmp     short loc_BB74D17C4
.text:B74D17C1
.text:B74D17C1 loc_B74D17C1:
.text:B74D17C1 add     esi, 10h 					[8]
.text:B74D17C4
.text:B74D17C4 loc_B74D17C4:
.text:B74D17C4 mov     eax, [esi] 					[3]
.text:B74D17C6 test    eax, eax						
.text:B74D17C8 jz      short loc_B74D17D3 			[4]
.text:B74D17CA mov     edx, [esp+4Ch+arg_4] 		
.text:B74D17CE call    sub_B74D14C0					[5]
.text:B74D17D3
.text:B74D17D3 loc_B74D17D3:
.text:B74D17D3 add     edi, 1 						[6]
.text:B74D17D6 mov     eax, [esp+4Ch+arg_4] 		
.text:B74D17DA cmp     [eax+1D70h], edi 			[7]
.text:B74D17E0 ja      short loc_B74D17C1

At [1], a pointer to the previously allocated buffer is moved into esi and used as a starting position of a loop. At [2], a counter is initialized to 0. At [3], a pointer stored in the buffer is copied into eax and tested agains being NULL at [4]. If it’s not NULL, a usercall function who’s first argument is eax is called at [5]. After the function call, or if the pointer was NULL, a counter in edi is advanced by one and then compared to the upper bound which is the original Size value as specified in the file (before the overflow) at [7]. Finally, the code jumps back to [8], where the pointer into the buffer is increased by 16.

It is now clear that if only first 16 bytes of the buffer are initialized, when the code executes the loop for the second time, at [3] it will be accessing memory that is uninitialized to zero effectively turning this into a sort of use-after-free vulnerability. Function called at [5] deals with heap structures and, if sufficient heap control is achieved, leftover data present in uninitialized part can cause further memory corruption, potentially leading to code execution.

TIMELINE

2016-04-12 - Initial Vendor Communication
2016-07-19 - Public Disclosure