Talos Vulnerability Report


GTKWave FST fstReaderIterBlocks2 vesc allocation integer overflow vulnerability

January 8, 2024
CVE Number



An integer overflow vulnerability exists in the FST fstReaderIterBlocks2 vesc allocation functionality of GTKWave 3.3.115, when compiled as a 32-bit binary. A specially crafted .fst file can lead to memory corruption. A victim would need to open a malicious file to trigger this vulnerability.


The versions below were either tested or verified to be vulnerable by Talos or confirmed to be vulnerable by the vendor.

GTKWave 3.3.115


GTKWave - https://gtkwave.sourceforge.net


7.0 - CVSS:3.1/AV:L/AC:H/PR:N/UI:R/S:U/C:H/I:H/A:H


CWE-190 - Integer Overflow or Wraparound


GTKWave is a wave viewer, often used to analyze FPGA simulations and logic analyzer captures. It includes a GUI to view and analyze traces, as well as convert across several file formats (.lxt, .lxt2, .vzt, .fst, .ghw, .vcd, .evcd) either by using the UI or its command line tools. GTKWave is available for Linux, Windows and MacOS. Trace files can be shared within teams or organizations, for example to compare results of simulation runs across different design implementations, to analyze protocols captured with logic analyzers or just as a reference when porting design implementations.

GTKWave sets up mime types for its supported extensions. For example, it’s enough for a victim to double-click on a wave file received by e-mail to trigger the vulnerability described in this advisory.

The function fstReaderOpen is used to parse .fst files.

     void *fstReaderOpen(const char *nam)
[1]  struct fstReaderContext *xc = (struct fstReaderContext *)calloc(1, sizeof(struct fstReaderContext));
[2]  if((!nam)||(!(xc->f=fopen(nam, "rb"))))
             int flen = strlen(nam);
             char *hf = (char *)calloc(1, flen + 6);
             int rc;
     #if defined(FST_UNBUFFERED_IO)
             setvbuf(xc->f, (char *)NULL, _IONBF, 0);   /* keeps gzip from acting weird in tandem with fopen */
             memcpy(hf, nam, flen);
             strcpy(hf + flen, ".hier");
             xc->fh = fopen(hf, "rb");
             xc->filename = strdup(nam);
[3]          rc = fstReaderInit(xc);

At [1] the xc structure is created. It is a big (about 65KB) structure used to hold information regarding the .fst file.
At [2] the xc->f field is set to the file pointer returned by fopen after opening the file being parsed.

Then, the function fstReaderInit is called [3] to parse the various .fst sections from the input file.

     int fstReaderInit(struct fstReaderContext *xc)
     fst_off_t blkpos = 0;
     fst_off_t endfile;
     uint64_t seclen;
     int sectype;
     uint64_t vc_section_count_actual = 0;
     int hdr_incomplete = 0;
     int hdr_seen = 0;
     int gzread_pass_status = 1;

     sectype = fgetc(xc->f);
             fstReaderFseeko(xc, xc->f, 0, SEEK_END);
             endfile = ftello(xc->f);

             while(blkpos < endfile)
                     fstReaderFseeko(xc, xc->f, blkpos, SEEK_SET);

[4]                  sectype = fgetc(xc->f);
                     seclen = fstReaderUint64(xc->f);

                     if(sectype == EOF)

[5]                  if((hdr_incomplete) && (!seclen))

[6]                  if(!hdr_seen && (sectype != FST_BL_HDR))


At [4], the sectype (1 byte) is read off the file. This variable will tell us which kind of sector we’re parsing. The checks at [5] and [6] are important because we don’t want to trigger them in order to reach the vulnerable code: we need to make sure hdr_incomplete stays 0 and that we start with a FST_BL_HDR sector, which will set hdr_seen to 1.

So, if the input file starts with a 0x00, we’ll land inside the following condition:

     if(sectype == FST_BL_HDR)
                     int ch;
                     double dcheck;

                     xc->start_time = fstReaderUint64(xc->f);
                     xc->end_time = fstReaderUint64(xc->f);

[7]                  hdr_incomplete = (xc->start_time == 0) && (xc->end_time == 0);

[8]                  hdr_seen = 1;

[9]                  xc->mem_used_by_writer = fstReaderUint64(xc->f);
                     xc->scope_count = fstReaderUint64(xc->f);
                     xc->var_count = fstReaderUint64(xc->f);

After parsing the header, fstReaderInit then proceeds to parse other blocks, among them, the FST_BL_GEOM block:

     else if(sectype == FST_BL_GEOM)
                     uint64_t clen = seclen - 24;
                     uint64_t uclen = fstReaderUint64(xc->f);
                     unsigned char *ucdata = (unsigned char *)malloc(uclen);
                     unsigned char *pnt = ucdata;
                     unsigned int i;

                     xc->contains_geom_section = 1;
                     xc->maxhandle = fstReaderUint64(xc->f);
                     xc->longest_signal_value_len = 32; /* arbitrarily set at 32...this is much longer than an expanded double */

                     xc->process_mask = (unsigned char *)calloc(1, (xc->maxhandle+7)/8);

[10]                 if(clen != uclen)
                             unsigned char *cdata = (unsigned char *)malloc(clen);
                             unsigned long destlen = uclen;
                             unsigned long sourcelen = clen;
                             int rc;

                             fstFread(cdata, clen, 1, xc->f);
                             rc = uncompress(ucdata, &destlen, cdata, sourcelen);

                             if(rc != Z_OK)
                                     fprintf(stderr, FST_APIMESS "fstReaderInit(), geom uncompress rc = %d, exiting.\n", rc);

                             fstFread(ucdata, uclen, 1, xc->f);

[11]                 xc->signal_lens = (uint32_t *)malloc(sizeof(uint32_t) * xc->maxhandle);
                     xc->signal_typs = (unsigned char *)malloc(sizeof(unsigned char) * xc->maxhandle);

                             int skiplen;
[12]                         uint64_t val = fstGetVarint32(pnt, &skiplen);

                             pnt += skiplen;

[13]                                 xc->signal_lens[i] = (val != 0xFFFFFFFF) ? val : 0;
                                     xc->signal_typs[i] = FST_VT_VCD_WIRE;
                                     if(xc->signal_lens[i] > xc->longest_signal_value_len)
                                             xc->longest_signal_value_len = xc->signal_lens[i];

At [10], the FST_BL_GEOM section is optionally decompressed with zlib. At [11] the signal_lens buffer is allocated, with xc->maxhandle elements. Finally, the signal_lens is filled [13] by reading 32-bit LEB128-encoded integers [12] from file.

If there’s no FST_BL_GEOM section, the signal_lens is filled by fstReaderProcessHier.

Upon return from fstReaderOpen, gtkwave eventually calls fstReaderIterBlocks2.
If, however, the file is opened by fst2vcd, a call to fstReaderProcessHier happens first. In that case, the “hier” section will overwrite any signal_lens structure defined in the FST_BL_GEOM section.

The way fstReaderProcessHier is processed is of interest for this advisory. It suffices to say that it’s an alternative way to fill signal_lens, and the section can be zlib-compressed (FST_BL_HIER), lz4-compressed (FST_BL_HIER_LZ4) or lz4-compressed twice (FST_BL_HIER_LZ4DUO).

Eventually, fstReaderIterBlocks2 is called to parse all VCDATA blocks, as they haven’t been fully parsed in fstReaderInit.

fstReaderIterBlocks2 is a pretty large function, so we’ll focus on the interesting lines only:

     int fstReaderIterBlocks2(void *ctx,
                              void (*value_change_callback)(void *user_callback_data_pointer, uint64_t time, fstHandle facidx, const unsigned char *value),
                              void (*value_change_callback_varlen)(void *user_callback_data_pointer, uint64_t time, fstHandle facidx, const unsigned char *value, uint32_t len),
                              void *user_callback_data_pointer, FILE *fv) {
         struct fstReaderContext *xc = (struct fstReaderContext *)ctx;

         uint64_t previous_time = UINT64_MAX;
         uint64_t *time_table = NULL;
         uint64_t tsec_nitems;
         unsigned int secnum = 0;
         int blocks_skipped = 0;
         fst_off_t blkpos = 0;
         uint64_t seclen, beg_tim;
         uint64_t end_tim;
         uint64_t frame_uclen, frame_clen, frame_maxhandle, vc_maxhandle;
         fst_off_t vc_start;
         fst_off_t indx_pntr, indx_pos;
         fst_off_t *chain_table = NULL;
         uint32_t *chain_table_lengths = NULL;
         unsigned char *chain_cmem;
         unsigned char *pnt;
         long chain_clen;
         fstHandle idx, pidx = 0, i;
         uint64_t pval;
         uint64_t vc_maxhandle_largest = 0;
         uint64_t tsec_uclen = 0, tsec_clen = 0;
         int sectype;
         uint64_t mem_required_for_traversal;
         unsigned char *mem_for_traversal = NULL;
         uint32_t traversal_mem_offs;
         uint32_t *scatterptr, *headptr, *length_remaining;
         uint32_t cur_blackout = 0;
         int packtype;
         unsigned char *mc_mem = NULL;
         uint32_t mc_mem_len; /* corresponds to largest value encountered in chain_table_lengths[i] */
         int dumpvars_state = 0;

[14]     for (;;) {
             uint32_t *tc_head = NULL;
             traversal_mem_offs = 0;

             fstReaderFseeko(xc, xc->f, blkpos, SEEK_SET);

[15]         sectype = fgetc(xc->f);
             seclen = fstReaderUint64(xc->f);

             if ((sectype == EOF) || (sectype == FST_BL_SKIP)) {
     #ifdef FST_DEBUG
                 fprintf(stderr, FST_APIMESS "<< EOF >>\n");

[16]         if ((sectype != FST_BL_VCDATA) && (sectype != FST_BL_VCDATA_DYN_ALIAS) && (sectype != FST_BL_VCDATA_DYN_ALIAS2)) {
                 blkpos += seclen;

             if (!seclen) break;

As previously said, all blocks are scanned [14], and only the blocks with a sectype [15] containing VCDATA are parsed [16]. These blocks are FST_BL_VCDATA, FST_BL_VCDATA_DYN_ALIAS and FST_BL_VCDATA_DYN_ALIAS2.

 mem_required_for_traversal = fstReaderUint64(xc->f);
 mem_for_traversal = (unsigned char *)malloc(mem_required_for_traversal + 66); /* add in potential fastlz overhead */

Later the mem_required_for_traversal is a 64-bit (big endian) integer, taken directly from the .FST file and used to allocate the mem_for_traversal buffer. This buffer will be used to parse the VC (Value Change) data off the input file.

Later on, the code extracts the start of the VCDATA section [17] and the compression type [18]:

        fstReaderFseeko(xc, xc->f, (fst_off_t)frame_clen, SEEK_CUR); /* skip past compressed data */

        vc_maxhandle = fstReaderVarint64(xc->f);
[17]    vc_start = ftello(xc->f); /* points to '!' character */
[18]    packtype = fgetc(xc->f);

Right after this, the chain_clen is extracted, which is the size of the chain section, extracted at [20].

        indx_pntr = blkpos + seclen - 24 - tsec_clen - 8;
        fstReaderFseeko(xc, xc->f, indx_pntr, SEEK_SET);
[19]    chain_clen = fstReaderUint64(xc->f);
        indx_pos = indx_pntr - chain_clen;
[20]    chain_cmem = (unsigned char *)malloc(chain_clen);
        if (!chain_cmem) goto block_err;
        fstReaderFseeko(xc, xc->f, indx_pos, SEEK_SET);
        fstFread(chain_cmem, chain_clen, 1, xc->f);

        if (vc_maxhandle > vc_maxhandle_largest) {

            vc_maxhandle_largest = vc_maxhandle;
[21]        chain_table = (fst_off_t *)calloc((vc_maxhandle + 1), sizeof(fst_off_t));
[22]        chain_table_lengths = (uint32_t *)calloc((vc_maxhandle + 1), sizeof(uint32_t));

The chain_table is cached across the parsing of the different FST blocks. However, initially chain_table will be zero, so the initialization happens at [21], allocating vc_maxhandle+1 items. vc_maxhandle corresponds to the number of VC elements in VCDATA.

The chain section and VCDATA have different formats depending on the sectype. In the case of a FST_BL_VCDATA sectype, chain_cmem will contain a series of offsets into VCDATA, to point at each VC element. The way the chain_table is parsed is not of interest for this advisory, so let’s skip over to how the VCDATA elements are extracted:

     mc_mem_len = 16384;
     mc_mem = (unsigned char *)malloc(mc_mem_len); /* buffer for compressed reads */

     /* check compressed VC data */
     if (idx > xc->maxhandle) idx = xc->maxhandle;
[23] for (i = 0; i < idx; i++) {
         if (chain_table[i]) {
             int process_idx = i / 8;
             int process_bit = i & 7;

             if (xc->process_mask[process_idx] & (1 << process_bit)) {
                 int rc = Z_OK;
                 uint32_t val;
                 uint32_t skiplen;
                 uint32_t tdelta;

[24]             fstReaderFseeko(xc, xc->f, vc_start + chain_table[i], SEEK_SET);
                 val = fstReaderVarint32WithSkip(xc->f, &skiplen);
                 if (val) {
[25]                 unsigned char *mu = mem_for_traversal + traversal_mem_offs; /* uncomp: dst */
                     unsigned char *mc;                                          /* comp:   src */
[26]                 unsigned long destlen = val;
[27]                 unsigned long sourcelen = chain_table_lengths[i];

                     if (mc_mem_len < chain_table_lengths[i]) {
                         mc_mem = (unsigned char *)malloc(mc_mem_len = chain_table_lengths[i]);
                     mc = mc_mem;

[28]                 fstFread(mc, chain_table_lengths[i], 1, xc->f);

[29]                 switch (packtype) {
                         case '4':
[30]                         rc = (destlen == (unsigned long)LZ4_decompress_safe_partial((char *)mc, (char *)mu, sourcelen, destlen, destlen)) ? Z_OK : Z_DATA_ERROR;
                         case 'F':
[31]                         fastlz_decompress(mc, sourcelen, mu, destlen); /* rc appears unreliable */
[32]                         rc = uncompress(mu, &destlen, mc, sourcelen);

                     /* data to process is for(j=0;j<destlen;j++) in mu[j] */
[34]                 headptr[i] = traversal_mem_offs;
                     length_remaining[i] = val;
                     traversal_mem_offs += val;
                 } else {
                     int destlen = chain_table_lengths[i] - skiplen;
                     unsigned char *mu = mem_for_traversal + traversal_mem_offs;
[33]                 fstFread(mu, destlen, 1, xc->f);
                     /* data to process is for(j=0;j<destlen;j++) in mu[j] */
[34]                 headptr[i] = traversal_mem_offs;
                     length_remaining[i] = destlen;
                     traversal_mem_offs += destlen;

                 if (rc != Z_OK) {
                     fprintf(stderr, FST_APIMESS "fstReaderIterBlocks2(), fac: %d clen: %d (rc=%d), exiting.\n", (int)i, (int)val, rc);

[35]             if (xc->signal_lens[i] == 1) {
                     uint32_t vli = fstGetVarint32NoSkip(mem_for_traversal + headptr[i]);
                     uint32_t shcnt = 2 << (vli & 1);
                     tdelta = vli >> shcnt;
                 } else {
                     uint32_t vli = fstGetVarint32NoSkip(mem_for_traversal + headptr[i]);
                     tdelta = vli >> 1;

[36]             scatterptr[i] = tc_head[tdelta];
[37]             tc_head[tdelta] = i + 1;

The point of the code above is to decompress or copy each element of VCDATA into the mem_for_traversal buffer, then store the time values for each VCDATA element.

For each element [23] in VCDATA, the file cursor is set to the relative element within VCDATA [24], using the chain table previously built. The first value in VCDATA tells us if we’re dealing with a compressed or uncompressed VC element.

If val is not 0, then there’s compression and val represents destlen (see below).
Then mu buffer (u stands for uncompressed) is set to mem_for_traversal (traversal_mem_offs is initially 0). Recall that we have full control of the size of mem_for_traversal via a specific field (mem_required_for_traversal) in the .fst file, and this field does not depend on nor influence any of the chain table fields.

The mc buffer (c stands for compressed) is also allocated, with a size of sourcelen.

At [26], the destlen for the mu buffer is set from val; this is a 32-bit varint. This is the uncompressed size of the current VC element.
At [27], the sourcelen is set from the precalculated chain_table. This is the compressed size of the current VC element.
At [28], the compressed VC element is read into mc.

At [29], depending on the compression type, the mc (compressed) buffer is decompressed into mu at [30], [31] and [32]. Similarly, if there’s no compression when reading val at [24], we’ll land at [33]. In this case, mu is simply populated directly from file.

After the uncompressed VCDATA has been copied into mu, the offset for this element is saved into headptr[i], and it is parsed at [35], depending on the signal_lens for the current index. A tdelta is then initialized by storing the time value directly from file into tc_head[tdelta].

After initialization of the tc_head elements, the tc_head array is scanned:

         while (tc_head[i]) {
             idx = tc_head[i] - 1;
[38]         vli = fstGetVarint32(mem_for_traversal + headptr[idx], &skiplen);

             if (xc->signal_lens[idx] <= 1) {
                 if (xc->signal_lens[idx] == 1) {
[39]             } else {
                     unsigned char *vdata;
                     uint32_t len;

[40]                 vli = fstGetVarint32(mem_for_traversal + headptr[idx], &skiplen);
                     len = fstGetVarint32(mem_for_traversal + headptr[idx] + skiplen, &skiplen2);
                     /* tdelta = vli >> 1; */ /* scan-build */
                     skiplen += skiplen2;
                     vdata = mem_for_traversal + headptr[idx] + skiplen;

[41]                 if (!(vli & 1)) {
                         if (value_change_callback_varlen) {
                             value_change_callback_varlen(user_callback_data_pointer, time_table[i], idx + 1, vdata, len);
                         } else {
                             if (fv) {
                                 char vcd_id[16];
                                 int vcdid_len;

                                 vcd_id[0] = 's';
                                 fstWritex(xc, vcd_id, 1);

                                 vcdid_len = fstVcdIDForFwrite(vcd_id + 1, idx + 1);
[42]                                 unsigned char *vesc = (unsigned char *)malloc(len * 4 + 1);
[43]                                 int vlen = fstUtilityBinToEsc(vesc, vdata, len);
                                     fstWritex(xc, vesc, vlen);

                                 vcd_id[0] = ' ';
                                 vcd_id[vcdid_len + 1] = '\n';
                                 fstWritex(xc, vcd_id, vcdid_len + 2);

First a varint is read from the current index [38]. If signal_lens for the current index is 0, we enter the block at [39].
At [40], vli and len are read from file as 32-bit varints.
If the LSB of vli is 0 [41], we eventually reach [42] where vesc is allocated with a size of len * 4 + 1. As len is controlled from file, this multiplication can wrap around if the code is running in 32-bit mode. Note that since len is declared as uint32_t, this issue only affects code compiled as 32-bit.
For example, by setting len to the value 0x40000000, malloc(1) will be called, allocating a very small buffer for vesc (depending on the malloc implementation).

After vesc is allocated, fstUtilityBinToEsc is called, passing vesc as destination, vdata as source and the arbitrary len field as length argument.

     int fstUtilityBinToEsc(unsigned char *d, const unsigned char *s, int len) {
         const unsigned char *src = s;
         unsigned char *dst = d;
         unsigned char val;
         int i;

         for (i = 0; i < len; i++) {
             switch (src[i]) {
                 case '\a':
                     *(dst++) = '\\';
                     *(dst++) = 'a';
                 case '\b':
                     *(dst++) = '\\';
                     *(dst++) = 'b';
                     if ((src[i] > ' ') && (src[i] <= '~')) /* no white spaces in output */
                         *(dst++) = src[i];
                     } else {
                         val = src[i];
                         *(dst++) = '\\';
                         *(dst++) = (val / 64) + '0';
                         val = val & 63;
                         *(dst++) = (val / 8) + '0';
                         val = val & 7;
                         *(dst++) = (val) + '0';

         return (dst - d);

fstUtilityBinToEsc is simply escaping the bytes passed in the s and storing them in the d buffer. As the vesc buffer is too small for len (because of the integer wrap-around), any write inside fstUtilityBinToEsc can write out-of-bounds on the heap, leading to memory corruption. Because of the multi-threaded nature of GTKWave, an attacker may be able to exploit this issue to execute arbitrary code.

Crash Information

==14733==ERROR: AddressSanitizer: heap-buffer-overflow on address 0xf5f00531 at pc 0x565ae9af bp 0xffffd1e8 sp 0xffffd1dc
WRITE of size 1 at 0xf5f00531 thread T0
    #0 0x565ae9ae in fstUtilityBinToEsc fst/fstapi.c:6901
    #1 0x565a4b43 in fstReaderIterBlocks2 fst/fstapi.c:5731
    #2 0x5659ea84 in fstReaderIterBlocks fst/fstapi.c:4999
    #3 0x5655908d in main src/helpers/fst2vcd.c:185
    #4 0xf7659294 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #5 0xf7659357 in __libc_start_main_impl ../csu/libc-start.c:381
    #6 0x56558686 in _start (fst2vcd+0x3686)

0xf5f00531 is located 0 bytes to the right of 1-byte region [0xf5f00530,0xf5f00531)
allocated by thread T0 here:
    #0 0xf7a55ffb in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x565a4b1f in fstReaderIterBlocks2 fst/fstapi.c:5730
    #2 0x5659ea84 in fstReaderIterBlocks fst/fstapi.c:4999
    #3 0x5655908d in main src/helpers/fst2vcd.c:185
    #4 0xf7659294 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58

SUMMARY: AddressSanitizer: heap-buffer-overflow fst/fstapi.c:6901 in fstUtilityBinToEsc
Shadow bytes around the buggy address:
  0x3ebe0050: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x3ebe0060: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x3ebe0070: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x3ebe0080: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x3ebe0090: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x3ebe00a0: fa fa fa fa fa fa[01]fa fa fa 00 04 fa fa 02 fa
  0x3ebe00b0: fa fa fd fa fa fa 00 fa fa fa 00 fa fa fa 00 fa
  0x3ebe00c0: fa fa 01 fa fa fa 02 fa fa fa 00 fa fa fa fd fa
  0x3ebe00d0: fa fa fd fa fa fa fd fa fa fa fd fa fa fa 00 04
  0x3ebe00e0: fa fa 00 03 fa fa 00 04 fa fa 00 05 fa fa 00 04
  0x3ebe00f0: fa fa 00 05 fa fa 00 00 fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb

Fixed in version 3.3.118, available from https://sourceforge.net/projects/gtkwave/files/gtkwave-3.3.118/


2023-07-10 - Initial Vendor Contact
2023-07-18 - Vendor Disclosure
2023-12-31 - Vendor Patch Release
2024-01-08 - Public Release


Discovered by Claudio Bozzato of Cisco Talos.