Talos Vulnerability Report


GTKWave FST fstReaderIterBlocks2 chain_table allocation integer overflow vulnerabilities

January 8, 2024
CVE Number



Multiple integer overflow vulnerabilities exist in the FST fstReaderIterBlocks2 chain_table allocation functionality of GTKWave 3.3.115. A specially crafted .fst file can lead to arbitrary code execution. A victim would need to open a malicious file to trigger these vulnerabilities.


The versions below were either tested or verified to be vulnerable by Talos or confirmed to be vulnerable by the vendor.

GTKWave 3.3.115


GTKWave - https://gtkwave.sourceforge.net


7.8 - CVSS:3.1/AV:L/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H


CWE-190 - Integer Overflow or Wraparound


GTKWave is a wave viewer, often used to analyze FPGA simulations and logic analyzer captures. It includes a GUI to view and analyze traces, as well as convert across several file formats (.lxt, .lxt2, .vzt, .fst, .ghw, .vcd, .evcd) either by using the UI or its command line tools. GTKWave is available for Linux, Windows and MacOS. Trace files can be shared within teams or organizations, for example to compare results of simulation runs across different design implementations, to analyze protocols captured with logic analyzers or just as a reference when porting design implementations.

GTKWave sets up mime types for its supported extensions. For example, it’s enough for a victim to double-click on a wave file received by e-mail to trigger the vulnerability described in this advisory.

The function fstReaderOpen is used to parse .fst files.

     void *fstReaderOpen(const char *nam)
[1]  struct fstReaderContext *xc = (struct fstReaderContext *)calloc(1, sizeof(struct fstReaderContext));
[2]  if((!nam)||(!(xc->f=fopen(nam, "rb"))))
             int flen = strlen(nam);
             char *hf = (char *)calloc(1, flen + 6);
             int rc;
     #if defined(FST_UNBUFFERED_IO)
             setvbuf(xc->f, (char *)NULL, _IONBF, 0);   /* keeps gzip from acting weird in tandem with fopen */
             memcpy(hf, nam, flen);
             strcpy(hf + flen, ".hier");
             xc->fh = fopen(hf, "rb");
             xc->filename = strdup(nam);
[3]          rc = fstReaderInit(xc);

At [1] the xc structure is created. This big (about 65KB) structure is used to hold information regarding the .fst file.
At [2] the xc->f field is set to the file pointer returned by fopen after opening the file being parsed.

Then, the function fstReaderInit is called [3] to parse the various .fst sections from the input file.

     int fstReaderInit(struct fstReaderContext *xc)
     fst_off_t blkpos = 0;
     fst_off_t endfile;
     uint64_t seclen;
     int sectype;
     uint64_t vc_section_count_actual = 0;
     int hdr_incomplete = 0;
     int hdr_seen = 0;
     int gzread_pass_status = 1;

     sectype = fgetc(xc->f);
             fstReaderFseeko(xc, xc->f, 0, SEEK_END);
             endfile = ftello(xc->f);

             while(blkpos < endfile)
                     fstReaderFseeko(xc, xc->f, blkpos, SEEK_SET);

[4]                  sectype = fgetc(xc->f);
                     seclen = fstReaderUint64(xc->f);

                     if(sectype == EOF)

[5]                  if((hdr_incomplete) && (!seclen))

[6]                  if(!hdr_seen && (sectype != FST_BL_HDR))


At [4], the sectype (1 byte) is read off the file. This variable will tell us which kind of sector we’re parsing. The checks at [5] and [6] are important because we don’t want to trigger them in order to reach the vulnerable code: we need to make sure hdr_incomplete stays 0 and we start with a FST_BL_HDR sector, which will set hdr_seen to 1.

So, if the input file starts with a 0x00, we’ll land inside the following condition:

     if(sectype == FST_BL_HDR)
                     int ch;
                     double dcheck;

                     xc->start_time = fstReaderUint64(xc->f);
                     xc->end_time = fstReaderUint64(xc->f);

[7]                  hdr_incomplete = (xc->start_time == 0) && (xc->end_time == 0);

[8]                  hdr_seen = 1;

[9]                  xc->mem_used_by_writer = fstReaderUint64(xc->f);
                     xc->scope_count = fstReaderUint64(xc->f);
                     xc->var_count = fstReaderUint64(xc->f);

fstReaderUint64 is used to read a 64-bit unsigned integer (big endian) off the file, incrementing the current file offset.
We need to make sure that either start_time or end_time are not 0, to not set hdr_incomplete [7].
At [8] hdr_seen is set, so we can pass the check at [6] when parsing the next sections.
Finally at [9], other values are read off the file and saved into the xc structure.

After parsing the header, fstReaderInit then proceeds to parse other blocks which are not relevant for this advisory.

Upon return from fstReaderOpen, GTKWave eventually calls fstReaderIterBlocks2 to parse all VCDATA blocks, as they haven’t been fully parsed in fstReaderInit.

fstReaderIterBlocks2 is a pretty large function, so we’ll focus on the interesting lines only:

     int fstReaderIterBlocks2(void *ctx,
                              void (*value_change_callback)(void *user_callback_data_pointer, uint64_t time, fstHandle facidx, const unsigned char *value),
                              void (*value_change_callback_varlen)(void *user_callback_data_pointer, uint64_t time, fstHandle facidx, const unsigned char *value, uint32_t len),
                              void *user_callback_data_pointer, FILE *fv) {
         struct fstReaderContext *xc = (struct fstReaderContext *)ctx;

         uint64_t previous_time = UINT64_MAX;
         uint64_t *time_table = NULL;
         uint64_t tsec_nitems;
         unsigned int secnum = 0;
         int blocks_skipped = 0;
         fst_off_t blkpos = 0;
         uint64_t seclen, beg_tim;
         uint64_t end_tim;
         uint64_t frame_uclen, frame_clen, frame_maxhandle, vc_maxhandle;
         fst_off_t vc_start;
         fst_off_t indx_pntr, indx_pos;
         fst_off_t *chain_table = NULL;
         uint32_t *chain_table_lengths = NULL;
         unsigned char *chain_cmem;
         unsigned char *pnt;
         long chain_clen;
         fstHandle idx, pidx = 0, i;
         uint64_t pval;
         uint64_t vc_maxhandle_largest = 0;
         uint64_t tsec_uclen = 0, tsec_clen = 0;
         int sectype;
         uint64_t mem_required_for_traversal;
         unsigned char *mem_for_traversal = NULL;
         uint32_t traversal_mem_offs;
         uint32_t *scatterptr, *headptr, *length_remaining;
         uint32_t cur_blackout = 0;
         int packtype;
         unsigned char *mc_mem = NULL;
         uint32_t mc_mem_len; /* corresponds to largest value encountered in chain_table_lengths[i] */
         int dumpvars_state = 0;

[10]     for (;;) {
             uint32_t *tc_head = NULL;
             traversal_mem_offs = 0;

             fstReaderFseeko(xc, xc->f, blkpos, SEEK_SET);

[11]         sectype = fgetc(xc->f);
             seclen = fstReaderUint64(xc->f);

             if ((sectype == EOF) || (sectype == FST_BL_SKIP)) {
     #ifdef FST_DEBUG
                 fprintf(stderr, FST_APIMESS "<< EOF >>\n");

[12]         if ((sectype != FST_BL_VCDATA) && (sectype != FST_BL_VCDATA_DYN_ALIAS) && (sectype != FST_BL_VCDATA_DYN_ALIAS2)) {
                 blkpos += seclen;

             if (!seclen) break;

As previously said, all blocks are scanned [10], and only the blocks with a sectype [11] containing VCDATA are parsed [12]. These blocks are FST_BL_VCDATA, FST_BL_VCDATA_DYN_ALIAS and FST_BL_VCDATA_DYN_ALIAS2.

Later on, the code extracts vc_maxhandle, declared as a uint64_t variable:

        fstReaderFseeko(xc, xc->f, (fst_off_t)frame_clen, SEEK_CUR); /* skip past compressed data */

[13]    vc_maxhandle = fstReaderVarint64(xc->f);
        vc_start = ftello(xc->f); /* points to '!' character */
        packtype = fgetc(xc->f);

Right after this, the chain_clen is extracted [14], which is the size of the chain section, extracted at [15].

        indx_pntr = blkpos + seclen - 24 - tsec_clen - 8;
        fstReaderFseeko(xc, xc->f, indx_pntr, SEEK_SET);
[14]    chain_clen = fstReaderUint64(xc->f);
        indx_pos = indx_pntr - chain_clen;
[15]    chain_cmem = (unsigned char *)malloc(chain_clen);
        if (!chain_cmem) goto block_err;
        fstReaderFseeko(xc, xc->f, indx_pos, SEEK_SET);
        fstFread(chain_cmem, chain_clen, 1, xc->f);

        if (vc_maxhandle > vc_maxhandle_largest) {

            vc_maxhandle_largest = vc_maxhandle;
[16]        chain_table = (fst_off_t *)calloc((vc_maxhandle + 1), sizeof(fst_off_t));
[17]        chain_table_lengths = (uint32_t *)calloc((vc_maxhandle + 1), sizeof(uint32_t));

The chain_table and chain_table_lengths buffers are cached across the parsing of the different FST blocks. However, initially chain_table and chain_table_lengths will be zero, so the initialization happens at [16] and [17], allocating vc_maxhandle+1 items. vc_maxhandle should correspond to the number of VC elements in VCDATA.

Note that the first parameter for calloc (number of elements) is set as vc_maxhandle + 1. However, as we can control vc_maxhandle arbitrarily, its value could be -1 (0xffffffffffffffff for 64-bit and 0xffffffff for 32-bit). This would make the addition wrap around, leading to calling calloc(0, 8), which results in allocating a zero (or small, depending on the malloc implementation) sized buffer.

Moreover, calloc internally multiplies vc_maxhandle+1 by sizeof(fst_off_t), which could lead to an integer overflow, depending on the calloc implementation: some implementations (for example glibc) will detect the overflow and return 0, leading to just crashing GTKWave. If the calloc implementation, however, doesn’t detect the overflow, calloc may end up allocating a very small buffer. For example, on 64-bit systems, a vc_maxhandle value of 0x1fffffffffffffff (or 0x1fffffff for 32-bit systems) will lead to allocating a small buffer (again, the actual size depends on the implementation).

Later on, these tables are written to:

     pnt = chain_cmem;
     idx = 0;
     pval = 0;

     if (sectype == FST_BL_VCDATA_DYN_ALIAS2) {
[18] } else {
         do {
             int skiplen;
[20]         uint64_t val = fstGetVarint32(pnt, &skiplen);

[21]         if (!val) {
                 pnt += skiplen;
                 val = fstGetVarint32(pnt, &skiplen);
[22]             chain_table[idx] = 0;            /* need to explicitly zero as calloc above might not run */
[23]             chain_table_lengths[idx] = -val; /* because during this loop iter would give stale data! */
             } else if (val & 1) {
[22]             pval = chain_table[idx] = pval + (val >> 1);
                 if (idx) {
[23]                 chain_table_lengths[pidx] = pval - chain_table[pidx];
                 pidx = idx++;
             } else {
                 fstHandle loopcnt = val >> 1;
                 for (i = 0; i < loopcnt; i++) {
[22]                 chain_table[idx++] = 0;

             pnt += skiplen;
[19]     } while (pnt != (chain_cmem + chain_clen));

[22] chain_table[idx] = indx_pos - vc_start;
[23] chain_table_lengths[pidx] = chain_table[idx] - chain_table[pidx];

For FST_BL_VCDATA and FST_BL_VCDATA_DYN_ALIAS sections, we enter the block at [18]. At each iteration of the loop [19], a varint is read [20], which eventually leads to a controlled write in both the chain_table [22] and chain_table_lengths arrays.

The loop termination depends on chain_clen, which is independent from vc_maxhandle. This means that an attacker can allocate a very small buffer at [16] and [17] by controlling vc_maxhandle and later write out-of-bounds an arbitrary amount of data (controlled by chain_clen) with arbitrary contents out-of-bounds in the heap. An attacker can exploit this to execute arbitrary code.

CVE-2023-36915 - fstReaderIterBlocks2 chain_table allocation

By not checking the value of vc_maxhandle, the calculation at [16] may overflow (or the calloc calculation may overflow). The loop at [19], controlled by an independent value chain_clen, can be used to control the amount of bytes written in the heap, while the contents are controlled with varints read at [20]. An attacker can exploit this to execute arbitrary code.

CVE-2023-36916 - fstReaderIterBlocks2 chain_table_lengths allocation

By not checking the value of vc_maxhandle, the calculation at [16] may overflow (or the calloc calculation may overflow). The loop at [19], controlled by an independent value chain_clen, can be used to control the amount of bytes written in the heap, while the contents are controlled with varints read at [20]. An attacker can exploit this to execute arbitrary code.


Fixed in version 3.3.118, available from https://sourceforge.net/projects/gtkwave/files/gtkwave-3.3.118/


2023-07-10 - Initial Vendor Contact
2023-07-18 - Vendor Disclosure
2023-12-31 - Vendor Patch Release
2024-01-08 - Public Release


Discovered by Claudio Bozzato of Cisco Talos.