Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8184306

zlib 1.2.11 upgrade triggers j.u.zip.Deflater regression

    Details

    • Subcomponent:
    • Resolved In Build:
      b179
    • OS:
      windows
    • Verification:
      Verified

      Backports

        Description

        It appears the j.u.Deflater compression functionality is broken when

         (1) level/strategy is changed (to new value) + (2) reset,()

        which is a normal combination/use scenario when the deflater is being cached and reused. The attached test case fails with 1.2.11 (with data size > the DeflaterOutputStream's internal buffer size).

        My reading of the 1.2.11 changes suggests the direct trigger is that the internal state "deflate_state.high_water" is not being reset correctly/appropriately in deflateReset/deflateResetKeep/lm_init().

        It triggers a regression from 1.2.8 to 1.2.11 in the scenario that the "strategy/levle is changed" AND a followup "reset" is called (withe the assumption that everything should be reset back to initial state and we start a new compression circle, with the new level and strategy. This is how it worked in 1.2.8). Because one of the condition check is changed from "strm->total_in!=0" to "s->high_water" (for the flush last buffer" branch) in deflate.c/deflateParams().

        from (1.2.8's)

        if ((strategy != s->strategy || func != configuration_table[level].func) &&
            strm->total_in != 0) {
            /* Flush the last buffer: */
            err = deflate(strm, Z_BLOCK);
            if (err == Z_BUF_ERROR && s->pending == 0)
                err = Z_OK;
        }

        to (1.2.11's)

        if ((strategy != s->strategy || func != configuration_table[level].func) &&
            s->high_water) {
            /* Flush the last buffer: */
            int err = deflate(strm, Z_BLOCK);
            if (err == Z_STREAM_ERROR)
                return err;
            if (strm->avail_out == 0)
                return Z_BUF_ERROR;
        }

        Given the nature of "reset" it seems reasonable to assume the "high_water" should be reset to 0, its initial value as well, inside either deflateResetKeep() or lm_init().

        I have logged an issue at

        https://github.com/madler/zlib/issues/275

        Given jdk9 has been changed to uses the system zlib for all platforms except the windows, this one currently only fails on windows (but I would assume it will fails if the underlying system zlib is 1.2.11)

        -------------------------------- test case ----------------------------------------------------
           public static void main(final String[] args) throws IOException {
                Path p = Paths.get("jms.compression.data.large");
                int[] levels = new int[] {
                    Deflater.DEFAULT_COMPRESSION,
                    0,
                    Deflater.BEST_SPEED,
                    // 2, 3, 4, 5, 6, 7, 8,
                    Deflater.BEST_COMPRESSION,
                };
                try {
        byte[] src = Files.readAllBytes(p);
                    if (src.length == 0) return;

                    Deflater def = new Deflater(Deflater.DEFAULT_COMPRESSION); //, true);
                    for (int level : levels) {
                        ByteArrayOutputStream baos = new ByteArrayOutputStream();
                        System.out.println(" -------------- level:" + level + " ------------------");

        def.setLevel(level);
                        def.reset();

        try (DeflaterOutputStream defos = new DeflaterOutputStream(baos, def)) {
                            defos.write(src);
                        }
                        byte[] compressed = baos.toByteArray();

        ByteArrayInputStream bais = new ByteArrayInputStream(compressed);
                        Inflater inf = new Inflater();
                        try (InflaterInputStream infis = new InflaterInputStream(bais, inf)) {
                            byte[] bytes = infis.readAllBytes();
                            if (!Arrays.equals(src, bytes)) {
                                System.out.println(" ==> NG");
                                new Exception().printStackTrace();
                            } else {
                                System.out.println(" ==> OK");
                            }
                        } catch (Exception x) {
                            System.out.println(" ==> NG");
                            x.printStackTrace();
                        }
                    }
        } catch (IOException x) {
                    x.printStackTrace();
        }
        }

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  sherman Xueming Shen
                  Reporter:
                  sherman Xueming Shen
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  9 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: