Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8265978

make test should look for more locations when searching for exit code

    XMLWordPrintable

    Details

    • Subcomponent:
    • Resolved In Build:
      b01

      Description

      JDK-8222737 introduced tier1-like testing to 8u. It does report non-zero exit code on test failures.

      Unfortunately, that is only because the summary_tier1 target specifically aggregates the per-suite test reports. So, "make test" does not exit with non-zero exit code, when a suite is run separately. This affects (jdk|langtools|hotspot)_tier1 suites and other predefined test groups. It is easy to demonstrate by defining the jdk_adhoc with known-failing test:

      $ head jdk/test/TEST.groups
      jdk_adhoc = \
          sun/text/IntHashtable/Bug4170614Test.sh

      $ make test TEST=jdk_adhoc
      ...
      FAILED: sun/text/IntHashtable/Bug4170614Test.sh
      TEST STATS: name=jdk_adhoc run=1 pass=0 fail=1
      EXIT CODE: 2
      EXIT CODE: 2
      make[3]: *** [Makefile:310: jtreg_tests] Error 2
      make[3]: Leaving directory '/home/shade/trunks/jdk8u-dev/jdk/test'
      make[2]: *** [Makefile:261: jdk_adhoc] Error 2
      make[2]: Leaving directory '/home/shade/trunks/jdk8u-dev/jdk/test'
      make[1]: *** [Makefile:80: jdk_adhoc] Error 2
      ## Finished test (build time 00:00:05)

      $ echo $?
      0

      So, CIs and scripts that look for that non-zero exit code would happily be green even when tests are failing.

      I believe that is because the added code looks at the wrong path for exitcode.txt:
      https://hg.openjdk.java.net/jdk8u/jdk8u/rev/ef851705e3e1#l2.7

      There are actually two path patterns for exitcode.txt:

      $ find | grep exitcode
      ./build/linux-x86_64-normal-server-fastdebug/testoutput/jdk_adhoc/exitcode.txt
      ./build/linux-x86_64-normal-server-fastdebug/testoutput/tier1_exitcode.txt
      ./build/linux-x86_64-normal-server-fastdebug/testoutput/langtools_tier1/exitcode.txt
      ./build/linux-x86_64-normal-server-fastdebug/testoutput/hotspot_tier1/exitcode.txt
      ./build/linux-x86_64-normal-server-fastdebug/testoutput/jdk_tier1/exitcode.txt

      Existing pattern -- ${TEST}_exitcode.txt -- seems to look for tier1_exitcode.txt, which is fine for complete tier1 runs.
      Individual suites store exit codes in ${TEST}/exitcode.txt -- and so non-tier1 runs still omit the exit code.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              shade Aleksey Shipilev
              Reporter:
              shade Aleksey Shipilev
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: