Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8237914

The test-make target does not fail on test failure

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P2
    • Resolution: Fixed
    • Affects Version/s: 15
    • Fix Version/s: 15
    • Component/s: infrastructure
    • Labels:
      None
    • Subcomponent:
    • Resolved In Build:
      b09

      Description

      Some while ago, the top level test-make target was refactored into the RunTest.gmk framework. I understand the rationale behind this, but it had some rather server consequences.

      1. RunTest.gmk does not return a failure if a test fails. Since test-make is considered a "build" task in our distributed testing framework, there is no test parsing happening. A build failure is expected to return non zero. This means we would not see failures in CI runs of testmake today.

      2. The "special" tests in RunTest.gmk do not have any results parsing, so it's completely hidden from the user even running locally if a make test fails unless the build log is carefully inspected.

      3. Since more makefile bootstrapping and setting up is run through before even getting to the tests, it's harder to experiment with basic make functionality and run the tests as unit tests while developing.

      I just hit all of these pretty hard while investigating JDK-8237879. I would like all 3 to be addressed:

      1. An option to fail on test failure is needed in RunTest.gmk.

      2. We should add a very rudimentary parsing of results for test-make, so that it at least shows that there is 1 failure if EXITCODE is non zero.

      3. I would like to readd a basic "test-make-raw" target that just plainly calls TestMake.gmk like before.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              erikj Erik Joelsson
              Reporter:
              erikj Erik Joelsson
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: