Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8233360

Run and evaluate tests that are currently ignored

    Details

    • Type: Task
    • Status: Open
    • Priority: P3
    • Resolution: Unresolved
    • Affects Version/s: openjfx14
    • Fix Version/s: tbd
    • Component/s: javafx
    • Labels:

      Description

      We have many tests that are currently ignored, especially in the "controls", "graphics", and "systemTests" projects. The task tracks the effort of evaluating why the tests are ignored, fixing them if possible, and filing a bug to track it if not.

      We should start by doing the following for each area that we will evaluate (I will file a sub-task for each of controls, graphics, and maybe web to record the results of each area. This will include the system tests related to that area.

      The main task will be:

      1. Remove the `@Ingore` tags from all tests in an area, do a test run
      2. Record the results.
      3. Analyze the results and file bugs

      After running an analyzing the tests, please record your results in the appropriate sub-task. Then you can file new bugs relating to the test failures. We don't need a separate bug for each test method, but instead a single bug for a related group of tests (use your judgment as to the granularity of number of tests per bug). If there is already a bug filed for a test failure, then note that in your report and don't file a new bug (but do link it to the appropriate sub-task).

      The next step will be fixing the bugs. Any commit that is pushed to the repo needs to be done using a different bug ID and *not* using this Task ID or any of the Sub-task IDs. This task, and its sub-tasks, will be used to track the failures. Each new bug that is filed should be linked to the component-specific sub-task in question.

      One of the following things might be done for each of the failing tests.

      A. If a test passes on all platforms, and is stable, then the @Ignore can be removed.

      B. If the test can be made to run on all platforms by just changing the test itself, then that change can be done along with removing the @Ignore.

      C. If the test can be made to run on some platforms, then the failing platform(s) can be skipped with `assumeTrue`

      D. If the test is inherently unstable, or if it is failing as a result of a product bug, then the bug ID of the bug (test or product) that is causing the failure can be added as a string argument to the `@Ignore` (if it isn't already there).

        Attachments

          Activity

            People

            • Assignee:
              aghaisas Ajit Ghaisas
              Reporter:
              kcr Kevin Rushforth
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: