Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8238247

CTW runner should sweep nmethods more aggressively

    Details

    • Subcomponent:
    • Resolved In Build:
      b10

      Backports

        Description

        This is very visible when compiling the JAR with many files. For example, compiling this one:
        $ wget https://repo1.maven.org/maven2/org/apache/servicemix/bundles/org.apache.servicemix.bundles.quickfix/1.6.3_1/org.apache.servicemix.bundles.quickfix-1.6.3_1.jar
        $ ./ctw.sh org.apache.servicemix.bundles.quickfix-1.6.3_1.jar

        It contains about 0.5M methods, which ends up doing 2M compilations on all tiers, and CTW run on it takes multiple days -- I basically terminated the run after 40 hours, with about half done.

        Profiling with this patch: http://cr.openjdk.java.net/~shade/8238247/8238247-tracing.patch -- shows that compilation and deoptimization time grows nearly-linearly with the number of nmethods in the system, and CTW stabilizes at about 110K nmethods present. This is, alas, too much for the stable point when aggressive compilation and deoptimization is needed.

        Making the sweeper more aggressive with e.g. -XX:NmethodSweepActivity=2000 keeps nmethod population very low. The CTW now finishes in about 2 hours, which is at least 20x improvement. See the example chart from my experiments below. Default run was terminated early, because it was evident it goes nowhere.
         https://cr.openjdk.java.net/~shade/8238247/plot.png

        So, my suggestion would be to put -XX:NmethodSweepActivity in here:

        diff -r c3c9f45a18cf test/hotspot/jtreg/testlibrary/ctw/Makefile
        --- a/test/hotspot/jtreg/testlibrary/ctw/Makefile Thu Jan 30 13:31:24 2020 +0100
        +++ b/test/hotspot/jtreg/testlibrary/ctw/Makefile Thu Jan 30 20:41:36 2020 +0100
        @@ -67,11 +67,11 @@
         
         $(DST_DIR):
                @mkdir -p $@
         
         $(DST_DIR)/ctw.sh: $(DST_DIR)
        - echo '$${JAVA_HOME}/bin/java $${JAVA_OPTIONS} $(EXPORTS) -XX:-UseCounterDecay -Xbatch "-XX:CompileCommand=exclude,java/lang/invoke/MethodHandle.*" -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI -Xbootclasspath/a:wb.jar -jar ctw.jar $$@' > $@
        + echo '$${JAVA_HOME}/bin/java $${JAVA_OPTIONS} $(EXPORTS) -XX:-UseCounterDecay -Xbatch -XX:NmethodSweepActivity=2000 "-XX:CompileCommand=exclude,java/lang/invoke/MethodHandle.*" -XX:+UnlockDiagnosticVMOptions -XX:+WhiteBoxAPI -Xbootclasspath/a:wb.jar -jar ctw.jar $$@' > $@
                chmod a+x $@
         
         $(DST_DIR)/ctw.jar: filelist $(DST_DIR)/wb.jar
                @mkdir -p $(OUTPUT_DIR)
                $(JAVAC) $(EXPORTS) -sourcepath $(SRC_DIR) -d $(OUTPUT_DIR) -cp $(DST_DIR)/wb.jar @filelist

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  shade Aleksey Shipilev
                  Reporter:
                  shade Aleksey Shipilev
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  4 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: