Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-6919117

20% s10u9_01 sci_32bit-lpages_Monte performance regression x86

    Details

    • Type: Bug
    • Status: Closed
    • Priority: P2
    • Resolution: Won't Fix
    • Affects Version/s: solaris_10u9
    • Fix Version/s: None
    • Component/s: performance
    • Labels:
    • Subcomponent:
    • Introduced In Version:
    • CPU:
      x86, sparc
    • OS:
      solaris_10

      Description

      20% regression in sci_32bit-lpages_Monte in build s10u9_01
      sci_32bit-lpages_Monte :

      --------------------------------------------------------------------------------
      Historical sci_32bit-lpages_Monte results on oaf403_64
      --------------------------------------------------------------------------------
      Build Result Status Description Related Bugs
      --------------------------------------------------------------------------------
      s10u9_01 57.6 RED 20% regression
      s10u8_08a 71.97 GREEN unchanged
      s10u8_08
      s10u8_07 71.98 GREEN unchanged
      s10u8_06
      s10u8_05 71.96 GREEN unchanged
      s10u8_04 71.97 GREEN unchanged
      s10u8_03 71.96 GREEN unchanged
      s10u8_02 ~ YELLOW Comparison not statistically significant6858135
      s10u8_01 ~ YELLOW Comparison not statistically significant
      s10u7_08 72 GREEN unchanged
      s10u7_07 72.02 GREEN unchanged
      s10u7_06 72 GREEN unchanged
      s10u7_05 71.99 GREEN unchanged
      s10u7_04 72.01 GREEN unchanged
      s10u7_03
      s10u7_02 71.96 GREEN unchanged
      s10u7_01
      s10u6_07b 71.97 GREEN unchanged
      s10u6_07a 71.95 GREEN unchanged
      s10u6_07
      s10u6_06
      s10u6_05 71.93 GREEN unchanged
      s10u6_04
      s10u6_03
      s10u6_02 71.94 GREEN unchanged
      s10u6_01
      s10u5_10 71.98 GREEN unchanged
      s10u5_09
      s10u5_08
      s10u5_07 71.97 GREEN unchanged
      s10u5_06
      s10u5_05 71.93 GREEN unchanged
      s10u5_05 71.93 GREEN unchanged
      s10u5_04
      s10u5_03 71.96 First Result

      Using s10u8_08a as baseline
      13 baselines. Average is 71.969 PctStDevis 0.00%
      Plot_mean: 57.60 , overall change: 19.97 %
      Iteration Result Change Colour
      1 57.591 -19.98 % RED
      2 57.579 -20.00 % RED
      3 57.604 -19.96 % RED
      4 57.604 -19.96 % RED
      5 57.604 -19.96 % RED
      6 57.604 -19.96 % RED
      7 57.604 -19.96 % RED
      Number of runs: 7
      With all numbers shown, overall change will be -19.97%
      Previous build:s10u8_08a 71.974 0.01 % YELLOW
      The found results are: 1 YELLOW, 7 RED including the old result
      Std. Deviation as % from Mean is 0.02
      TTest against baseline: 0.0000000000

      Outliers
      =========
      Can't find the machine in /etc/bootparams.
      Benchmark res_range: 10:-1 (not ship)
      Driver Version changes...
        -scsi_vhci SCSI VHCI Driver 1.57 (1.57 -> 1.60)
        -cmlb Common Labeling module 1.7 (1.7 -> 1.8)
        -consconfig console configuration 1.35 (1.35 -> 1.36)
        -consconfig_dacf Consconfig DACF 1.36 (1.36 -> 1.37)
        -mpt MPT HBA Driver v1.92 (1.92 -> 1.96)
        -sd SCSI Disk Driver 1.547 (1.547 -> 1.550)
        -st SCSI tape Driver 1.255 (1.255 -> 1.256)


      Detailed statistical comparison of results is available at:
      http://perfwww.ireland/cgi-bin/detail.cgi?rig=oaf403_64&benchmark=sci_32bit-lpages_Monte&build=s10u9_01&base=s10u8_08a&clean=FALSE

      Using s10_74L2a as baseline
      No Baselines
      Plot_mean: 57.60 , overall change: 100.00 %
      Warning: plot_per is 0.00, but overall change is 100.00. Will update later.
      Iteration Result Change Colour
      1 57.591
      2 57.579
      3 57.604
      4 57.604
      5 57.604
      6 57.604
      7 57.604
      Number of runs: 7
      With all numbers shown, overall change will be 0.00%
      Previous build:s10u8_08a 71.974
      Std. Deviation as % from Mean is 0.02
      TTest against baseline: 1.0000000000

      Outliers
      =========
      Can't find the machine in /etc/bootparams.
      Benchmark res_range: 10:-1 (not ship)

      Detailed statistical comparison of results is available at:
      http://perfwww.ireland/cgi-bin/detail.cgi?rig=oaf403_64&benchmark=sci_32bit-lpages_Monte&build=s10u9_01&base=s10_74L2a&clean=FALSE

      http://perfwww.ireland/cgi-bin/publictools/shipII.cgi?&BUILD=s10u9&BNAMES=sci_32bit-lpages_Monte&tSysteM=oaf403_64

      The machine is a :X3220 1soc 4 x 2.4GHz 4 GB ram
      The results are in for a run of s10u8_08a with the java packages from s10u9_01.
      The regression in s10u9_01 is being introduced by the upgrade of the default java version
      from 1.5.0_20 to 1.5.0_23.

      mean change% num stddev% build
      71.97 0.01% 13 s10u8_08a
      57.60 -19.97% 7 0.02 s10u9_01
      56.46 -21.55% 3 0.01 s10u8_08a-6919117-java-s10u9_01

      Baseline used: s10u8_08a


      he s10u8_08a-6919117-java-s10u9_01 run here is a fresh install of s10u8_08a with these packages
      from s10u9_01: SUNWj6rt SUNWj6rtx SUNWj6cfg SUNWj5rt SUNWj5rtx SUNWj5cfg
      So it looks like this regression is being introduced by the upgrade in the java version.
      Looking at the other scimark metrics we are seeing regressions in

      s10u8_08a -> s10u9_01
      Mean Change #-res StdDev% Metric
      57.59 -20.01% 7 0.02 sci_32bit_Monte
      57.60 -19.97% 7 0.02 sci_32bit-lpages_Monte
      47.41 -13.53% 7 0.39 sci_32bit_FFT
      47.32 -13.28% 7 1.13 sci_32bit-lpages_FFT
      535.73 -2.18% 7 2.06 sci_64bit_Sparse
      513.95 -2.12% 7 0.07 sci_32bit-lpages_Sparse
      51.21 -1.42% 8 0.71 sci_64bit-lpages_FFT
      466.42 -1.36% 7 0.07 sci_32bit-lpages_Composite
      466.11 -1.12% 7 0.07 sci_32bit_Composite
      51.42 -1.11% 7 0.28 sci_64bit_FFT
      513.91 -0.65% 7 0.09 sci_32bit_Sparse
      807.03 -0.23% 7 0.11 sci_32bit_LU
      71.75 0.01% 8 0.01 sci_64bit-lpages_Monte
      71.75 0.02% 7 0.01 sci_64bit_Monte
      808.26 0.04% 7 0.11 sci_32bit-lpages_LU
      904.99 0.04% 7 0.12 sci_32bit-lpages_SOR
      904.61 0.07% 7 0.11 sci_32bit_SOR
      901.19 0.09% 8 0.07 sci_64bit-lpages_SOR
      902.63 0.14% 7 0.09 sci_64bit_SOR
      540.03 0.21% 8 0.11 sci_64bit-lpages_Sparse
      406.69 0.47% 8 2.93 sci_64bit-lpages_Composite
      407.38 1.41% 7 3.18 sci_64bit_Composite

      s10u8_08a -> s10u8_08a with JVM 1.5.0_23
      Mean Change #-res StdDev% Metric
      56.46 -21.55% 3 0.01 sci_32bit-lpages_Monte
      57.60 -20.00% 3 0.01 sci_32bit_Monte
      46.75 -14.74% 3 0.22 sci_32bit_FFT
      46.97 -13.92% 3 0.62 sci_32bit-lpages_FFT
      426.35 -7.50% 3 0.08 sci_64bit-lpages_LU
      508.61 -3.14% 3 0.03 sci_32bit-lpages_Sparse
      50.67 -2.47% 3 0.88 sci_64bit-lpages_FFT
      464.83 -1.69% 3 0.04 sci_32bit-lpages_Composite
      398.64 -1.52% 3 0.04 sci_64bit-lpages_Composite
      51.23 -1.47% 3 0.63 sci_64bit_FFT
      539.94 -1.41% 3 0.11 sci_64bit_Sparse
      467.27 -0.87% 3 0.40 sci_32bit_Composite
      807.27 -0.08% 3 0.12 sci_32bit-lpages_LU
      808.33 -0.07% 3 0.00 sci_32bit_LU
      71.74 -0.01% 3 0.03 sci_64bit-lpages_Monte
      71.74 0.00% 3 0.02 sci_64bit_Monte
      904.83 0.02% 3 0.09 sci_32bit-lpages_SOR
      901.72 0.04% 3 0.00 sci_64bit_SOR
      904.38 0.05% 3 0.13 sci_32bit_SOR
      540.23 0.25% 3 0.03 sci_64bit-lpages_Sparse
      519.30 0.39% 3 1.88 sci_32bit_Sparse
      904.21 0.42% 3 0.06 sci_64bit-lpages_SOR


      The rig seeing this regression is:
      X3220:oaf403_64:4 x 2.4GHz:4 GB:

      The regession was in sci_32bit-lpages_Monte which can be found:
      http://perfwww.ireland/Benchmarks/Java/scimark_specific.tgz
      (using RUN_specific_jvm_tuned)

      Explorer:
      http://perfwww.ireland/cgi-bin/internaltools/machineStats.cgi?s10u9_01:oaf403_64

      Statis before regression:
      http://perfwww.ireland/Results9/s10u8_08a.oaf403_64/sci_32bit-lpages_Monte/9/statit.bin.txt
      http://perfwww.ireland/Results9/s10u8_08a.oaf403_64/sci_32bit-lpages_Monte/3/statit.bin.txt
      http://perfwww.ireland/Results9/s10u8_08a.oaf403_64/sci_32bit-lpages_Monte/4/statit.bin.txt

      Statits after regression:
      http://perfwww.ireland/Results9/s10u8_08a-6919117-java-s10u9_01.oaf403_64/sci_32bit-lpages_Monte/2/statit.bin.txt
      http://perfwww.ireland/Results9/s10u8_08a-6919117-java-s10u9_01.oaf403_64/sci_32bit-lpages_Monte/1/statit.bin.txt
      http://perfwww.ireland/Results9/s10u8_08a-6919117-java-s10u9_01.oaf403_64/sci_32bit-lpages_Monte/3/statit.bin.txt

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                coffeys Sean Coffey
                Reporter:
                gmurdoch Greg Murdoch (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  Imported:
                  Indexed: