Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8219321

JDK 12 RC (build 32) crashes when G1PeriodicGCInterval is active

    Details

    • Subcomponent:
      gc
    • CPU:
      x86_64
    • OS:
      linux_ubuntu

      Description

      ADDITIONAL SYSTEM INFORMATION :
      Ubuntu 64

      A DESCRIPTION OF THE PROBLEM :
      When I set G1PeriodicGCInterval=8000 the application crashes after some time, I can't reproduce it exactly, it happens after the application is running for at least 5-10 minutes.

      These are relevant logs:

      #
      # A fatal error has been detected by the Java Runtime Environment:
      #
      # Internal Error (mutex.cpp:1073), pid=4189, tid=4279
      # guarantee(no_safepoint_check || Self->is_Java_thread()) failed: invariant
      #
      # JRE version: OpenJDK Runtime Environment (12.0+32) (build 12+32)
      # Java VM: OpenJDK 64-Bit Server VM (12+32, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)
      # Problematic frame:
      # V [libjvm.so+0xbae17c] Monitor::wait(bool, long, bool)+0x29c
      #
      # Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport %p %s %c %d %P" (or dumping to /home/travelc/core.4189)
      #
      # If you would like to submit a bug report, please visit:
      # http://bugreport.java.com/bugreport/crash.jsp
      #

      --------------- S U M M A R Y ------------

      Command Line: -Djava.util.logging.config.file=/opt/servers/tomee-S001/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms6g -Xmx25g -XX:G1HeapRegionSize=8M -XX:+UnlockExperimentalVMOptions -XX:G1NewSizePercent=15 -XX:G1PeriodicGCInterval=8000 -DLOGFOLDER=/opt/logs/S001/ -Djdk.attach.allowAttachSelf=true -Duser.timezone=UTC -Duser.language=en -Djdk.tls.client.protocols=TLSv1.1,TLSv1.2 -Xshare:on -XX:+HeapDumpOnOutOfMemoryError -XX:G1ReservePercent=15 -XX:HeapDumpPath=/opt/logs/S001/ -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=32001 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.rmi.server.hostname=PRODUCTION -XX:+ExplicitGCInvokesConcurrent -XX:+UseG1GC -XX:MaxGCPauseMillis=300 -XX:+ParallelRefProcEnabled -XX:-OmitStackTraceInFastThrow -XX:+UseStringDeduplication -XX:+AlwaysPreTouch -Xss384k -XX:ReservedCodeCacheSize=384M -XX:MetaspaceSize=1024m -XX:-ClassUnloading -XX:-ClassUnloadingWithConcurrentMark -XX:G1MixedGCCountTarget=15 -Xlog:gc*=debug,gc+stringdedup=debug:file=/opt/logs/S001/gc.log:time:filesize=50m -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -javaagent:/opt/servers/tomee-S001/lib/openejb-javaagent.jar -Dcatalina.base=/opt/servers/tomee-S001 -Dcatalina.home=/opt/servers/tomee-S001 -Djava.io.tmpdir=/opt/servers/temp/S001 org.apache.catalina.startup.Bootstrap start

      Host: Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz, 72 cores, 141G, Ubuntu 16.04.1 LTS
      Time: Fri Feb 15 11:40:13 2019 UTC elapsed time: 379 seconds (0d 0h 6m 19s)

      --------------- T H R E A D ---------------

      Current thread (0x00007f746c88a000): ConcurrentGCThread "G1 Young RemSet Sampling" [stack: 0x00007f73fc5e7000,0x00007f73fc6e7000] [id=4279]

      Stack: [0x00007f73fc5e7000,0x00007f73fc6e7000], sp=0x00007f73fc6e5c60, free space=1019k
      Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [libjvm.so+0xbae17c] Monitor::wait(bool, long, bool)+0x29c
      V [libjvm.so+0x7ad3de] GCLocker::stall_until_clear()+0xee
      V [libjvm.so+0x736d62] G1CollectedHeap::collect(GCCause::Cause)+0x192
      V [libjvm.so+0x7aa460] G1YoungRemSetSamplingThread::run_service()+0x1d0
      V [libjvm.so+0x621d82] ConcurrentGCThread::run()+0x82
      V [libjvm.so+0xdd1a1f] Thread::call_run()+0x16f
      V [libjvm.so+0xbfe18e] thread_native_entry(Thread*)+0xee

      Register to memory mapping:

      RAX=0x00007f7474081000 points into unknown readable memory: 58 00 00 00 00 00 00 00
      RBX=0x00007f746c010be0 points into unknown readable memory: 01 07 ad 6d 74 7f 00 00
      RCX=0x00007f7472cb96cf: <offset 0x0000000000f526cf> in /opt/java/jdk-12/lib/server/libjvm.so at 0x00007f7471d67000
      RDX=0x00007f7472cb9a20: <offset 0x0000000000f52a20> in /opt/java/jdk-12/lib/server/libjvm.so at 0x00007f7471d67000
      RSP=0x00007f73fc6e5c60 points into unknown readable memory: 56 00 00 00 00 00 00 00
      RBP=0x00007f73fc6e5ca0 points into unknown readable memory: 10 5d 6e fc 73 7f 00 00
      RSI=0x0000000000000431 is an unknown value
      RDI=0x00007f7472cb9908: <offset 0x0000000000f52908> in /opt/java/jdk-12/lib/server/libjvm.so at 0x00007f7471d67000
      R8 =0x0000000000000040 is an unknown value
      R9 =0x00007f73fc6e6700 points into unknown readable memory: 00 67 6e fc 73 7f 00 00
      R10=0x00007f746c88b5e0 points into unknown readable memory: 03 00 00 00 00 00 00 00
      R11=0x0 is NULL
      R12=0x0 is NULL
      R13=0x0 is NULL
      R14=0x0 is NULL
      R15=0x00007f746c88a000 points into unknown readable memory: 70 00 18 73 74 7f 00 00


      Registers:
      RAX=0x00007f7474081000, RBX=0x00007f746c010be0, RCX=0x00007f7472cb96cf, RDX=0x00007f7472cb9a20
      RSP=0x00007f73fc6e5c60, RBP=0x00007f73fc6e5ca0, RSI=0x0000000000000431, RDI=0x00007f7472cb9908
      R8 =0x0000000000000040, R9 =0x00007f73fc6e6700, R10=0x00007f746c88b5e0, R11=0x0000000000000000
      R12=0x0000000000000000, R13=0x0000000000000000, R14=0x0000000000000000, R15=0x00007f746c88a000
      RIP=0x00007f747291517c, EFLAGS=0x0000000000010246, CSGSFS=0x0000000000000033, ERR=0x0000000000000006
        TRAPNO=0x000000000000000e

      Top of Stack: (sp=0x00007f73fc6e5c60)
      0x00007f73fc6e5c60: 0000000000000056 00007f747318afc8
      0x00007f73fc6e5c70: 00007f73fc6e5d10 00007f747320b888
      0x00007f73fc6e5c80: 00007f746c010be0 00007f746c88a600
      0x00007f73fc6e5c90: 00007f746c88a5c0 00007f746c88a610

      Instructions: (pc=0x00007f747291517c)
      0x00007f747291515c: b2 87 00 48 8d 0d 69 45 3a 00 48 8d 15 b3 48 3a
      0x00007f747291516c: 00 48 8d 3d 94 47 3a 00 be 31 04 00 00 48 8b 00
      0x00007f747291517c: c6 00 58 31 c0 e8 ca f4 aa ff e8 e5 6d 04 00 e9
      0x00007f747291518c: a3 fd ff ff 49 8b 87 08 01 00 00 a8 08 0f 84 86

      Stack slot to memory mapping:
      stack at sp + 0 slots: 0x0000000000000056 is an unknown value
      stack at sp + 1 slots: 0x00007f747318afc8: <offset 0x0000000001423fc8> in /opt/java/jdk-12/lib/server/libjvm.so at 0x00007f7471d67000
      stack at sp + 2 slots: 0x00007f73fc6e5d10 points into unknown readable memory: d0 5d 6e fc 73 7f 00 00
      stack at sp + 3 slots: 0x00007f747320b888: <offset 0x00000000014a4888> in /opt/java/jdk-12/lib/server/libjvm.so at 0x00007f7471d67000
      stack at sp + 4 slots: 0x00007f746c010be0 points into unknown readable memory: 01 07 ad 6d 74 7f 00 00
      stack at sp + 5 slots: 0x00007f746c88a600 points into unknown readable memory: 00 00 00 00 00 00 00 00
      stack at sp + 6 slots: 0x00007f746c88a5c0 points into unknown readable memory: 02 00 00 00 00 00 00 00
      stack at sp + 7 slots: 0x00007f746c88a610 points into unknown readable memory: 00 00 00 00 00 00 00 00

      ....


      VM Arguments:
      jvm_args: -Djava.util.logging.config.file=/opt/servers/tomee-S001/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xms6g -Xmx25g -XX:G1HeapRegionSize=8M -XX:+UnlockExperimentalVMOptions -XX:G1NewSizePercent=15 -XX:G1PeriodicGCInterval=8000 -DLOGFOLDER=/opt/logs/S001/ -Djdk.attach.allowAttachSelf=true -Duser.timezone=UTC -Duser.language=en -Djdk.tls.client.protocols=TLSv1.1,TLSv1.2 -Xshare:on -XX:+HeapDumpOnOutOfMemoryError -XX:G1ReservePercent=15 -XX:HeapDumpPath=/opt/logs/S001/ -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=32001 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -Djava.rmi.server.hostname=PRODUCTION -XX:+ExplicitGCInvokesConcurrent -XX:+UseG1GC -XX:MaxGCPauseMillis=300 -XX:+ParallelRefProcEnabled -XX:-OmitStackTraceInFastThrow -XX:+UseStringDeduplication -XX:+AlwaysPreTouch -Xss384k -XX:ReservedCodeCacheSize=384M -XX:MetaspaceSize=1024m -XX:-ClassUnloading -XX:-ClassUnloadingWithConcurrentMark -XX:G1MixedGCCountTarget=15 -Xlog:gc*=debug,gc+stringdedup=debug:file=/opt/logs/S001/gc.log:time:filesize=50m -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -javaagent:/opt/servers/tomee-S001/lib/openejb-javaagent.jar -Dcatalina.base=/opt/servers/tomee-S001 -Dcatalina.home=/opt/servers/tomee-S001 -Djava.io.tmpdir=/opt/servers/temp/S001
      java_command: org.apache.catalina.startup.Bootstrap start
      java_class_path (initial): /opt/servers/tomee-S001/endorsed/*:/opt/servers/tomee-S001/conf/logback-config/PRODUCTION/:/opt/servers/tomee-S001/bin/bootstrap.jar:/opt/servers/tomee-S001/bin/tomcat-juli.jar
      Launcher Type: SUN_STANDARD

      [Global flags]
           bool AlwaysPreTouch = true {product} {command line}
           intx CICompilerCount = 18 {product} {ergonomic}
           bool ClassUnloading = false {product} {command line}
           bool ClassUnloadingWithConcurrentMark = false {product} {command line}
           uint ConcGCThreads = 12 {product} {ergonomic}
           bool ExplicitGCInvokesConcurrent = true {product} {command line}
           uint G1ConcRefinementThreads = 48 {product} {ergonomic}
         size_t G1HeapRegionSize = 8388608 {product} {command line}
          uintx G1MixedGCCountTarget = 15 {product} {command line}
          uintx G1NewSizePercent = 15 {experimental} {command line}
          uintx G1PeriodicGCInterval = 8000 {manageable} {command line}
          uintx G1ReservePercent = 15 {product} {command line}
          uintx GCDrainStackTargetSize = 64 {product} {ergonomic}
           bool HeapDumpOnOutOfMemoryError = true {manageable} {command line}
          ccstr HeapDumpPath = /opt/logs/S001/ {manageable} {command line}
         size_t InitialHeapSize = 6442450944 {product} {command line}
           bool ManagementServer = true {product} {command line}
         size_t MarkStackSize = 4194304 {product} {ergonomic}
          uintx MaxGCPauseMillis = 300 {product} {command line}
         size_t MaxHeapSize = 26843545600 {product} {command line}
         size_t MaxNewSize = 16106127360 {product} {ergonomic}
         size_t MetaspaceSize = 1073741824 {pd product} {command line}
         size_t MinHeapDeltaBytes = 8388608 {product} {ergonomic}
          uintx NonNMethodCodeHeapSize = 8769992 {pd product} {ergonomic}
          uintx NonProfiledCodeHeapSize = 196941596 {pd product} {ergonomic}
           bool OmitStackTraceInFastThrow = false {product} {command line}
           bool ParallelRefProcEnabled = true {product} {command line}
          uintx ProfiledCodeHeapSize = 196941596 {pd product} {ergonomic}
           bool RequireSharedSpaces = true {product} {command line}
          uintx ReservedCodeCacheSize = 402653184 {pd product} {command line}
           bool SegmentedCodeCache = true {product} {ergonomic}
           intx ThreadStackSize = 384 {pd product} {command line}
           bool UnlockExperimentalVMOptions = true {experimental} {command line}
           bool UseCompressedClassPointers = true {lp64_product} {ergonomic}
           bool UseCompressedOops = true {lp64_product} {ergonomic}
           bool UseG1GC = true {product} {command line}
           bool UseSharedSpaces = true {product} {command line}
           bool UseStringDeduplication = true {product} {command line}

      Logging:
      Log output configuration:
       #0: stdout all=warning uptime,level,tags
       #1: stderr all=off uptime,level,tags
       #2: file=/opt/logs/S001/gc.log all=off,gc*=debug time filecount=5,filesize=50M

      Environment Variables:
      PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
      SHELL=/bin/bash

      Signal Handlers:
      SIGSEGV: [libjvm.so+0xe44ea0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGBUS: [libjvm.so+0xe44ea0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGFPE: [libjvm.so+0xe44ea0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGPIPE: [libjvm.so+0xbf4390], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGXFSZ: [libjvm.so+0xbf4390], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGILL: [libjvm.so+0xe44ea0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGUSR2: [libjvm.so+0xbf4230], sa_mask[0]=00000000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
      SIGHUP: [libjvm.so+0xbf4590], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGINT: SIG_IGN, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
      SIGTERM: [libjvm.so+0xbf4590], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGQUIT: [libjvm.so+0xbf4590], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO


      --------------- S Y S T E M ---------------

      OS:DISTRIB_ID=Ubuntu
      DISTRIB_RELEASE=16.04
      DISTRIB_CODENAME=xenial
      DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
      uname:Linux 4.4.0-142-generic #168-Ubuntu SMP Wed Jan 16 21:00:45 UTC 2019 x86_64
      libc:glibc 2.23 NPTL 2.23
      rlimit: STACK 8192k, CORE 0k, NPROC 580043, NOFILE 16384, AS infinity, DATA infinity, FSIZE infinity
      load average:7.66 8.58 8.67

      /proc/meminfo:
      MemTotal: 148532924 kB
      MemFree: 870900 kB
      MemAvailable: 49312036 kB
      Buffers: 601892 kB
      Cached: 47035464 kB
      SwapCached: 786016 kB
      Active: 112132992 kB
      Inactive: 31522796 kB
      Active(anon): 88096592 kB
      Inactive(anon): 8110608 kB
      Active(file): 24036400 kB
      Inactive(file): 23412188 kB
      Unevictable: 3652 kB
      Mlocked: 3652 kB
      SwapTotal: 143359996 kB
      SwapFree: 140034508 kB
      Dirty: 3348 kB
      Writeback: 0 kB
      AnonPages: 95990492 kB
      Mapped: 193264 kB
      Shmem: 186676 kB
      Slab: 2178476 kB
      SReclaimable: 1855164 kB
      SUnreclaim: 323312 kB
      KernelStack: 76384 kB
      PageTables: 224888 kB
      NFS_Unstable: 0 kB
      Bounce: 0 kB
      WritebackTmp: 0 kB
      CommitLimit: 217626456 kB
      Committed_AS: 85895472 kB
      VmallocTotal: 34359738367 kB
      VmallocUsed: 0 kB
      VmallocChunk: 0 kB
      HardwareCorrupted: 0 kB
      AnonHugePages: 0 kB
      CmaTotal: 0 kB
      CmaFree: 0 kB
      HugePages_Total: 0
      HugePages_Free: 0
      HugePages_Rsvd: 0
      HugePages_Surp: 0
      Hugepagesize: 2048 kB
      DirectMap4k: 6318016 kB
      DirectMap2M: 141531136 kB
      DirectMap1G: 3145728 kB


      /proc/sys/kernel/threads-max (system-wide limit on the number of threads):
      1160086


      /proc/sys/vm/max_map_count (maximum number of memory map areas a process may have):
      65530


      /proc/sys/kernel/pid_max (system-wide limit on number of process identifiers):
      131072



      container (cgroup) information:
      container_type: cgroupv1
      cpu_cpuset_cpus: 0-71
      cpu_memory_nodes: 0
      active_processor_count: 72
      cpu_quota: -1
      cpu_period: 100000
      cpu_shares: -1
      memory_limit_in_bytes: -1
      memory_and_swap_limit_in_bytes: -2
      memory_soft_limit_in_bytes: -1
      memory_usage_in_bytes: 70375907328
      memory_max_usage_in_bytes: 123247005696


      CPU:total 72 (initial active 72) (32 cores per cpu, 2 threads per core) family 6 model 85 stepping 4, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, rtm, 3dnowpref, lzcnt, ht, tsc, bmi1, bmi2, adx, evex, fma
      CPU Model and flags from /proc/cpuinfo:
      model name : Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz
      flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single kaiser tpr_shadow vnmi ept vpid fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f rdseed adx smap clflushopt avx512cd xsaveopt xsavec

      Memory: 4k page, physical 148532924k(870900k free), swap 143359996k(140034508k free)

      vm_info: OpenJDK 64-Bit Server VM (12+32) for linux-amd64 JRE (12+32), built on Feb 13 2019 21:04:50 by "mach5one" with gcc 7.3.0

      END.


      REGRESSION : Last worked in version 12

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Run the web application with G1PeriodicGCInterval for some time.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      The application runs
      ACTUAL -
      The JVM crashes after some time

      CUSTOMER SUBMITTED WORKAROUND :
      set G1PeriodicGCInterval=0 to disable

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                webbuggrp Webbug Group
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: