Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8216364

C2 compilation fails with SIGSEGV

    Details

    • Subcomponent:
      gc
    • CPU:
      x86_64
    • OS:
      linux

      Description

      ADDITIONAL SYSTEM INFORMATION :
      CentOS Linux release 7.5.1804 running on AWS on m5.xlarge instance type, using Java 11.0.1+13-LTS build obtained from https://centos.pkgs.org/7/centos-updates-x86_64/java-11-openjdk-devel-11.0.1.13-3.el7_6.x86_64.rpm.html


      A DESCRIPTION OF THE PROBLEM :
      We have observed crashes when C2 tries to compile com.lmax.disruptor.MultiProducerSequencer::initialiseAvailableBuffer. Attached are excerpts from the crash logs. We have not been able to reproduce reliably, crashes happened a few times in production on an internal server application. We have enabled compiler logs (-XX:+UnlockDiagnosticVMOptions -XX:+LogCompilation), but the application has not crashed since. I will update this ticket once we have those.

      ERROR MESSAGES/STACK TRACES THAT OCCUR :
      #
      # A fatal error has been detected by the Java Runtime Environment:
      #
      # SIGSEGV (0xb) at pc=0x00007fe5cd2f63cd, pid=29619, tid=29632
      #
      # JRE version: OpenJDK Runtime Environment (11.0.1+13) (build 11.0.1+13-LTS)
      # Java VM: OpenJDK 64-Bit Server VM (11.0.1+13-LTS, mixed mode, sharing, tiered, compressed oops, shenandoah gc, linux-amd64)
      # Problematic frame:
      # V [libjvm.so+0xcac3cd]
      #
      # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
      #
      # If you would like to submit a bug report, please visit:
      # http://bugreport.java.com/bugreport/crash.jsp
      #

      --------------- S U M M A R Y ------------

      Command Line: -XX:+CrashOnOutOfMemoryError -Djava.io.tmpdir=var/data/tmp -XX:ErrorFile=var/log/hs_err_pid%p.log -Dsun.net.inetaddr.ttl=20 -XX:+UseShenandoahGC -XX:+ExplicitGCInvokesConcurrent -XX:+ClassUnloadingWithConcurrentMark -Xmx2g -Xms2g <**REDACTED**>

      Host: Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz, 4 cores, 15G, CentOS Linux release 7.5.1804 (Core)
      Time: Thu Dec 20 20:58:08 2018 UTC elapsed time: 92779 seconds (1d 1h 46m 19s)

      --------------- T H R E A D ---------------

      Current thread (0x00007fe5c4166800): JavaThread "C2 CompilerThread0" daemon [_thread_in_native, id=29632, stack(0x00007fe5a58f8000,0x00007fe5a59f9000)]


      Current CompileTask:
      C2:92779895 38520 % 4 com.lmax.disruptor.MultiProducerSequencer::initialiseAvailableBuffer @ 8 (31 bytes)

      Stack: [0x00007fe5a58f8000,0x00007fe5a59f9000], sp=0x00007fe5a59f3890, free space=1006k
      Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [libjvm.so+0xcac3cd]
      V [libjvm.so+0xdac520]
      V [libjvm.so+0xdbfc47]
      V [libjvm.so+0xaf4de0]
      V [libjvm.so+0xdae9f4]
      V [libjvm.so+0x6b6616]
      V [libjvm.so+0x6b774c]
      V [libjvm.so+0x5d6496]
      V [libjvm.so+0x6c02cb]
      V [libjvm.so+0x6c1d78]
      V [libjvm.so+0xed4472]
      V [libjvm.so+0xed47d8]
      V [libjvm.so+0xc58482]


      siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x0000000000000000

      Register to memory mapping:

      RAX=0x00007fe5a59f6dc0 is pointing into the stack for thread: 0x00007fe5c4166800
      RBX=0x0 is NULL
      RCX=0x0000000000000291 is an unknown value
      RDX=0x00007fe5a12283a8 points into unknown readable memory
      RSP=0x00007fe5a59f3890 is pointing into the stack for thread: 0x00007fe5c4166800
      RBP=0x00007fe5a59f38a0 is pointing into the stack for thread: 0x00007fe5c4166800
      RSI=0x0 is NULL
      RDI=0x0 is NULL
      R8 =0x00007fe5a1e43490 points into unknown readable memory
      R9 =0x00007fe5a1a00488 points into unknown readable memory
      R10=0x0 is NULL
      R11=0x00007fe5cddcf4a0: <offset 0x000000000018d4a0> in /lib64/libc.so.6 at 0x00007fe5cdc42000
      R12=0x00007fe5a59f55b8 is pointing into the stack for thread: 0x00007fe5c4166800
      R13=0x00007fe5a59f4c30 is pointing into the stack for thread: 0x00007fe5c4166800
      R14=0x00007fe5a1a013c8 points into unknown readable memory
      R15=0x0 is NULL


      Registers:
      RAX=0x00007fe5a59f6dc0, RBX=0x0000000000000000, RCX=0x0000000000000291, RDX=0x00007fe5a12283a8
      RSP=0x00007fe5a59f3890, RBP=0x00007fe5a59f38a0, RSI=0x0000000000000000, RDI=0x0000000000000000
      R8 =0x00007fe5a1e43490, R9 =0x00007fe5a1a00488, R10=0x0000000000000000, R11=0x00007fe5cddcf4a0
      R12=0x00007fe5a59f55b8, R13=0x00007fe5a59f4c30, R14=0x00007fe5a1a013c8, R15=0x0000000000000000
      RIP=0x00007fe5cd2f63cd, EFLAGS=0x0000000000010202, CSGSFS=0x0000000000000033, ERR=0x0000000000000004
        TRAPNO=0x000000000000000e

      Top of Stack: (sp=0x00007fe5a59f3890)
      0x00007fe5a59f3890: 0000000000000000 00007fe5a12283a8
      0x00007fe5a59f38a0: 00007fe5a59f38d0 00007fe5cd3f6520
      0x00007fe5a59f38b0: 00007fe5a59f3fe0 00007fe5a1a01610
      0x00007fe5a59f38c0: 00007fe5a1a015a8 00007fe5a1a013c8

      Instructions: (pc=0x00007fe5cd2f63cd)
      0x00007fe5cd2f63ad: c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 90 66 0f 1f
      0x00007fe5cd2f63bd: 44 00 00 55 48 89 e5 41 54 49 89 fc 48 89 f7 53
      0x00007fe5cd2f63cd: 48 8b 06 48 89 f3 ff 50 58 89 c2 31 c0 85 d2 74
      0x00007fe5cd2f63dd: 4a 41 8b 7c 24 08 4d 8b 44 24 18 8d 77 ff 21 f2

      **snip**

      Threads with active compile tasks:
      C2 CompilerThread038520 % 4 com.lmax.disruptor.MultiProducerSequencer::initialiseAvailableBuffer @ 8 (31 bytes)

      VM state:not at safepoint (normal execution)

      VM Mutex/Monitor currently owned by a thread: None

      Heap address: 0x0000000080000000, size: 2048 MB, Compressed Oops mode: 32-bit
      Narrow klass base: 0x0000000800000000, Narrow klass shift: 3
      Compressed class space size: 1073741824 Address: 0x0000000840000000

      Heap:
      Shenandoah Heap
       2097152K total, 1850368K committed, 876652K used
       2048 x 1024K regions
      Status: not cancelled
      Reserved region:
       - [0x0000000080000000, 0x0000000100000000)

      **snip a lot of details**

      ShenandoahBarrierSet
      Polling page: 0x00007fe5cea75000

      Metaspace:

      Usage:
        Non-class: 61.69 MB capacity, 60.24 MB ( 98%) used, 1.11 MB ( 2%) free+waste, 345.19 KB ( <1%) overhead.
            Class: 8.24 MB capacity, 7.25 MB ( 88%) used, 868.63 KB ( 10%) free+waste, 143.81 KB ( 2%) overhead.
             Both: 69.93 MB capacity, 67.49 MB ( 97%) used, 1.96 MB ( 3%) free+waste, 489.00 KB ( <1%) overhead.

      Virtual space:
        Non-class space: 64.00 MB reserved, 61.94 MB ( 97%) committed
            Class space: 1.00 GB reserved, 8.36 MB ( <1%) committed
                   Both: 1.06 GB reserved, 70.30 MB ( 6%) committed

      Chunk freelists:
         Non-Class: 4.25 KB
             Class: 1.50 KB
              Both: 5.75 KB

      CodeHeap 'non-profiled nmethods': size=120032Kb used=27433Kb max_used=27527Kb free=92598Kb
       bounds [0x00007fe5b4455000, 0x00007fe5b5fa5000, 0x00007fe5bb98d000]
      CodeHeap 'profiled nmethods': size=120032Kb used=14980Kb max_used=36803Kb free=105052Kb
       bounds [0x00007fe5acf1d000, 0x00007fe5af36d000, 0x00007fe5b4455000]
      CodeHeap 'non-nmethods': size=5696Kb used=1344Kb max_used=1405Kb free=4352Kb
       bounds [0x00007fe5ac98d000, 0x00007fe5acbfd000, 0x00007fe5acf1d000]
       total_blobs=65057 nmethods=12706 adapters=675
       compilation: enabled
                    stopped_count=0, restarted_count=0
       full_count=0

      Compilation events (250 events):
      **snip potentially non-relevant ones: can provide if requested**
      Event: 92779.582 Thread 0x00007fe5c4168800 nmethod 38546 0x00007fe5ada39f10 code [0x00007fe5ada3a0c0, 0x00007fe5ada3a288]
      Event: 92779.629 Thread 0x00007fe5c4168800 38547 3 java.util.concurrent.LinkedBlockingQueue::<init> (94 bytes)
      Event: 92779.631 Thread 0x00007fe5c4168800 nmethod 38547 0x00007fe5ada87590 code [0x00007fe5ada878a0, 0x00007fe5ada89148]
      Event: 92779.685 Thread 0x00007fe5c4168800 38548 3 java.util.concurrent.ScheduledThreadPoolExecutor::scheduleWithFixedDelay (90 bytes)
      Event: 92779.686 Thread 0x00007fe5c4168800 nmethod 38548 0x00007fe5ada85b90 code [0x00007fe5ada85e40, 0x00007fe5ada86fa8]
      Event: 92779.786 Thread 0x00007fe5c4168800 38549 3 sun.nio.ch.Util::free (13 bytes)
      Event: 92779.793 Thread 0x00007fe5c4168800 nmethod 38549 0x00007fe5ada79490 code [0x00007fe5ada796c0, 0x00007fe5ada79ca8]
      Event: 92779.793 Thread 0x00007fe5c4168800 38550 ! 3 jdk.internal.ref.Cleaner::clean (35 bytes)
      Event: 92779.793 Thread 0x00007fe5c4168800 nmethod 38550 0x00007fe5ada7be90 code [0x00007fe5ada7c080, 0x00007fe5ada7c478]
      Event: 92779.888 Thread 0x00007fe5c4166800 nmethod 38518 0x00007fe5b5d46e10 code [0x00007fe5b5d477a0, 0x00007fe5b5d52970]
      Event: 92779.888 Thread 0x00007fe5c4166800 38521 4 java.util.concurrent.atomic.AtomicReferenceFieldUpdater$AtomicReferenceFieldUpdaterImpl::compareAndSet (24 bytes)
      Event: 92779.892 Thread 0x00007fe5c4166800 nmethod 38521 0x00007fe5b5aa3190 code [0x00007fe5b5aa3340, 0x00007fe5b5aa36f8]
      Event: 92779.892 Thread 0x00007fe5c4166800 38520 % 4 com.lmax.disruptor.MultiProducerSequencer::initialiseAvailableBuffer @ 8 (31 bytes)

      **snip - happy to provide the rest if it's useful**





      FREQUENCY : rarely


        Attachments

          Activity

            People

            • Assignee:
              rkennke Roman Kennke
              Reporter:
              webbuggrp Webbug Group
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: