Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-6263371

SEGV in MarkFromRootsClosure::scanOopsInOop



    • Type: Bug
    • Status: Closed
    • Priority: P4
    • Resolution: Duplicate
    • Affects Version/s: 1.4.2_04, 1.4.2_05, 1.4.2_06, 5.0, 5.0u9, 5.0u2, 5.0u11, 5.0u7, 6
    • Fix Version/s: 7
    • Component/s: hotspot
    • Labels:
    • Subcomponent:
    • CPU:
      generic, x86, sparc
    • OS:
      generic, solaris, solaris_8, solaris_9, solaris_10



        Customer encounters crash in ConcurrentMarkSweepThread when using CMS.
        (dbx) where -l
          [1] libthread.so.1:__sigprocmask(0x3, 0xdc240444, 0x0), at 0xdfb66b94
          [2] libthread.so.1:_resetsig(0x0, 0x6, 0xdfb7a000, 0x0, 0x1, 0xdc240d00), at 0xdfb5d9d4
          [3] libthread.so.1:_sigon(0xdfb7a000, 0xdc2404a0, 0xdfb5f953, 0xdfb812f8, 0x6, 0x6), at 0xdfb5d2fd
          [4] libthread.so.1:_lmutex_unlock(0xdfb812f8), at 0xdfb5ac70
          [5] libthread.so.1:_thrp_kill(0xdc240d68, 0x6, 0x6, 0xdef9808d, 0xdefe08c4, 0xdfb35000), at 0xdfb5f953
          [6] libthread.so.1:_pthread_kill(0x6, 0x6), at 0xdfb5f843
          [7] libc.so.1:raise(0x6), at 0xdfadbd87
          [8] libc.so.1:abort(0xdefc8000, 0xdc2405c0, 0xdef39955, 0x1, 0x0, 0x0), at 0xdfacc14c
          [9] libjvm.so:os::abort(0x1), at 0xdeeec4ff
          [10] libjvm.so:VMError::report_and_die(0xdc2405d8, 0xdc2407bc, 0xdc240d68, 0xdefc8000, 0xb, 0xdef98562), at 0xdef39955
          [11] libjvm.so:JVM_handle_solaris_signal(0xb, 0xdc240a3c, 0xdc24083c, 0x1), at 0xdedd284b
          [12] libjvm.so:signalHandler(0xb, 0xdc240a3c, 0xdc24083c, 0xdc240828, 0xdfb65b55, 0xb), at 0xdedd20a0
          [13] libthread.so.1:__sighndlr(0xb, 0xdc240a3c, 0xdc24083c, 0xdedd2080), at 0xdfb57a8b
          ---- called from signal handler with signal 11 (SIGSEGV) ------
        =>[14] libjvm.so:MarkFromRootsClosure::scanOopsInOop(0xdc240b70, 0xcb6519e8), at 0xdee17915
          [15] libjvm.so:MarkFromRootsClosure::do_bit(0xdc240b70, 0x89467a), at 0xdee1770b
          [16] libjvm.so:BitMap::iterate(0x8128bd4, 0xdc240b70, 0x0, 0x3f00000), at 0xdeddbf48
          [17] libjvm.so:CMSCollector::markFromRootsWork(0x8128b60, 0x1), at 0xdee12c79
          [18] libjvm.so:CMSCollector::markFromRoots(0x8128b60, 0x1), at 0xdee12ab7
          [19] libjvm.so:CMSCollector::collect_in_background(0x8128b60, 0x0), at 0xdee10c7c
          [20] libjvm.so:ConcurrentMarkSweepThread::run(0x8139060), at 0xdee1a2de
          [21] libjvm.so:_start(0x8139060, 0xdeeec230), at 0xdeeec302

        The heap shows

         par new generation total 347328K, used 190482K [0xb4000000, 0xc9400000, 0xc9400000)
          eden space 346496K, 54% used [0xb4000000, 0xbfa04b78, 0xc9260000)
          from space 832K, 0% used [0xc9330000, 0xc9330000, 0xc9400000)
          to space 832K, 0% used [0xc9260000, 0xc9260000, 0xc9330000)
         concurrent mark-sweep generation total 176128K, used 43830K [0xc9400000, 0xd4000000, 0xd4000000)
         concurrent-mark-sweep perm gen total 81920K, used 38705K [0xd4000000, 0xd9000000, 0xd9000000)

        jvm_args: -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+CMSParallelRemarkEnabled -Xms512m -Xmx512m -verbose:gc -XX:+PrintGCTimeStamps -XX:PermSize=80m -XX:MaxPermSize=80m -XX:NewSize=340m -XX:MaxNewSize=340m -XX:SoftRefLRUPolicyMSPerMB=125 -XX:SurvivorRatio=400

        ###@###.### 2005-04-29 05:30:31 GMT
        In Nokia we have seen a similar problem using Java 5:

        Anyone any ideas why our jvm server crashed with the following stack trace ?
        This is a Weblogic process, running version 9 of WL on Solaris 9 sparc.
        The crashed happened in QA not PROD env.

        Is this a crash from the CMS collector ? If the problem reoccurs I will open a CR.
        There are no test cases which can reproduce the crash.

        Many thanks,

        # An unexpected error has been detected by HotSpot Virtual Machine:
        # SIGBUS (0xa) at pc=0xfec282d8, pid=9234, tid=6
        # Java VM: Java HotSpot(TM) Server VM (1.5.0_09-b03 mixed mode)
        # Problematic frame:
        # V [libjvm.so+0x4282d8]

        --------------- T H R E A D ---------------

        Current thread (0x00125410): ConcurrentMarkSweepThread [id=6]

        siginfo:si_signo=10, si_errno=0, si_code=1, si_addr=0x00690077

         O0=0x00000000 O1=0xc108c560 O2=0xfb17f8b4 O3=0x00000002
         O4=0x00000009 O5=0x00104f44 O6=0xfb17f850 O7=0x000038f0
         G1=0x000038f0 G2=0x61980000 G3=0x0000e3bc G4=0x000038ef
         G5=0x00000000 G6=0x00000000 G7=0xff350a00 Y=0x00000000
         PC=0xfec282d8 nPC=0xfec282dc

        Top of Stack: (sp=0xfb17f850)
        0xfb17f850: fb17f8b4 00000001 0069006f 00008000
        0xfb17f860: feed26a8 00007000 ffffffff ff0171b4
        0xfb17f870: fb17fa30 00009118 ff01407c ff014080
        0xfb17f880: ff0174fc fefce000 fb17f8e0 febe40fc
        0xfb17f890: 00000000 001258bc fb17f948 fec22f48
        0xfb17f8a0: fb17f8c8 fb17f900 00000000 00000001
        0xfb17f8b0: 00000001 ff013fb0 001b50b0 00104e50
        0xfb17f8c0: 90c00000 1a000000 00104e84 00104f44

        Instructions: (pc=0xfec282d8)
        0xfec282c8: c8 23 60 3c 87 29 20 02 d2 00 80 03 e4 02 60 04
        0xfec282d8: d6 04 a0 08 e8 02 e0 d0 9f c5 00 00 90 04 a0 08

        Stack: [0xfb100000,0xfb180000), sp=0xfb17f850, free space=510k
        Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
        V [libjvm.so+0x4282d8]
        V [libjvm.so+0x3e4104]
        V [libjvm.so+0x421864]
        V [libjvm.so+0x4215a4]
        V [libjvm.so+0x41ee58]
        V [libjvm.so+0x42bf2c]
        V [libjvm.so+0x670740]

        The thread 6 died. Its stack trace is below:
        tid 6 died. Analyzing the core dump looks like
        the CMS might be the problem here. Im not sure. It could be a
        problem between CMS and the VM.

        ----------------- t@6 -----------------
        0xff31fc54 _lwp_kill + 0x8
        0xff2b6d50 abort + 0x100
        0xfee70c1c void os::abort(int) + 0x58
        0xfeeff200 void VMError::report_and_die() + 0xc84
        0xfea73480 JVM_handle_solaris_signal + 0xaac
        0xff385bac __sighndlr + 0xc
        0xff37f804 call_user_handler + 0x234
        0xff37f9b4 sigacthandler + 0x64
        0xfec282d8 void MarkFromRootsClosure::scanOopsInOop(HeapWord*) + 0x174
        0xfebe40fc void BitMap::iterate(BitMapClosure*,unsigned,unsigned) + 0x80
        0xfec2185c int CMSCollector::markFromRootsWork(int) + 0x168
        0xfec2159c int CMSCollector::markFromRoots(int) + 0x134
        0xfec1ee50 void CMSCollector::collect_in_background(int) + 0x2d0
        0xfec2bf24 void ConcurrentMarkSweepThread::run() + 0x4fc
        0xfee70738 void*_start(void*) + 0x208
        0xff385854 _lwp_start

        Our VM options:

        Other Threads:
          0x001b5bd8 VMThread [id=7]
          0x001cad68 WatcherThread [id=16]

        VM state:not at safepoint (normal execution)

        VM Mutex/Monitor currently owned by a thread: None

         par new generation total 523840K, used 299620K [0x70c00000, 0x90c00000, 0x90c00000)
          eden space 523392K, 57% used [0x70c00000, 0x83099360, 0x90b20000)
          from space 448K, 0% used [0x90b20000, 0x90b20000, 0x90b90000)
          to space 448K, 0% used [0x90b90000, 0x90b90000, 0x90c00000)
         concurrent mark-sweep generation total 1572864K, used 1040001K [0x90c00000, 0xf0c00000, 0xf0c00000)
         concurrent-mark-sweep perm gen total 131072K, used 74795K [0xf0c00000, 0xf8c00000, 0xf8c00000)

        VM Arguments:
        jvm_args: -XX:+UseConcMarkSweepGC -Xms2048m -Xmx2048m -XX:NewSize=512m -XX:MaxNewSize=512m -XX:PermSize=128m -XX:MaxPermSize=128m -Dcom.sun.xml.namespace.QName.useCompatibleSerialVersionUID=1.0 -Dcommerce.properties=/opt/nokiacif/app/swd/current/config/swdDomain/properties/weblogiccommerce.properties -DmasterReference=/opt/nok
        iacif/app/swd/current/config/swdDomain/properties/master.properties -Djava.awt.headless=true -Dweblogic.Name=swdServer2 -Dbea.home=/opt/nokiacif/bea9 -Dweblogic.man
        agement.username=system -Dweblogic.management.password=weblogic -Dweblogic.ProductionModeEnabled=true -Dweblogic.management.discover=true -Djavax.xml.soap.MessageFa
        ctory=com.sun.xml.messaging.saaj.soap.ver1_1.SOAPMessageFactory1_1Impl -Dweblogic.management.server=wl-swd-admin:7110 -Djava.security.policy=/opt/nokiacif/bea9/webl
        java_command: weblogic.Server
        Launcher Type: SUN_STANDARD

        Environment Variables:

        Signal Handlers:
        SIGSEGV: [libjvm.so+0x6ff5b8], sa_mask[0]=0x7fbffeff, sa_flags=0x00000004
        SIGBUS: [libjvm.so+0x6ff5b8], sa_mask[0]=0x7fbffeff, sa_flags=0x00000004
        SIGFPE: [libjvm.so+0x27299c], sa_mask[0]=0x7fbffeff, sa_flags=0x0000000c
        SIGPIPE: [libjvm.so+0x27299c], sa_mask[0]=0x7fbffeff, sa_flags=0x0000000c
        SIGILL: [libjvm.so+0x27299c], sa_mask[0]=0x7fbffeff, sa_flags=0x0000000c
        SIGUSR1: [libjvm.so+0x672ca4], sa_mask[0]=0x00000000, sa_flags=0x00000008
        SIGUSR2: [libjvm.so+0x27299c], sa_mask[0]=0x7fbffeff, sa_flags=0x0000000c
        SIGHUP: [libjvm.so+0x67191c], sa_mask[0]=0x7fbffeff, sa_flags=0x00000004
        SIGINT: [libjvm.so+0x67191c], sa_mask[0]=0x7fbffeff, sa_flags=0x00000004
        SIGQUIT: [libjvm.so+0x67191c], sa_mask[0]=0x7fbffeff, sa_flags=0x00000004
        SIGTERM: [libjvm.so+0x67191c], sa_mask[0]=0x7fbffeff, sa_flags=0x00000004

        --------------- S Y S T E M ---------------

        OS: Solaris 9 9/04 s9s_u7wos_09 SPARC
                   Copyright 2004 Sun Microsystems, Inc. All Rights Reserved.
                                Use is subject to license terms.
                                     Assembled 29 June 2004

        uname:SunOS 5.9 Generic_118558-30 sun4u (T2 libthread)
        rlimit: STACK 8192k, CORE infinity, NOFILE 8192, AS infinity
        load average:0.71 0.19 0.09

        CPU:total 4 has_v8, has_v9, has_vis1, has_vis2, is_ultra3

        Memory: 8k page, physical 16777216k(10392456k free)

        vm_info: Java HotSpot(TM) Server VM (1.5.0_09-b03) for solaris-sparc, built on Oct 12 2006 01:30:08 by unknown with unknown Workshop:0x550
        Another customer (Intralinks) has run into the same issue with Java 5.
        Actually, they see it with a 5.0u11-based FVB for CR 6492085 and
        CR 6518092 provided by Java sustaining. We found no connection of
        the newly observed crash to the original fix.

        I recommended to use -XX:-CMSParallelRemarkEnabled option as a workaround
        and asked to try to reproduce the crash with fastdebug version and
        -Verify{Before,After,During}GC options specified.


            Issue Links



                ysr Y. Ramakrishna
                fchoong Fui-Shien Choong (Inactive)
                0 Vote for this issue
                12 Start watching this issue