Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8182741

Fatal server error on com.sun.crypto.provider.GCTR.dofinal

    Details

      Description

      FULL PRODUCT VERSION :
      java version "1.8.0_45"
      Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
      Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

      ADDITIONAL OS VERSION INFORMATION :
      Linux rpc4117 2.6.32-642.11.1.el6.x86_64 #1 SMP Wed Oct 26 10:25:23 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux


      EXTRA RELEVANT SYSTEM CONFIGURATION :
      Our web application is configured with a com.sun.identity.agents.filter.AmAgentFilter filter.

      A DESCRIPTION OF THE PROBLEM :
      Several times a month our server shuts down due to a fatal error. Our server is setup with an AMAgentFilter for authentication though the policy agent war provided by ForgeRock continues to redirect requests while access to the Spring Security web application is completely broken until the server is restarted.

      We are working on updating the versions of the JRE and OpenAM policy agent we are using in order to try to mitigate the issue.

      I believe this is related to JDK-8049855 though as JDK-8150150 states the former issue and JDK-8067755 were closed as duplicates of each other. I am unable to find a related issue that is still open or closed as fixed.


      ERROR MESSAGES/STACK TRACES THAT OCCUR :
      #
      # A fatal error has been detected by the Java Runtime Environment:
      #
      # Internal Error (sharedRuntime.cpp:834), pid=19418, tid=140192806569728
      # fatal error: exception happened outside interpreter, nmethods and vtable stubs at pc 0x00007f815905432f
      #
      # JRE version: Java(TM) SE Runtime Environment (8.0_45-b14) (build 1.8.0_45-b14)
      # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.45-b02 mixed mode linux-amd64 compressed oops)
      # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
      #
      # If you would like to submit a bug report, please visit:
      # http://bugreport.java.com/bugreport/crash.jsp
      #

      --------------- T H R E A D ---------------

      Current thread (0x00007f8168556800): JavaThread "http-nio-8250-exec-6" daemon [_thread_in_Java, id=19468, stack(0x00007f812e5f8000,0x00007f812e6f9000)]

      Stack: [0x00007f812e5f8000,0x00007f812e6f9000], sp=0x00007f812e6f6050, free space=1016k
      Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
      V [libjvm.so+0xaac99a] VMError::report_and_die()+0x2ba
      V [libjvm.so+0x4f2de9] report_fatal(char const*, int, char const*)+0x59
      V [libjvm.so+0x9ab5ba] SharedRuntime::continuation_for_implicit_exception(JavaThread*, unsigned char*, SharedRuntime::ImplicitExceptionKind)+0x33a
      V [libjvm.so+0x914f1a] JVM_handle_linux_signal+0x48a
      V [libjvm.so+0x90b493] signalHandler(int, siginfo*, void*)+0x43
      C [libpthread.so.0+0xf7e0]
      J 9464 C2 com.sun.crypto.provider.GCTR.doFinal([BII[BI)I (130 bytes) @ 0x00007f8159e0f352 [0x00007f8159e0f100+0x252]


      --------------- P R O C E S S ---------------

      Java Threads: ( => current thread )
        0x00007f8168682000 JavaThread "http-nio-8250-exec-25" daemon [_thread_blocked, id=19492, stack(0x00007f812cbe6000,0x00007f812cce7000)]
        0x00007f8168680000 JavaThread "http-nio-8250-exec-24" daemon [_thread_blocked, id=19491, stack(0x00007f812cce7000,0x00007f812cde8000)]
        0x00007f816867e800 JavaThread "http-nio-8250-exec-23" daemon [_thread_blocked, id=19490, stack(0x00007f812cde8000,0x00007f812cee9000)]
        0x00007f816867c800 JavaThread "http-nio-8250-exec-22" daemon [_thread_blocked, id=19489, stack(0x00007f812cee9000,0x00007f812cfea000)]
        0x00007f816867b000 JavaThread "http-nio-8250-exec-21" daemon [_thread_blocked, id=19488, stack(0x00007f812cfea000,0x00007f812d0eb000)]
        0x00007f8168679000 JavaThread "http-nio-8250-exec-20" daemon [_thread_blocked, id=19487, stack(0x00007f812d0eb000,0x00007f812d1ec000)]
        0x00007f8168677800 JavaThread "http-nio-8250-exec-19" daemon [_thread_blocked, id=19486, stack(0x00007f812d1ec000,0x00007f812d2ed000)]
        0x00007f8168675800 JavaThread "http-nio-8250-exec-18" daemon [_thread_blocked, id=19485, stack(0x00007f812d2ed000,0x00007f812d3ee000)]
        0x00007f8168674800 JavaThread "http-nio-8250-exec-17" daemon [_thread_blocked, id=19484, stack(0x00007f812d5ee000,0x00007f812d6ef000)]
        0x00007f8168667000 JavaThread "http-nio-8250-exec-16" daemon [_thread_blocked, id=19483, stack(0x00007f812d8ef000,0x00007f812d9f0000)]
        0x00007f8168665000 JavaThread "http-nio-8250-exec-15" daemon [_thread_blocked, id=19482, stack(0x00007f812dbf0000,0x00007f812dcf1000)]
        0x00007f8168663800 JavaThread "http-nio-8250-exec-14" daemon [_thread_blocked, id=19481, stack(0x00007f812dcf1000,0x00007f812ddf2000)]
        0x00007f8168661800 JavaThread "http-nio-8250-exec-13" daemon [_thread_blocked, id=19480, stack(0x00007f812ddf2000,0x00007f812def3000)]
        0x00007f8168660000 JavaThread "http-nio-8250-exec-12" daemon [_thread_blocked, id=19479, stack(0x00007f812def3000,0x00007f812dff4000)]
        0x00007f816865e800 JavaThread "http-nio-8250-exec-11" daemon [_thread_blocked, id=19478, stack(0x00007f812dff4000,0x00007f812e0f5000)]
        0x00007f8168672000 JavaThread "http-nio-8250-exec-10" daemon [_thread_blocked, id=19477, stack(0x00007f812e0f5000,0x00007f812e1f6000)]
        0x00007f8168671000 JavaThread "http-nio-8250-exec-9" daemon [_thread_blocked, id=19476, stack(0x00007f812e1f6000,0x00007f812e2f7000)]
        0x00007f816866a800 JavaThread "http-nio-8250-exec-8" daemon [_thread_blocked, id=19473, stack(0x00007f8145264000,0x00007f8145365000)]
        0x00007f816861d000 JavaThread "http-nio-8250-exec-7" daemon [_thread_blocked, id=19472, stack(0x00007f8145163000,0x00007f8145264000)]
      =>0x00007f8168556800 JavaThread "http-nio-8250-exec-6" daemon [_thread_in_Java, id=19468, stack(0x00007f812e5f8000,0x00007f812e6f9000)]
        0x00007f8168389000 JavaThread "http-nio-8250-exec-5" daemon [_thread_blocked, id=19467, stack(0x00007f812e6f9000,0x00007f812e7fa000)]
        0x00007f8168387800 JavaThread "http-nio-8250-exec-4" daemon [_thread_blocked, id=19466, stack(0x00007f812e7fa000,0x00007f812e8fb000)]
        0x00007f8168385800 JavaThread "http-nio-8250-exec-3" daemon [_thread_blocked, id=19465, stack(0x00007f812e8fb000,0x00007f812e9fc000)]
        0x00007f8168384000 JavaThread "http-nio-8250-exec-2" daemon [_thread_blocked, id=19464, stack(0x00007f812e9fc000,0x00007f812eafd000)]
        0x00007f816811f000 JavaThread "http-nio-8250-exec-1" daemon [_thread_blocked, id=19463, stack(0x00007f812eafd000,0x00007f812ebfe000)]
        0x00007f816811b000 JavaThread "ajp-nio-8009-Acceptor-0" daemon [_thread_in_native, id=19462, stack(0x00007f812ebfe000,0x00007f812ecff000)]
        0x00007f8168119000 JavaThread "ajp-nio-8009-ClientPoller-0" daemon [_thread_in_native, id=19461, stack(0x00007f812ecff000,0x00007f812ee00000)]
        0x00007f816856f800 JavaThread "http-nio-8250-Acceptor-0" daemon [_thread_in_native, id=19460, stack(0x00007f814405e000,0x00007f814415f000)]
        0x00007f816856d800 JavaThread "http-nio-8250-ClientPoller-0" daemon [_thread_in_native, id=19459, stack(0x00007f8144c60000,0x00007f8144d61000)]
        0x00007f816837c000 JavaThread "http-nio-8080-Acceptor-0" daemon [_thread_in_native, id=19458, stack(0x00007f8144d61000,0x00007f8144e62000)]
        0x00007f816837a800 JavaThread "http-nio-8080-ClientPoller-0" daemon [_thread_in_native, id=19457, stack(0x00007f8144e62000,0x00007f8144f63000)]
        0x00007f81682ce000 JavaThread "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon [_thread_blocked, id=19456, stack(0x00007f8158071000,0x00007f8158172000)]
        0x00007f81685e9800 JavaThread "NioBlockingSelector.BlockPoller-3" daemon [_thread_in_native, id=19446, stack(0x00007f8145365000,0x00007f8145466000)]
        0x00007f81685e8000 JavaThread "NioBlockingSelector.BlockPoller-2" daemon [_thread_in_native, id=19445, stack(0x00007f8145466000,0x00007f8145567000)]
        0x00007f81685e4800 JavaThread "NioBlockingSelector.BlockPoller-1" daemon [_thread_in_native, id=19444, stack(0x00007f8145567000,0x00007f8145668000)]
        0x00007f81685b4000 JavaThread "GC Daemon" daemon [_thread_blocked, id=19443, stack(0x00007f8145668000,0x00007f8145769000)]
        0x00007f8168577800 JavaThread "SystemTimerPool" [_thread_blocked, id=19442, stack(0x00007f8145969000,0x00007f8145a6a000)]
        0x00007f8168575800 JavaThread "SystemTimerPool" [_thread_blocked, id=19441, stack(0x00007f8145a6a000,0x00007f8145b6b000)]
        0x00007f8168574800 JavaThread "SystemTimerPool" [_thread_blocked, id=19440, stack(0x00007f8145b6b000,0x00007f8145c6c000)]
        0x00007f816837e800 JavaThread "SystemTimerPool-Scheduler" [_thread_blocked, id=19439, stack(0x00007f8145c6c000,0x00007f8145d6d000)]
        0x00007f8168533800 JavaThread "SystemTimer" [_thread_blocked, id=19432, stack(0x00007f8145d6d000,0x00007f8145e6e000)]
        0x00007f8168532800 JavaThread "SystemTimer-Scheduler" [_thread_blocked, id=19431, stack(0x00007f8145e6e000,0x00007f8145f6f000)]
        0x00007f8168100000 JavaThread "AsyncFileHandlerWriter-895328852" daemon [_thread_blocked, id=19428, stack(0x00007f816c571000,0x00007f816c672000)]
        0x00007f81680b4000 JavaThread "Service Thread" daemon [_thread_blocked, id=19426, stack(0x00007f816c773000,0x00007f816c874000)]
        0x00007f81680af000 JavaThread "C1 CompilerThread1" daemon [_thread_blocked, id=19425, stack(0x00007f816c874000,0x00007f816c975000)]
        0x00007f81680ac800 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=19424, stack(0x00007f816c975000,0x00007f816ca76000)]
        0x00007f81680ab000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=19423, stack(0x00007f816ca76000,0x00007f816cb77000)]
        0x00007f8168074800 JavaThread "Finalizer" daemon [_thread_blocked, id=19422, stack(0x00007f816cb77000,0x00007f816cc78000)]
        0x00007f8168072800 JavaThread "Reference Handler" daemon [_thread_blocked, id=19421, stack(0x00007f816cc78000,0x00007f816cd79000)]
        0x00007f8168009000 JavaThread "main" [_thread_in_native, id=19419, stack(0x00007f81711be000,0x00007f81712bf000)]

      Other Threads:
        0x00007f816806d800 VMThread [stack: 0x00007f816cd79000,0x00007f816ce7a000] [id=19420]
        0x00007f81680b7000 WatcherThread [stack: 0x00007f816c672000,0x00007f816c773000] [id=19427]

      VM state:not at safepoint (normal execution)

      VM Mutex/Monitor currently owned by a thread: None

      Heap:
       def new generation total 1258304K, used 180995K [0x00000006c0000000, 0x0000000715550000, 0x0000000715550000)
        eden space 1118528K, 13% used [0x00000006c0000000, 0x00000006c913caf0, 0x0000000704450000)
        from space 139776K, 23% used [0x0000000704450000, 0x00000007063d4378, 0x000000070ccd0000)
        to space 139776K, 0% used [0x000000070ccd0000, 0x000000070ccd0000, 0x0000000715550000)
       tenured generation total 2796224K, used 254749K [0x0000000715550000, 0x00000007c0000000, 0x00000007c0000000)
         the space 2796224K, 9% used [0x0000000715550000, 0x0000000724e17568, 0x0000000724e17600, 0x00000007c0000000)
       Metaspace used 73575K, capacity 76350K, committed 76632K, reserved 1118208K
        class space used 8063K, capacity 8503K, committed 8576K, reserved 1048576K

      Card table byte_map: [0x00007f816de6d000,0x00007f816e66e000] byte_map_base: 0x00007f816a86d000

      Polling page: 0x00007f81712ce000

      CodeCache: size=245760Kb used=51084Kb max_used=51228Kb free=194675Kb
       bounds [0x00007f8159000000, 0x00007f815c270000, 0x00007f8168000000]
       total_blobs=14562 nmethods=13926 adapters=547
       compilation: enabled

      Compilation events (10 events):
      Event: 619224.297 Thread 0x00007f81680af000 18779 ! 3 com.sun.identity.shared.encode.CookieUtils::addCookieToResponse (353 bytes)
      Event: 619224.303 Thread 0x00007f81680af000 nmethod 18779 0x00007f815b211e90 code [0x00007f815b2124c0, 0x00007f815b216320]
      Event: 619241.692 Thread 0x00007f81680ac800 18780 4 java.util.HashSet::<init> (17 bytes)
      Event: 619241.694 Thread 0x00007f81680ac800 nmethod 18780 0x00007f815c26e6d0 code [0x00007f815c26e820, 0x00007f815c26e9d8]
      Event: 619268.566 Thread 0x00007f81680af000 18781 3 java.lang.Double::compare (54 bytes)
      Event: 619268.567 Thread 0x00007f81680af000 nmethod 18781 0x00007f815a3b90d0 code [0x00007f815a3b9260, 0x00007f815a3b9770]
      Event: 620242.339 Thread 0x00007f81680af000 18782 3 java.util.concurrent.CopyOnWriteArrayList::size (6 bytes)
      Event: 620242.339 Thread 0x00007f81680af000 nmethod 18782 0x00007f815b01ca10 code [0x00007f815b01cb80, 0x00007f815b01cd70]
      Event: 620246.192 Thread 0x00007f81680af000 18783 ! 3 java.io.StringReader::read (52 bytes)
      Event: 620246.193 Thread 0x00007f81680af000 nmethod 18783 0x00007f815b211090 code [0x00007f815b211260, 0x00007f815b211990]

      GC Heap History (10 events):
      Event: 586585.315 GC heap before
      {Heap before GC invocations=134 (full 3):
       def new generation total 1258304K, used 1166251K [0x00000006c0000000, 0x0000000715550000, 0x0000000715550000)
        eden space 1118528K, 100% used [0x00000006c0000000, 0x0000000704450000, 0x0000000704450000)
        from space 139776K, 34% used [0x000000070ccd0000, 0x000000070fb6ac80, 0x0000000715550000)
        to space 139776K, 0% used [0x0000000704450000, 0x0000000704450000, 0x000000070ccd0000)
       tenured generation total 2796224K, used 254730K [0x0000000715550000, 0x00000007c0000000, 0x00000007c0000000)
         the space 2796224K, 9% used [0x0000000715550000, 0x0000000724e12948, 0x0000000724e12a00, 0x00000007c0000000)
       Metaspace used 73161K, capacity 75938K, committed 76120K, reserved 1116160K
        class space used 8057K, capacity 8499K, committed 8576K, reserved 1048576K
      Event: 586585.357 GC heap after
      Heap after GC invocations=135 (full 3):
       def new generation total 1258304K, used 52039K [0x00000006c0000000, 0x0000000715550000, 0x0000000715550000)
        eden space 1118528K, 0% used [0x00000006c0000000, 0x00000006c0000000, 0x0000000704450000)
        from space 139776K, 37% used [0x0000000704450000, 0x0000000707721d70, 0x000000070ccd0000)
        to space 139776K, 0% used [0x000000070ccd0000, 0x000000070ccd0000, 0x0000000715550000)
       tenured generation total 2796224K, used 254735K [0x0000000715550000, 0x00000007c0000000, 0x00000007c0000000)
         the space 2796224K, 9% used [0x0000000715550000, 0x0000000724e13ee8, 0x0000000724e14000, 0x00000007c0000000)
       Metaspace used 73161K, capacity 75938K, committed 76120K, reserved 1116160K
        class space used 8057K, capacity 8499K, committed 8576K, reserved 1048576K
      }
      Event: 594542.925 GC heap before
      {Heap before GC invocations=135 (full 3):
       def new generation total 1258304K, used 1170567K [0x00000006c0000000, 0x0000000715550000, 0x0000000715550000)
        eden space 1118528K, 100% used [0x00000006c0000000, 0x0000000704450000, 0x0000000704450000)
        from space 139776K, 37% used [0x0000000704450000, 0x0000000707721d70, 0x000000070ccd0000)
        to space 139776K, 0% used [0x000000070ccd0000, 0x000000070ccd0000, 0x0000000715550000)
       tenured generation total 2796224K, used 254735K [0x0000000715550000, 0x00000007c0000000, 0x00000007c0000000)
         the space 2796224K, 9% used [0x0000000715550000, 0x0000000724e13ee8, 0x0000000724e14000, 0x00000007c0000000)
       Metaspace used 73203K, capacity 75942K, committed 76120K, reserved 1116160K
        class space used 8058K, capacity 8499K, committed 8576K, reserved 1048576K
      Event: 594542.962 GC heap after
      Heap after GC invocations=136 (full 3):
       def new generation total 1258304K, used 40095K [0x00000006c0000000, 0x0000000715550000, 0x0000000715550000)
        eden space 1118528K, 0% used [0x00000006c0000000, 0x00000006c0000000, 0x0000000704450000)
        from space 139776K, 28% used [0x000000070ccd0000, 0x000000070f3f7d68, 0x0000000715550000)
        to space 139776K, 0% used [0x0000000704450000, 0x0000000704450000, 0x000000070ccd0000)
       tenured generation total 2796224K, used 254735K [0x0000000715550000, 0x00000007c0000000, 0x00000007c0000000)
         the space 2796224K, 9% used [0x0000000715550000, 0x0000000724e13ee8, 0x0000000724e14000, 0x00000007c0000000)
       Metaspace used 73203K, capacity 75942K, committed 76120K, reserved 1116160K
        class space used 8058K, capacity 8499K, committed 8576K, reserved 1048576K
      }
      Event: 602571.923 GC heap before
      {Heap before GC invocations=136 (full 3):
       def new generation total 1258304K, used 1158623K [0x00000006c0000000, 0x0000000715550000, 0x0000000715550000)
        eden space 1118528K, 100% used [0x00000006c0000000, 0x0000000704450000, 0x0000000704450000)
        from space 139776K, 28% used [0x000000070ccd0000, 0x000000070f3f7d68, 0x0000000715550000)
        to space 139776K, 0% used [0x0000000704450000, 0x0000000704450000, 0x000000070ccd0000)
       tenured generation total 2796224K, used 254735K [0x0000000715550000, 0x00000007c0000000, 0x00000007c0000000)
         the space 2796224K, 9% used [0x0000000715550000, 0x0000000724e13ee8, 0x0000000724e14000, 0x00000007c0000000)
       Metaspace used 73376K, capacity 76204K, committed 76376K, reserved 1118208K
        class space used 8059K, capacity 8500K, committed 8576K, reserved 1048576K
      Event: 602571.964 GC heap after
      Heap after GC invocations=137 (full 3):
       def new generation total 1258304K, used 44053K [0x00000006c0000000, 0x0000000715550000, 0x0000000715550000)
        eden space 1118528K, 0% used [0x00000006c0000000, 0x00000006c0000000, 0x0000000704450000)
        from space 139776K, 31% used [0x0000000704450000, 0x0000000706f55460, 0x000000070ccd0000)
        to space 139776K, 0% used [0x000000070ccd0000, 0x000000070ccd0000, 0x0000000715550000)
       tenured generation total 2796224K, used 254741K [0x0000000715550000, 0x00000007c0000000, 0x00000007c0000000)
         the space 2796224K, 9% used [0x0000000715550000, 0x0000000724e15578, 0x0000000724e15600, 0x00000007c0000000)
       Metaspace used 73376K, capacity 76204K, committed 76376K, reserved 1118208K
        class space used 8059K, capacity 8500K, committed 8576K, reserved 1048576K
      }
      Event: 613430.587 GC heap before
      {Heap before GC invocations=137 (full 3):
       def new generation total 1258304K, used 1162581K [0x00000006c0000000, 0x0000000715550000, 0x0000000715550000)
        eden space 1118528K, 100% used [0x00000006c0000000, 0x0000000704450000, 0x0000000704450000)
        from space 139776K, 31% used [0x0000000704450000, 0x0000000706f55460, 0x000000070ccd0000)
        to space 139776K, 0% used [0x000000070ccd0000, 0x000000070ccd0000, 0x0000000715550000)
       tenured generation total 2796224K, used 254741K [0x0000000715550000, 0x00000007c0000000, 0x00000007c0000000)
         the space 2796224K, 9% used [0x0000000715550000, 0x0000000724e15578, 0x0000000724e15600, 0x00000007c0000000)
       Metaspace used 73400K, capacity 76204K, committed 76376K, reserved 1118208K
        class space used 8059K, capacity 8500K, committed 8576K, reserved 1048576K
      Event: 613430.631 GC heap after
      Heap after GC invocations=138 (full 3):
       def new generation total 1258304K, used 50262K [0x00000006c0000000, 0x0000000715550000, 0x0000000715550000)
        eden space 1118528K, 0% used [0x00000006c0000000, 0x00000006c0000000, 0x0000000704450000)
        from space 139776K, 35% used [0x000000070ccd0000, 0x000000070fde5be0, 0x0000000715550000)
        to space 139776K, 0% used [0x0000000704450000, 0x0000000704450000, 0x000000070ccd0000)
       tenured generation total 2796224K, used 254741K [0x0000000715550000, 0x00000007c0000000, 0x00000007c0000000)
         the space 2796224K, 9% used [0x0000000715550000, 0x0000000724e155a8, 0x0000000724e15600, 0x00000007c0000000)
       Metaspace used 73400K, capacity 76204K, committed 76376K, reserved 1118208K
        class space used 8059K, capacity 8500K, committed 8576K, reserved 1048576K
      }
      Event: 619202.924 GC heap before
      {Heap before GC invocations=138 (full 3):
       def new generation total 1258304K, used 1168790K [0x00000006c0000000, 0x0000000715550000, 0x0000000715550000)
        eden space 1118528K, 100% used [0x00000006c0000000, 0x0000000704450000, 0x0000000704450000)
        from space 139776K, 35% used [0x000000070ccd0000, 0x000000070fde5be0, 0x0000000715550000)
        to space 139776K, 0% used [0x0000000704450000, 0x0000000704450000, 0x000000070ccd0000)
       tenured generation total 2796224K, used 254741K [0x0000000715550000, 0x00000007c0000000, 0x00000007c0000000)
         the space 2796224K, 9% used [0x0000000715550000, 0x0000000724e155a8, 0x0000000724e15600, 0x00000007c0000000)
       Metaspace used 73574K, capacity 76350K, committed 76632K, reserved 1118208K
        class space used 8063K, capacity 8503K, committed 8576K, reserved 1048576K
      Event: 619202.961 GC heap after
      Heap after GC invocations=139 (full 3):
       def new generation total 1258304K, used 32272K [0x00000006c0000000, 0x0000000715550000, 0x0000000715550000)
        eden space 1118528K, 0% used [0x00000006c0000000, 0x00000006c0000000, 0x0000000704450000)
        from space 139776K, 23% used [0x0000000704450000, 0x00000007063d4378, 0x000000070ccd0000)
        to space 139776K, 0% used [0x000000070ccd0000, 0x000000070ccd0000, 0x0000000715550000)
       tenured generation total 2796224K, used 254749K [0x0000000715550000, 0x00000007c0000000, 0x00000007c0000000)
         the space 2796224K, 9% used [0x0000000715550000, 0x0000000724e17568, 0x0000000724e17600, 0x00000007c0000000)
       Metaspace used 73574K, capacity 76350K, committed 76632K, reserved 1118208K
        class space used 8063K, capacity 8503K, committed 8576K, reserved 1048576K
      }

      Deoptimization events (10 events):
      Event: 615383.666 Thread 0x00007f8168663800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f815c096af8 method=java.text.DigitList.shouldRoundUp(IZZ)Z @ 344
      Event: 615489.226 Thread 0x00007f8168387800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00007f815b1ef67c method=org.apache.pdfbox.cos.COSString.writePDF(Ljava/io/OutputStream;)V @ 258
      Event: 615979.317 Thread 0x00007f8168665000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f815b203f98 method=java.io.BufferedOutputStream.write(I)V @ 9
      Event: 617321.974 Thread 0x00007f816867b000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x00007f815bb76c84 method=org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConverterMethodProcessor.writeWithMessageConverters(Ljava/lang/Object;Lorg/springframework/core/Meth
      Event: 617321.974 Thread 0x00007f816867b000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f815b74917c method=org.springframework.util.MimeType.checkParameters(Ljava/lang/String;Ljava/lang/String;)V @ 23
      Event: 617321.974 Thread 0x00007f816867b000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f815bcd325c method=sun.misc.FloatingDecimal.readJavaFormatString(Ljava/lang/String;)Lsun/misc/FloatingDecimal$ASCIIToBinaryConverter; @ 178
      Event: 617321.974 Thread 0x00007f816867b000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f815bee3c1c method=java.lang.Double.compare(DD)I @ 11
      Event: 617321.975 Thread 0x00007f816867b000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f815b7c0bb4 method=org.springframework.util.MimeType.isCompatibleWith(Lorg/springframework/util/MimeType;)Z @ 63
      Event: 617321.975 Thread 0x00007f816867b000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f815b3224b0 method=org.springframework.util.LinkedCaseInsensitiveMap.containsKey(Ljava/lang/Object;)Z @ 24
      Event: 620252.744 Thread 0x00007f8168556800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00007f815be5f3ec method=com.sun.crypto.provider.CipherCore.doFinal([BII[BI)I @ 405

      Internal exceptions (10 events):
      Event: 526535.156 Thread 0x00007f816866a800 Exception <a 'java/io/IOException'> (0x00000006e5744e48) thrown at [/RE-WORK/workspace/8-2-build-linux-amd64/jdk8u45/3457/hotspot/src/share/vm/prims/jni.cpp, line 709]
      Event: 568570.078 Thread 0x00007f8168680000 Exception <a 'java/io/IOException'> (0x00000006dcabd1a8) thrown at [/RE-WORK/workspace/8-2-build-linux-amd64/jdk8u45/3457/hotspot/src/share/vm/prims/jni.cpp, line 709]
      Event: 568570.078 Thread 0x00007f8168680000 Exception <a 'java/io/IOException'> (0x00000006dcabda50) thrown at [/RE-WORK/workspace/8-2-build-linux-amd64/jdk8u45/3457/hotspot/src/share/vm/prims/jni.cpp, line 709]
      Event: 571078.946 Thread 0x00007f8168661800 Exception <a 'java/io/IOException'> (0x00000006ea492c58) thrown at [/RE-WORK/workspace/8-2-build-linux-amd64/jdk8u45/3457/hotspot/src/share/vm/prims/jni.cpp, line 709]
      Event: 605319.294 Thread 0x00007f8168660000 Implicit null exception at 0x00007f815a1c02ac to 0x00007f815a1c0ca2
      Event: 605319.295 Thread 0x00007f8168660000 Exception <a 'java/lang/NullPointerException'> (0x00000006d05d9270) thrown at [/RE-WORK/workspace/8-2-build-linux-amd64/jdk8u45/3457/hotspot/src/share/vm/runtime/sharedRuntime.cpp, line 605]
      Event: 608130.974 Thread 0x00007f8168660000 Exception <a 'java/io/IOException'> (0x00000006dd949ed8) thrown at [/RE-WORK/workspace/8-2-build-linux-amd64/jdk8u45/3457/hotspot/src/share/vm/prims/jni.cpp, line 709]
      Event: 608130.976 Thread 0x00007f8168665000 Exception <a 'java/io/IOException'> (0x00000006dda62cb0) thrown at [/RE-WORK/workspace/8-2-build-linux-amd64/jdk8u45/3457/hotspot/src/share/vm/prims/jni.cpp, line 709]
      Event: 615419.913 Thread 0x00007f8168674800 Exception <a 'java/io/IOException'> (0x00000006d19760d0) thrown at [/RE-WORK/workspace/8-2-build-linux-amd64/jdk8u45/3457/hotspot/src/share/vm/prims/jni.cpp, line 709]
      Event: 617321.971 Thread 0x00007f816867b000 Exception <a 'java/lang/reflect/InvocationTargetException'> (0x00000006f11c0308) thrown at [/RE-WORK/workspace/8-2-build-linux-amd64/jdk8u45/3457/hotspot/src/share/vm/runtime/reflection.cpp, line 1080]

      Events (10 events):
      Event: 620252.525 Executing VM operation: RevokeBias done
      Event: 620252.526 Executing VM operation: RevokeBias
      Event: 620252.526 Executing VM operation: RevokeBias done
      Event: 620252.526 Executing VM operation: RevokeBias
      Event: 620252.526 Executing VM operation: RevokeBias done
      Event: 620252.530 Executing VM operation: RevokeBias
      Event: 620252.530 Executing VM operation: RevokeBias done
      Event: 620252.744 Thread 0x00007f8168556800 Uncommon trap: trap_request=0xffffff65 fr.pc=0x00007f815be5f3ec
      Event: 620252.744 Thread 0x00007f8168556800 DEOPT PACKING pc=0x00007f815be5f3ec sp=0x00007f812e6f6cf0
      Event: 620252.744 Thread 0x00007f8168556800 DEOPT UNPACKING pc=0x00007f8159005229 sp=0x00007f812e6f6c58 mode 2


      Dynamic libraries:
      00400000-00401000 r-xp 00000000 fd:00 33096 /usr/java/jdk1.8.0_45/bin/java
      00600000-00601000 rw-p 00000000 fd:00 33096 /usr/java/jdk1.8.0_45/bin/java
      00601000-00c08000 rw-p 00000000 00:00 0 [heap]
      6c0000000-7c0860000 rw-p 00000000 00:00 0
      7c0860000-800000000 ---p 00000000 00:00 0
      7f8123224000-7f81232a4000 rw-p 00000000 00:00 0
      7f81232a4000-7f8123427000 ---p 00000000 00:00 0
      7f8123427000-7f8123725000 rw-p 00000000 00:00 0
      7f8123725000-7f812372c000 r-xp 00000000 fd:00 33228 /usr/java/jdk1.8.0_45/jre/lib/amd64/libawt_headless.so
      7f812372c000-7f812392b000 ---p 00007000 fd:00 33228 /usr/java/jdk1.8.0_45/jre/lib/amd64/libawt_headless.so
      7f812392b000-7f812392c000 rw-p 00006000 fd:00 33228 /usr/java/jdk1.8.0_45/jre/lib/amd64/libawt_headless.so
      7f812392c000-7f81239ce000 r-xp 00000000 fd:00 33227 /usr/java/jdk1.8.0_45/jre/lib/amd64/libawt.so
      7f81239ce000-7f8123bce000 ---p 000a2000 fd:00 33227 /usr/java/jdk1.8.0_45/jre/lib/amd64/libawt.so
      7f8123bce000-7f8123bda000 rw-p 000a2000 fd:00 33227 /usr/java/jdk1.8.0_45/jre/lib/amd64/libawt.so
      7f8123bda000-7f8123bfe000 rw-p 00000000 00:00 0
      7f8123bfe000-7f8123c01000 ---p 00000000 00:00 0
      7f8123c01000-7f8123cff000 rw-p 00000000 00:00 0
      7f8123cff000-7f8123d02000 ---p 00000000 00:00 0
      7f8123d02000-7f8124000000 rw-p 00000000 00:00 0
      7f8124000000-7f8126959000 rw-p 00000000 00:00 0
      7f8126959000-7f8128000000 ---p 00000000 00:00 0
      7f8128000000-7f812c000000 rw-p 00000000 00:00 0
      7f812c0e3000-7f812c0e6000 ---p 00000000 00:00 0
      7f812c0e6000-7f812c3e4000 rw-p 00000000 00:00 0
      7f812c3e4000-7f812c5e4000 rw-p 00000000 00:00 0
      7f812c5e4000-7f812c5e7000 ---p 00000000 00:00 0
      7f812c5e7000-7f812c8e5000 rw-p 00000000 00:00 0
      7f812c8e5000-7f812c8e8000 ---p 00000000 00:00 0
      7f812c8e8000-7f812cbe6000 rw-p 00000000 00:00 0
      7f812cbe6000-7f812cbe9000 ---p 00000000 00:00 0
      7f812cbe9000-7f812cce7000 rw-p 00000000 00:00 0
      7f812cce7000-7f812ccea000 ---p 00000000 00:00 0
      7f812ccea000-7f812cde8000 rw-p 00000000 00:00 0
      7f812cde8000-7f812cdeb000 ---p 00000000 00:00 0
      7f812cdeb000-7f812cee9000 rw-p 00000000 00:00 0
      7f812cee9000-7f812ceec000 ---p 00000000 00:00 0
      7f812ceec000-7f812cfea000 rw-p 00000000 00:00 0
      7f812cfea000-7f812cfed000 ---p 00000000 00:00 0
      7f812cfed000-7f812d0eb000 rw-p 00000000 00:00 0
      7f812d0eb000-7f812d0ee000 ---p 00000000 00:00 0
      7f812d0ee000-7f812d1ec000 rw-p 00000000 00:00 0
      7f812d1ec000-7f812d1ef000 ---p 00000000 00:00 0
      7f812d1ef000-7f812d2ed000 rw-p 00000000 00:00 0
      7f812d2ed000-7f812d2f0000 ---p 00000000 00:00 0
      7f812d2f0000-7f812d5ee000 rw-p 00000000 00:00 0
      7f812d5ee000-7f812d5f1000 ---p 00000000 00:00 0
      7f812d5f1000-7f812d8e5000 rw-p 00000000 00:00 0
      7f812d8e5000-7f812d8ef000 ---p 00000000 00:00 0
      7f812d8ef000-7f812d8f2000 ---p 00000000 00:00 0
      7f812d8f2000-7f812dbf0000 rw-p 00000000 00:00 0
      7f812dbf0000-7f812dbf3000 ---p 00000000 00:00 0
      7f812dbf3000-7f812dcf1000 rw-p 00000000 00:00 0
      7f812dcf1000-7f812dcf4000 ---p 00000000 00:00 0
      7f812dcf4000-7f812ddf2000 rw-p 00000000 00:00 0
      7f812ddf2000-7f812ddf5000 ---p 00000000 00:00 0
      7f812ddf5000-7f812def3000 rw-p 00000000 00:00 0
      7f812def3000-7f812def6000 ---p 00000000 00:00 0
      7f812def6000-7f812dff4000 rw-p 00000000 00:00 0
      7f812dff4000-7f812dff7000 ---p 00000000 00:00 0
      7f812dff7000-7f812e0f5000 rw-p 00000000 00:00 0
      7f812e0f5000-7f812e0f8000 ---p 00000000 00:00 0
      7f812e0f8000-7f812e1f6000 rw-p 00000000 00:00 0
      7f812e1f6000-7f812e1f9000 ---p 00000000 00:00 0
      7f812e1f9000-7f812e2f7000 rw-p 00000000 00:00 0
      7f812e2f7000-7f812e2fa000 ---p 00000000 00:00 0
      7f812e2fa000-7f812e5f8000 rw-p 00000000 00:00 0
      7f812e5f8000-7f812e5fb000 ---p 00000000 00:00 0
      7f812e5fb000-7f812e6f9000 rw-p 00000000 00:00 0
      7f812e6f9000-7f812e6fc000 ---p 00000000 00:00 0
      7f812e6fc000-7f812e7fa000 rw-p 00000000 00:00 0
      7f812e7fa000-7f812e7fd000 ---p 00000000 00:00 0
      7f812e7fd000-7f812e8fb000 rw-p 00000000 00:00 0
      7f812e8fb000-7f812e8fe000 ---p 00000000 00:00 0
      7f812e8fe000-7f812e9fc000 rw-p 00000000 00:00 0
      7f812e9fc000-7f812e9ff000 ---p 00000000 00:00 0
      7f812e9ff000-7f812eafd000 rw-p 00000000 00:00 0
      7f812eafd000-7f812eb00000 ---p 00000000 00:00 0
      7f812eb00000-7f812ebfe000 rw-p 00000000 00:00 0
      7f812ebfe000-7f812ec01000 ---p 00000000 00:00 0
      7f812ec01000-7f812ecff000 rw-p 00000000 00:00 0
      7f812ecff000-7f812ed02000 ---p 00000000 00:00 0
      7f812ed02000-7f812f000000 rw-p 00000000 00:00 0
      7f812f000000-7f812f200000 rw-p 00000000 00:00 0
      7f812f200000-7f812f400000 rw-p 00000000 00:00 0
      7f812f400000-7f812f600000 rw-p 00000000 00:00 0
      7f812f600000-7f812f800000 rw-p 00000000 00:00 0
      7f812f800000-7f812fa00000 rw-p 00000000 00:00 0
      7f812fa00000-7f812fc00000 rw-p 00000000 00:00 0
      7f812fc00000-7f812fe00000 rw-p 00000000 00:00 0
      7f812fe00000-7f8130000000 rw-p 00000000 00:00 0
      7f8130000000-7f81300d2000 rw-p 00000000 00:00 0
      7f81300d2000-7f8134000000 ---p 00000000 00:00 0
      7f8134000000-7f8137ffb000 rw-p 00000000 00:00 0
      7f8137ffb000-7f8138000000 ---p 00000000 00:00 0
      7f8138000000-7f813b190000 rw-p 00000000 00:00 0
      7f813b190000-7f813c000000 ---p 00000000 00:00 0
      7f813c000000-7f813d178000 rw-p 00000000 00:00 0
      7f813d178000-7f8140000000 ---p 00000000 00:00 0
      7f8140000000-7f8140126000 rw-p 00000000 00:00 0
      7f8140126000-7f8144000000 ---p 00000000 00:00 0
      7f814405e000-7f8144061000 ---p 00000000 00:00 0
      7f8144061000-7f814435f000 rw-p 00000000 00:00 0
      7f814435f000-7f814455f000 rw-p 00000000 00:00 0
      7f814455f000-7f814475f000 rw-p 00000000 00:00 0
      7f814475f000-7f814495f000 rw-p 00000000 00:00 0
      7f81449b3000-7f81449b9000 r--s 002f7000 fd:00 33465 /usr/java/jdk1.8.0_45/jre/lib/charsets.jar
      7f81449b9000-7f81449d5000 r--s 001cb000 fd:00 33412 /usr/java/jdk1.8.0_45/jre/lib/ext/nashorn.jar
      7f81449d5000-7f8144bd5000 rw-p 00000000 00:00 0
      7f8144c60000-7f8144c63000 ---p 00000000 00:00 0
      7f8144c63000-7f8144d61000 rw-p 00000000 00:00 0
      7f8144d61000-7f8144d64000 ---p 00000000 00:00 0
      7f8144d64000-7f8144e62000 rw-p 00000000 00:00 0
      7f8144e62000-7f8144e65000 ---p 00000000 00:00 0
      7f8144e65000-7f8145163000 rw-p 00000000 00:00 0
      7f8145163000-7f8145166000 ---p 00000000 00:00 0
      7f8145166000-7f8145264000 rw-p 00000000 00:00 0
      7f8145264000-7f8145267000 ---p 00000000 00:00 0
      7f8145267000-7f8145365000 rw-p 00000000 00:00 0
      7f8145365000-7f8145368000 ---p 00000000 00:00 0
      7f8145368000-7f8145466000 rw-p 00000000 00:00 0
      7f8145466000-7f8145469000 ---p 00000000 00:00 0
      7f8145469000-7f8145567000 rw-p 00000000 00:00 0
      7f8145567000-7f814556a000 ---p 00000000 00:00 0
      7f814556a000-7f8145668000 rw-p 00000000 00:00 0
      7f8145668000-7f814566b000 ---p 00000000 00:00 0
      7f814566b000-7f8145969000 rw-p 00000000 00:00 0
      7f8145969000-7f814596c000 ---p 00000000 00:00 0
      7f814596c000-7f8145a6a000 rw-p 00000000 00:00 0
      7f8145a6a000-7f8145a6d000 ---p 00000000 00:00 0
      7f8145a6d000-7f8145b6b000 rw-p 00000000 00:00 0
      7f8145b6b000-7f8145b6e000 ---p 00000000 00:00 0
      7f8145b6e000-7f8145c6c000 rw-p 00000000 00:00 0
      7f8145c6c000-7f8145c6f000 ---p 00000000 00:00 0
      7f8145c6f000-7f8145d6d000 rw-p 00000000 00:00 0
      7f8145d6d000-7f8145d70000 ---p 00000000 00:00 0
      7f8145d70000-7f8145e6e000 rw-p 00000000 00:00 0
      7f8145e6e000-7f8145e71000 ---p 00000000 00:00 0
      7f8145e71000-7f814616f000 rw-p 00000000 00:00 0
      7f814616f000-7f814c000000 r--p 00000000 fd:00 34075 /usr/lib/locale/locale-archive
      7f814c000000-7f814d2bb000 rw-p 00000000 00:00 0
      7f814d2bb000-7f8150000000 ---p 00000000 00:00 0
      7f8150000000-7f8153ffa000 rw-p 00000000 00:00 0
      7f8153ffa000-7f8154000000 ---p 00000000 00:00 0
      7f8154000000-7f8154246000 rw-p 00000000 00:00 0
      7f8154246000-7f8158000000 ---p 00000000 00:00 0
      7f8158071000-7f8158074000 ---p 00000000 00:00 0
      7f8158074000-7f8158172000 rw-p 00000000 00:00 0
      7f8158172000-7f8158188000 r-xp 00000000 fd:00 3878 /lib64/libresolv-2.12.so
      7f8158188000-7f8158388000 ---p 00016000 fd:00 3878 /lib64/libresolv-2.12.so
      7f8158388000-7f8158389000 r--p 00016000 fd:00 3878 /lib64/libresolv-2.12.so
      7f8158389000-7f815838a000 rw-p 00017000 fd:00 3878 /lib64/libresolv-2.12.so
      7f815838a000-7f815838c000 rw-p 00000000 00:00 0
      7f815838c000-7f8158391000 r-xp 00000000 fd:00 3866 /lib64/libnss_dns-2.12.so
      7f8158391000-7f8158590000 ---p 00005000 fd:00 3866 /lib64/libnss_dns-2.12.so
      7f8158590000-7f8158591000 r--p 00004000 fd:00 3866 /lib64/libnss_dns-2.12.so
      7f8158591000-7f8158592000 rw-p 00005000 fd:00 3866 /lib64/libnss_dns-2.12.so
      7f8158592000-7f8158792000 rw-p 00000000 00:00 0
      7f8158792000-7f81587a8000 r-xp 00000000 fd:00 7387 /lib64/libgcc_s-4.4.7-20120601.so.1
      7f81587a8000-7f81589a7000 ---p 00016000 fd:00 7387 /lib64/libgcc_s-4.4.7-20120601.so.1
      7f81589a7000-7f81589a8000 rw-p 00015000 fd:00 7387 /lib64/libgcc_s-4.4.7-20120601.so.1
      7f81589a8000-7f81589da000 r-xp 00000000 fd:00 33277 /usr/java/jdk1.8.0_45/jre/lib/amd64/libsunec.so
      7f81589da000-7f8158bd9000 ---p 00032000 fd:00 33277 /usr/java/jdk1.8.0_45/jre/lib/amd64/libsunec.so
      7f8158bd9000-7f8158bdf000 rw-p 00031000 fd:00 33277 /usr/java/jdk1.8.0_45/jre/lib/amd64/libsunec.so
      7f8158bdf000-7f8158def000 rw-p 00000000 00:00 0
      7f8158def000-7f8158e00000 r-xp 00000000 fd:00 33267 /usr/java/jdk1.8.0_45/jre/lib/amd64/libnio.so
      7f8158e00000-7f8158fff000 ---p 00011000 fd:00 33267 /usr/java/jdk1.8.0_45/jre/lib/amd64/libnio.so
      7f8158fff000-7f8159000000 rw-p 00010000 fd:00 33267 /usr/java/jdk1.8.0_45/jre/lib/amd64/libnio.so
      7f8159000000-7f815c270000 rwxp 00000000 00:00 0
      7f815c270000-7f8168000000 ---p 00000000 00:00 0
      7f8168000000-7f8168c3c000 rw-p 00000000 00:00 0
      7f8168c3c000-7f816c000000 ---p 00000000 00:00 0
      7f816c050000-7f816c052000 rw-p 00000000 00:00 0
      7f816c052000-7f816c067000 r--s 000ff000 fd:00 33528 /usr/java/jdk1.8.0_45/jre/lib/resources.jar
      7f816c067000-7f816c06d000 r--s 0003e000 fd:00 33414 /usr/java/jdk1.8.0_45/jre/lib/ext/sunjce_provider.jar
      7f816c06d000-7f816c06e000 r--s 00009000 fd:00 33413 /usr/java/jdk1.8.0_45/jre/lib/ext/sunec.jar
      7f816c06e000-7f816c072000 r--s 00039000 fd:00 33415 /usr/java/jdk1.8.0_45/jre/lib/ext/sunpkcs11.jar
      7f816c072000-7f816c077000 r--s 00093000 fd:00 33529 /usr/java/jdk1.8.0_45/jre/lib/jsse.jar
      7f816c077000-7f816c079000 r--s 0001a000 fd:00 33458 /usr/java/jdk1.8.0_45/jre/lib/jce.jar
      7f816c079000-7f816c08f000 r-xp 00000000 fd:00 33266 /usr/java/jdk1.8.0_45/jre/lib/amd64/libnet.so
      7f816c08f000-7f816c28f000 ---p 00016000 fd:00 33266 /usr/java/jdk1.8.0_45/jre/lib/amd64/libnet.so
      7f816c28f000-7f816c290000 rw-p 00016000 fd:00 33266 /usr/java/jdk1.8.0_45/jre/lib/amd64/libnet.so
      7f816c290000-7f816c298000 r-xp 00000000 fd:00 33264 /usr/java/jdk1.8.0_45/jre/lib/amd64/libmanagement.so
      7f816c298000-7f816c498000 ---p 00008000 fd:00 33264 /usr/java/jdk1.8.0_45/jre/lib/amd64/libmanagement.so
      7f816c498000-7f816c499000 rw-p 00008000 fd:00 33264 /usr/java/jdk1.8.0_45/jre/lib/amd64/libmanagement.so
      7f816c499000-7f816c49b000 r--s 00011000 00:14 77528 /appl1/apache-tomcat-8.0.21/lib/catalina-storeconfig.jar
      7f816c49b000-7f816c49d000 r--s 00001000 00:14 77527 /appl1/apache-tomcat-8.0.21/lib/tomcat-api.jar
      7f816c49d000-7f816c4af000 r--s 00223000 00:14 77526 /appl1/apache-tomcat-8.0.21/lib/ecj-4.4.2.jar
      7f816c4af000-7f816c4b4000 r--s 0003a000 00:14 77524 /appl1/apache-tomcat-8.0.21/lib/catalina-tribes.jar
      7f816c4b4000-7f816c4b8000 r--s 0002d000 00:14 77523 /appl1/apache-tomcat-8.0.21/lib/tomcat-util-scan.jar
      7f816c4b8000-7f816c4cb000 r--s 00162000 00:14 77525 /appl1/apache-tomcat-8.0.21/lib/catalina.jar
      7f816c4cb000-7f816c4d3000 r--s 00087000 00:14 77520 /appl1/apache-tomcat-8.0.21/lib/jasper.jar
      7f816c4d3000-7f816c4d6000 r--s 0000e000 00:14 77519 /appl1/apache-tomcat-8.0.21/lib/tomcat-jni.jar
      7f816c4d6000-7f816c4d8000 r--s 00007000 00:14 77516 /appl1/apache-tomcat-8.0.21/lib/websocket-api.jar
      7f816c4d8000-7f816c4dd000 r--s 00036000 00:14 77512 /appl1/apache-tomcat-8.0.21/lib/tomcat-dbcp.jar
      7f816c4dd000-7f816c571000 r--s 00a0e000 00:14 149372 /appl1/apache-tomcat-8.0.21/lib/openssoclientsdk.jar
      7f816c571000-7f816c574000 ---p 00000000 00:00 0
      7f816c574000-7f816c672000 rw-p 00000000 00:00 0
      7f816c672000-7f816c673000 ---p 00000000 00:00 0
      7f816c673000-7f816c773000 rw-p 00000000 00:00 0
      7f816c773000-7f816c776000 ---p 00000000 00:00 0
      7f816c776000-7f816c874000 rw-p 00000000 00:00 0
      7f816c874000-7f816c877000 ---p 00000000 00:00 0
      7f816c877000-7f816c975000 rw-p 00000000 00:00 0
      7f816c975000-7f816c978000 ---p 00000000 00:00 0
      7f816c978000-7f816ca76000 rw-p 00000000 00:00 0
      7f816ca76000-7f816ca79000 ---p 00000000 00:00 0
      7f816ca79000-7f816cb77000 rw-p 00000000 00:00 0
      7f816cb77000-7f816cb7a000 ---p 00000000 00:00 0
      7f816cb7a000-7f816cc78000 rw-p 00000000 00:00 0
      7f816cc78000-7f816cc7b000 ---p 00000000 00:00 0
      7f816cc7b000-7f816cd79000 rw-p 00000000 00:00 0
      7f816cd79000-7f816cd7a000 ---p 00000000 00:00 0
      7f816cd7a000-7f816cec9000 rw-p 00000000 00:00 0
      7f816cec9000-7f816d0a1000 r--s 03cc3000 fd:00 33913 /usr/java/jdk1.8.0_45/jre/lib/rt.jar
      7f816d0a1000-7f816d917000 rw-p 00000000 00:00 0
      7f816d917000-7f816e66d000 rw-p 00000000 00:00 0
      7f816e66d000-7f816e738000 rw-p 00000000 00:00 0
      7f816e738000-7f816ea2e000 ---p 00000000 00:00 0
      7f816ea2e000-7f816ea48000 r-xp 00000000 fd:00 33281 /usr/java/jdk1.8.0_45/jre/lib/amd64/libzip.so
      7f816ea48000-7f816ec48000 ---p 0001a000 fd:00 33281 /usr/java/jdk1.8.0_45/jre/lib/amd64/libzip.so
      7f816ec48000-7f816ec49000 rw-p 0001a000 fd:00 33281 /usr/java/jdk1.8.0_45/jre/lib/amd64/libzip.so
      7f816ec49000-7f816ec55000 r-xp 00000000 fd:00 3868 /lib64/libnss_files-2.12.so
      7f816ec55000-7f816ee55000 ---p 0000c000 fd:00 3868 /lib64/libnss_files-2.12.so
      7f816ee55000-7f816ee56000 r--p 0000c000 fd:00 3868 /lib64/libnss_files-2.12.so
      7f816ee56000-7f816ee57000 rw-p 0000d000 fd:00 3868 /lib64/libnss_files-2.12.so
      7f816ee57000-7f816ee81000 r-xp 00000000 fd:00 33245 /usr/java/jdk1.8.0_45/jre/lib/amd64/libjava.so
      7f816ee81000-7f816f081000 ---p 0002a000 fd:00 33245 /usr/java/jdk1.8.0_45/jre/lib/amd64/libjava.so
      7f816f081000-7f816f083000 rw-p 0002a000 fd:00 33245 /usr/java/jdk1.8.0_45/jre/lib/amd64/libjava.so
      7f816f083000-7f816f090000 r-xp 00000000 fd:00 33280 /usr/java/jdk1.8.0_45/jre/lib/amd64/libverify.so
      7f816f090000-7f816f290000 ---p 0000d000 fd:00 33280 /usr/java/jdk1.8.0_45/jre/lib/amd64/libverify.so
      7f816f290000-7f816f292000 rw-p 0000d000 fd:00 33280 /usr/java/jdk1.8.0_45/jre/lib/amd64/libverify.so
      7f816f292000-7f816f299000 r-xp 00000000 fd:00 3880 /lib64/librt-2.12.so
      7f816f299000-7f816f498000 ---p 00007000 fd:00 3880 /lib64/librt-2.12.so
      7f816f498000-7f816f499000 r--p 00006000 fd:00 3880 /lib64/librt-2.12.so
      7f816f499000-7f816f49a000 rw-p 00007000 fd:00 3880 /lib64/librt-2.12.so
      7f816f49a000-7f816f51d000 r-xp 00000000 fd:00 3856 /lib64/libm-2.12.so
      7f816f51d000-7f816f71c000 ---p 00083000 fd:00 3856 /lib64/libm-2.12.so
      7f816f71c000-7f816f71d000 r--p 00082000 fd:00 3856 /lib64/libm-2.12.so
      7f816f71d000-7f816f71e000 rw-p 00083000 fd:00 3856 /lib64/libm-2.12.so
      7f816f71e000-7f81703cc000 r-xp 00000000 fd:00 33285 /usr/java/jdk1.8.0_45/jre/lib/amd64/server/libjvm.so
      7f81703cc000-7f81705cb000 ---p 00cae000 fd:00 33285 /usr/java/jdk1.8.0_45/jre/lib/amd64/server/libjvm.so
      7f81705cb000-7f81706a2000 rw-p 00cad000 fd:00 33285 /usr/java/jdk1.8.0_45/jre/lib/amd64/server/libjvm.so
      7f81706a2000-7f81706e6000 rw-p 00000000 00:00 0
      7f81706e6000-7f8170870000 r-xp 00000000 fd:00 3847 /lib64/libc-2.12.so
      7f8170870000-7f8170a70000 ---p 0018a000 fd:00 3847 /lib64/libc-2.12.so
      7f8170a70000-7f8170a74000 r--p 0018a000 fd:00 3847 /lib64/libc-2.12.so
      7f8170a74000-7f8170a75000 rw-p 0018e000 fd:00 3847 /lib64/libc-2.12.so
      7f8170a75000-7f8170a7a000 rw-p 00000000 00:00 0
      7f8170a7a000-7f8170a7c000 r-xp 00000000 fd:00 3854 /lib64/libdl-2.12.so
      7f8170a7c000-7f8170c7c000 ---p 00002000 fd:00 3854 /lib64/libdl-2.12.so
      7f8170c7c000-7f8170c7d000 r--p 00002000 fd:00 3854 /lib64/libdl-2.12.so
      7f8170c7d000-7f8170c7e000 rw-p 00003000 fd:00 3854 /lib64/libdl-2.12.so
      7f8170c7e000-7f8170c93000 r-xp 00000000 fd:00 33549 /usr/java/jdk1.8.0_45/lib/amd64/jli/libjli.so
      7f8170c93000-7f8170e93000 ---p 00015000 fd:00 33549 /usr/java/jdk1.8.0_45/lib/amd64/jli/libjli.so
      7f8170e93000-7f8170e94000 rw-p 00015000 fd:00 33549 /usr/java/jdk1.8.0_45/lib/amd64/jli/libjli.so
      7f8170e94000-7f8170eab000 r-xp 00000000 fd:00 3876 /lib64/libpthread-2.12.so
      7f8170eab000-7f81710ab000 ---p 00017000 fd:00 3876 /lib64/libpthread-2.12.so
      7f81710ab000-7f81710ac000 r--p 00017000 fd:00 3876 /lib64/libpthread-2.12.so
      7f81710ac000-7f81710ad000 rw-p 00018000 fd:00 3876 /lib64/libpthread-2.12.so
      7f81710ad000-7f81710b1000 rw-p 00000000 00:00 0
      7f81710b1000-7f81710d1000 r-xp 00000000 fd:00 28 /lib64/ld-2.12.so
      7f81710d2000-7f81710d4000 r--s 0001f000 00:14 77515 /appl1/apache-tomcat-8.0.21/lib/tomcat-jdbc.jar
      7f81710d4000-7f81710d7000 r--s 00017000 00:14 77511 /appl1/apache-tomcat-8.0.21/lib/tomcat-util.jar
      7f81710d7000-7f81710db000 r--s 00038000 00:14 77510 /appl1/apache-tomcat-8.0.21/lib/servlet-api.jar
      7f81710db000-7f81710dd000 r--s 0000b000 00:14 77504 /appl1/apache-tomcat-8.0.21/lib/catalina-ant.jar
      7f81710dd000-7f81710df000 r--s 00009000 00:14 77503 /appl1/apache-tomcat-8.0.21/lib/tomcat-i18n-ja.jar
      7f81710df000-7f81710e1000 r--s 00012000 00:14 77502 /appl1/apache-tomcat-8.0.21/lib/el-api.jar
      7f81710e1000-7f81710e5000 r--s 00024000 00:14 77501 /appl1/apache-tomcat-8.0.21/lib/jasper-el.jar
      7f81710e5000-7f81710e7000 r--s 0000c000 00:14 77500 /appl1/apache-tomcat-8.0.21/lib/tomcat-spdy.jar
      7f81710e7000-7f81710f1000 r--s 00076000 00:14 149371 /appl1/apache-tomcat-8.0.21/lib/agent.jar
      7f81710f1000-7f81710f4000 r--s 0000d000 00:14 77498 /appl1/apache-tomcat-8.0.21/lib/jsp-api.jar
      7f81710f4000-7f81710ff000 r--s 000a2000 00:14 77497 /appl1/apache-tomcat-8.0.21/lib/tomcat-coyote.jar
      7f81710ff000-7f8171101000 r--s 00003000 00:14 77496 /appl1/apache-tomcat-8.0.21/lib/annotations-api.jar
      7f8171101000-7f8171104000 r--s 0000e000 00:14 77495 /appl1/apache-tomcat-8.0.21/lib/tomcat-i18n-es.jar
      7f8171104000-7f8171106000 r--s 0001b000 00:14 77491 /appl1/apache-tomcat-8.0.21/lib/catalina-ha.jar
      7f8171106000-7f817110a000 r--s 00030000 00:14 77490 /appl1/apache-tomcat-8.0.21/lib/tomcat-websocket.jar
      7f817110a000-7f817110d000 r--s 00008000 00:14 77489 /appl1/apache-tomcat-8.0.21/lib/tomcat-i18n-fr.jar
      7f817110d000-7f817112f000 r--s 0038e000 fd:00 33407 /usr/java/jdk1.8.0_45/jre/lib/ext/cldrdata.jar
      7f817112f000-7f8171139000 r--s 0021b000 fd:00 33736 /usr/java/jdk1.8.0_45/jre/lib/ext/localedata.jar
      7f8171139000-7f817113b000 r--s 00009000 00:14 78501 /appl1/apache-tomcat-8.0.21/bin/tomcat-juli.jar
      7f817113b000-7f81711be000 rw-p 00000000 00:00 0
      7f81711be000-7f81711c1000 ---p 00000000 00:00 0
      7f81711c1000-7f81712c3000 rw-p 00000000 00:00 0
      7f81712c3000-7f81712c4000 r--s 00005000 00:14 78505 /appl1/apache-tomcat-8.0.21/bin/commons-daemon.jar
      7f81712c4000-7f81712c5000 r--s 00006000 00:14 78517 /appl1/apache-tomcat-8.0.21/bin/bootstrap.jar
      7f81712c5000-7f81712cd000 rw-s 00000000 fd:04 262149 /tmp/hsperfdata_arcs/19418
      7f81712cd000-7f81712ce000 rw-p 00000000 00:00 0
      7f81712ce000-7f81712cf000 r--p 00000000 00:00 0
      7f81712cf000-7f81712d0000 rw-p 00000000 00:00 0
      7f81712d0000-7f81712d1000 r--p 0001f000 fd:00 28 /lib64/ld-2.12.so
      7f81712d1000-7f81712d2000 rw-p 00020000 fd:00 28 /lib64/ld-2.12.so
      7f81712d2000-7f81712d3000 rw-p 00000000 00:00 0
      7ffe4688f000-7ffe468a4000 rw-p 00000000 00:00 0 [stack]
      7ffe468ab000-7ffe468ac000 r-xp 00000000 00:00 0 [vdso]
      ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]

      VM Arguments:
      jvm_args: -Djava.util.logging.config.file=/appl1/apache-tomcat-8.0.21/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dopenam.agents.bootstrap.dir=/appl1/td_sso_policy_agent/tomcat_v6_agent/Agent_001/config -Xms4096m -Xmx4096m -Djava.endorsed.dirs=/appl1/apache-tomcat-8.0.21/endorsed -Dcatalina.base=/appl1/apache-tomcat-8.0.21 -Dcatalina.home=/appl1/apache-tomcat-8.0.21 -Djava.io.tmpdir=/appl1/apache-tomcat-8.0.21/temp
      java_command: org.apache.catalina.startup.Bootstrap start
      java_class_path (initial): /appl1/apache-tomcat-8.0.21/bin/bootstrap.jar:/appl1/apache-tomcat-8.0.21/bin/tomcat-juli.jar
      Launcher Type: SUN_STANDARD

      Environment Variables:
      JAVA_HOME=/usr/java/jdk1.8.0_45
      PATH=.:/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/usr/lib:/usr/java/jdk1.8.0_45/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/java/jdk1.8.0_45/bin:/appl1/apache-tomcat-8.0.21/common/lib
      SHELL=/bin/bash

      Signal Handlers:
      SIGSEGV: [libjvm.so+0xaad1e0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGBUS: [libjvm.so+0xaad1e0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGFPE: [libjvm.so+0x90b450], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGPIPE: [libjvm.so+0x90b450], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGXFSZ: [libjvm.so+0x90b450], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGILL: [libjvm.so+0x90b450], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGUSR1: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
      SIGUSR2: [libjvm.so+0x90ca90], sa_mask[0]=00000000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
      SIGHUP: SIG_IGN, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
      SIGINT: SIG_IGN, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
      SIGTERM: [libjvm.so+0x90dde0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
      SIGQUIT: [libjvm.so+0x90dde0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO


      --------------- S Y S T E M ---------------

      OS:Red Hat Enterprise Linux Server release 6.7 (Santiago)

      uname:Linux 2.6.32-642.11.1.el6.x86_64 #1 SMP Wed Oct 26 10:25:23 EDT 2016 x86_64
      libc:glibc 2.12 NPTL 2.12
      rlimit: STACK 10240k, CORE 0k, NPROC 1024, NOFILE 4096, AS infinity
      load average:0.00 0.00 0.00

      /proc/meminfo:
      MemTotal: 8061512 kB
      MemFree: 252420 kB
      Buffers: 172004 kB
      Cached: 5047288 kB
      SwapCached: 64 kB
      Active: 2178732 kB
      Inactive: 5369308 kB
      Active(anon): 1808316 kB
      Inactive(anon): 520924 kB
      Active(file): 370416 kB
      Inactive(file): 4848384 kB
      Unevictable: 4 kB
      Mlocked: 4 kB
      SwapTotal: 6291452 kB
      SwapFree: 6285100 kB
      Dirty: 68 kB
      Writeback: 0 kB
      AnonPages: 2328724 kB
      Mapped: 60812 kB
      Shmem: 464 kB
      Slab: 161788 kB
      SReclaimable: 131168 kB
      SUnreclaim: 30620 kB
      KernelStack: 4048 kB
      PageTables: 14468 kB
      NFS_Unstable: 0 kB
      Bounce: 0 kB
      WritebackTmp: 0 kB
      CommitLimit: 10322208 kB
      Committed_AS: 4867456 kB
      VmallocTotal: 34359738367 kB
      VmallocUsed: 172172 kB
      VmallocChunk: 34359552828 kB
      HardwareCorrupted: 0 kB
      AnonHugePages: 2193408 kB
      HugePages_Total: 0
      HugePages_Free: 0
      HugePages_Rsvd: 0
      HugePages_Surp: 0
      Hugepagesize: 2048 kB
      DirectMap4k: 10240 kB
      DirectMap2M: 8378368 kB


      CPU:total 1 (1 cores per cpu, 1 threads per core) family 6 model 45 stepping 7, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, aes, clmul, tsc, tscinvbit, tscinv

      /proc/cpuinfo:
      processor : 0
      vendor_id : GenuineIntel
      cpu family : 6
      model : 45
      model name : Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz
      stepping : 7
      microcode : 1805
      cpu MHz : 2200.000
      cache size : 20480 KB
      physical id : 0
      siblings : 1
      core id : 0
      cpu cores : 1
      apicid : 0
      initial apicid : 0
      fpu : yes
      fpu_exception : yes
      cpuid level : 13
      wp : yes
      flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc up arch_perfmon pebs bts xtopology tsc_reliable nonstop_tsc aperfmperf unfair_spinlock pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx hypervisor lahf_lm ida arat pln pts dtherm
      bogomips : 4400.00
      clflush size : 64
      cache_alignment : 64
      address sizes : 40 bits physical, 48 bits virtual
      power management:



      Memory: 4k page, physical 8061512k(252420k free), swap 6291452k(6285100k free)

      vm_info: Java HotSpot(TM) 64-Bit Server VM (25.45-b02) for linux-amd64 JRE (1.8.0_45-b14), built on Apr 10 2015 10:07:45 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)

      time: Tue Jun 20 17:42:39 2017
      elapsed time: 620252 seconds (7d 4h 17m 32s)



      REPRODUCIBILITY :
      This bug can be reproduced occasionally.

        Attachments

          Activity

            People

            • Assignee:
              psonal Pallavi Sonal (Inactive)
              Reporter:
              webbuggrp Webbug Group
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: