Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8044192

JEP 195: Scalable Native Memory Tracking

    XMLWordPrintable

    Details

    • Type: JEP
    • Status: Closed
    • Priority: P2
    • Resolution: Delivered
    • Fix Version/s: 8u40
    • Component/s: hotspot
    • Labels:
    • Author:
      Zhengyu Gu
    • JEP Type:
      Feature
    • Exposure:
      Open
    • Subcomponent:
    • Scope:
      JDK
    • Discussion:
      hotspot dash dev at openjdk dot java dot net
    • Effort:
      L
    • Duration:
      L
    • Alert Status:
       Green
    • Alert Reason:
      Integrated into 8u40-b04
    • JEP Number:
      195

      Description

      Summary

      Improve the scalability of native memory tracking.

      Non-Goals

      We do not intend to add any new functionality.

      It is not a goal of this effort to reduce memory consumption or improve performance, or to make summary tracking the default.

      Success Metrics

      Native memory tracking can be run without self-shutdown on large systems without causing a significant performance impact beyond what is currently considered acceptable for small projects.

      Motivation

      The Native Memory Tracking feature is designed to help diagnose VM memory leaks as well as to clarify for users when memory leaks are not in the VM. The current implementation tends to perform poorly on large systems, when there are a large number of malloc calls in a short period. It not only consumes significant amounts of memory for tracking, but also CPU cycles to process the tracking data.

      Description

      This project is focused on solving the bottlenecks in the current native memory tracking implementation, especially the memory and CPU overheads on tracking malloc activities.

      The new approach eliminates malloc tracking records by co-locating tracking data along with malloc'd memory, which not only reduces the memory for the tracking data but also avoids the processing of raw tracking data. A lock-free hash table is used to track malloc allocation sites; this data structure further reduces the memory overhead for malloc detail tracking and query time. It also avoids blocking calls on regular JVM execution paths while tracking malloc activities, which should also reduce the impact to JVM performance.

      The new implementation co-locates the malloc tracking header with user data, it ensures that user data aligns with malloc alignment in a production build (8 bytes on 32-bit systems and 16 bytes on 64-bit systems), and 8-byte alignment on debug build.

      Testing

      Current native memory tracking tests should cover the basic/functionality tests. New scalability tests may be needed.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              ctornqvi Christian Tornqvist
              Reporter:
              zgu Zhengyu Gu
              Owner:
              Mary Lautman Mary Lautman (Inactive)
              Reviewed By:
              Karen Kinnear (Inactive), Mikael Vidstedt
              Endorsed By:
              Mikael Vidstedt
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Due:
                Created:
                Updated:
                Resolved:
                Integration Due: