Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8267652

c2 loop unrolling by 8 results in reading memory past array

    XMLWordPrintable

    Details

    • Subcomponent:
    • Resolved In Build:
      b28
    • CPU:
      x86
    • OS:
      generic

      Backports

        Description

        I have found a bug in c2 (x86_64/AVX2), which can be reproduced on latest ojdk17.

        running java with these options:
        -XX:UseAVX=2 -XX:LoopMaxUnroll=8
        emits following code

           vmovq 0x10(%r8,%rdi,1),%xmm0 <- read 8 bytes from byteArray1(r8)
           vpxor 0x10(%r11,%rdi,1),%xmm0,%xmm0 <- read 16 bytes from byteArray2 (r11) and xor them with xmm0
           vmovq %xmm0,0x10(%r12,%rdi,1) ;*bastore {reexecute=0 rethrow=0 return_oop=0} <- write 8 bytes to byteArray3 (r12)
                                                                    ; - repro::xor_array@18 (line 10)
           add $0x8,%ebx ;*iinc {reexecute=0 rethrow=0 return_oop=0}
                                                                    ; - repro::xor_array@19 (line 10)
           cmp %esi,%ebx

        the problem is vpxor reading 16 bytes, not 8 bytes like vmovq before it.
        it may sound like not a big deal, except one case, when there are no mapped memory after byteArray pointed by %r11, then vpxor will try to access unmapped memory and crash with seg fault.

        Attaching reproducer which generates such assemly code, making it crash is very hard, as the object has to be located at very end of the region. But I have seen such crash in the wild, a snippet from such hs_err

        siginfo: si_signo: 11 (SIGSEGV), si_code: 1 (SEGV_MAPERR), si_addr: 0x00007f5d7a000000

        RBP=0x00007f5d79ffffc8 is an oop: [B
        {0x00007f5d79ffffc8} - klass: {type array byte}
         - length: 32

        CompressedOops disabled, so header is 0x18 bytes

        0001 movslq %r11d, %r10
        0003 vmovq 0x18(%rcx, %r10, 1), %xmm0
        000a vpxor 0x18(%rbp, %r10, 1), %xmm0, %xmm0 <- reading 16 bytes result in reading past mapped memory region
        0011 vmovq %xmm0, 0x18(%r8, %r19, 1)
        0018 add $0x8, %r11d
        001c cmp $0x19, %r11d
        0020 jl 0x00

          Attachments

            Issue Links

              Activity

                People

                Assignee:
                neliasso Nils Eliasson
                Reporter:
                vkempik Vladimir Kempik
                Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved: