Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8189416

Image scaling is bad with D3D pipeline

    Details

    • Subcomponent:
      2d
    • Introduced In Version:
      9
    • CPU:
      x86
    • OS:
      other

      Description

      FULL PRODUCT VERSION :
      java version "9"
      Java(TM) SE Runtime Environment (build 9+181)
      Java HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)

      ADDITIONAL OS VERSION INFORMATION :
      Windows 10

      EXTRA RELEVANT SYSTEM CONFIGURATION :
      4K monitor

      A DESCRIPTION OF THE PROBLEM :
      Java 9 distorts BufferedImages when it automatically resizes screens. I'm suspecting it has something to do with this ==> http://openjdk.java.net/jeps/263 being delivered in Java 9.

      Related question asked on Stack Overflow with example output==>https://stackoverflow.com/questions/46736757/disable-automatic-rescaling-of-bufferedimage-in-jdk9

      REGRESSION. Last worked in version 8u144

      ADDITIONAL REGRESSION INFORMATION:
      java version "1.8.0_144"
      Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
      Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Execute attached source code on Windows 10 using Java 9.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      Lines in buffered image should alternate between back and white (using attached source code)
      ACTUAL -
      Distortion of lines occur in a triangular pattern

      REPRODUCIBILITY :
      This bug can be reproduced always.

      ---------- BEGIN SOURCE ----------
      public static void main(String[] args){
              JFrame frame=new JFrame("Java version 9");
              frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
              BufferedImage image =new BufferedImage(800,600,BufferedImage.TYPE_INT_RGB);
              Graphics2D g=image.createGraphics();
              g.setColor(Color.WHITE);
              for(int y=0;y<image.getHeight();y+=2){
                  g.drawLine(0,y,image.getWidth(),y);
              }
              g.dispose();
              ImageIcon imageIcon=new ImageIcon(image);
              JLabel label=new JLabel(imageIcon);
              frame.add(label);
              frame.pack();
              frame.setVisible(true);
          }
      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      Don't upgrade to Java 9

      1. JavaHiDPI.java
        1.0 kB
        Pardeep Sharma
      1. 9vs8u144.png
        35 kB

        Activity

        webbuggrp Webbug Group created issue -
        pardesha Pardeep Sharma made changes -
        Field Original Value New Value
        Assignee Pardeep Sharma [ pardesha ]
        pardesha Pardeep Sharma made changes -
        Status New [ 10000 ] Open [ 1 ]
        Hide
        pardesha Pardeep Sharma added a comment -
        JDK 9
        Windows 10
        4k Monitor

        This need to be checked for 4K monitor. When checked on display with DPI set to 175% with Resolution 1920 * 1080, couldn't reproduce the issue as reported.
        The buffered images looks fine when checked with JDK 9. See attached screenshots depicting result with JDK 9 and JDK 8u144 with above mentioned resolution.
        However, the results may vary with 4K monitor as reported by the submitter. Checking for required infra, meanwhile keeping this open for dev. to assess the effect.
        Reported at stackoverflow:
        https://stackoverflow.com/questions/46736757/disable-automatic-rescaling-of-bufferedimage-in-jdk9
        Show
        pardesha Pardeep Sharma added a comment - JDK 9 Windows 10 4k Monitor This need to be checked for 4K monitor. When checked on display with DPI set to 175% with Resolution 1920 * 1080, couldn't reproduce the issue as reported. The buffered images looks fine when checked with JDK 9. See attached screenshots depicting result with JDK 9 and JDK 8u144 with above mentioned resolution. However, the results may vary with 4K monitor as reported by the submitter. Checking for required infra, meanwhile keeping this open for dev. to assess the effect. Reported at stackoverflow: https://stackoverflow.com/questions/46736757/disable-automatic-rescaling-of-bufferedimage-in-jdk9
        pardesha Pardeep Sharma made changes -
        Labels regression webbug dcspks regression reproducer-yes webbug
        pardesha Pardeep Sharma made changes -
        Attachment JavaHiDPI.java [ 73055 ]
        Attachment 9vs8u144.png [ 73056 ]
        pardesha Pardeep Sharma made changes -
        Labels dcspks regression reproducer-yes webbug dcspks hidpi regression reproducer-yes webbug
        pardesha Pardeep Sharma made changes -
        Project Java Incidents [ 10301 ] JDK [ 10100 ]
        Key JI-9051185 JDK-8189416
        Workflow JBS Incident Workflow [ 4961038 ] JBS Workflow [ 4961038 ]
        Component/s client-libs [ 10307 ]
        Component/s client-libs [ 10700 ]
        Affects Version/s 9 [ 14949 ]
        Affects Version/s 9 [ 16400 ]
        pardesha Pardeep Sharma made changes -
        Subcomponent 2d [ 482 ] 2d [ 197 ]
        pardesha Pardeep Sharma made changes -
        Assignee Pardeep Sharma [ pardesha ]
        pardesha Pardeep Sharma made changes -
        Status Open [ 1 ] New [ 10000 ]
        vdyakov Victor Dyakov made changes -
        Fix Version/s 10 [ 16302 ]
        vdyakov Victor Dyakov made changes -
        Assignee Philip Race [ prr ]
        Hide
        prr Philip Race added a comment -
        This looks like a D3D-specific bug. I can reproduce on Windows with D3D but it works fine
        with OpenGL or software pipelines.
        Show
        prr Philip Race added a comment - This looks like a D3D-specific bug. I can reproduce on Windows with D3D but it works fine with OpenGL or software pipelines.
        prr Philip Race made changes -
        Status New [ 10000 ] Open [ 1 ]
        prr Philip Race made changes -
        Summary Java SDK 9 hidpi rescaling Image scaling is bad with D3D pipeline
        Hide
        prr Philip Race added a comment -
        The workaround is -Dsun.java2d.d3d=false
        Show
        prr Philip Race added a comment - The workaround is -Dsun.java2d.d3d=false
        prr Philip Race made changes -
        Assignee Philip Race [ prr ] Prahalad Kumar Narayanan [ pnarayanan ]
        psrivastava Praveen Srivastava made changes -
        Labels dcspks hidpi regression reproducer-yes webbug dcspks hidpi regression reproducer-yes sn_5 webbug
        pnarayanan Prahalad Kumar Narayanan made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Hide
        pnarayanan Prahalad Kumar Narayanan added a comment -
        Quick observations on this issue
        . I tested this issue on a D3D compatible Windows10 desktop running JDK10 build.
        . Yes the issue could be reproduced & I could see distortions with higher DPI settings.
        . Interestingly, the distortion was observed only with non-integer scale factors. Scale factors of 2.0, 3.0 didn't distort the image.
        . I will continue to inspect on this issue and update when I have concrete findings.
        Show
        pnarayanan Prahalad Kumar Narayanan added a comment - Quick observations on this issue . I tested this issue on a D3D compatible Windows10 desktop running JDK10 build. . Yes the issue could be reproduced & I could see distortions with higher DPI settings. . Interestingly, the distortion was observed only with non-integer scale factors. Scale factors of 2.0, 3.0 didn't distort the image. . I will continue to inspect on this issue and update when I have concrete findings.
        pardesha Pardeep Sharma made changes -
        Introduced In Version 9 [ 14949 ]
        pardesha Pardeep Sharma made changes -
        Assignee Prahalad Kumar Narayanan [ pnarayanan ] Pardeep Sharma [ pardesha ]
        vdyakov Victor Dyakov made changes -
        Affects Version/s 10 [ 16302 ]
        pardesha Pardeep Sharma made changes -
        Assignee Pardeep Sharma [ pardesha ] Prahalad Kumar Narayanan [ pnarayanan ]
        psrivastava Praveen Srivastava made changes -
        Labels dcspks hidpi regression reproducer-yes sn_5 webbug dcspks hidpi regression reproducer-yes sn_5 sn_6 webbug
        Hide
        pnarayanan Prahalad Kumar Narayanan added a comment - - edited
        The issue is tricky.
        Here are my inferences while inspecting this issue-

        . The output of image rendering (embedded in a Label) differs across multiple iterations-
              . Iteration 1:
                    . The input BufferedImage is blitted onto an accelerated surface using D3DSwToSurfaceScale.
                    . Artifacts are not observed during this operation.
                    . However, D3DSurfaceDataProxy attempts to cache the buffered image on GPU.
              . Iteration 2:
                    . In the second iteration, the buffered image gets cached by D3DSurfaceDataProxy using D3DSwToTextureBlit
                    . Once caching is complete, a valid D3DSurfaceData for cached BufferedImage is returned for the blitting operation.
                    . Thus, the logic for blitting gets changed to D3DTextureToSurfaceScale. It is here that the artifacts are observed.

        . While debugging the code, I found that the Blit logic further differs based on the Interpolation type- Nearest Neighbor, Bi-Linear, Bi-cubic
        . When experimented with different interpolation values, I observed that, the issue is visible with Nearest neighbor interpolation alone.
              . With Bi-linear and Bi-cubic interpolations- D3DTextureToSurfaceTransform gets executed.
              . Whereas with Nearest neighbor interpolation- D3DTextureToSurfaceScale gets executed.

        . Both the above methods of D3DBlitLoops- D3DTextureToSurfaceScale, D3DTextureToSurfaceTransform get realized with IsoBlit. Hence the backend logic at the native code is the same. It's the interpolation that results in such artifacts.
        . Changing the interpolation from Nearest Neighbor to Bilinear in D3DTextureToSurfaceScale fixes the issue.

        . In another observation, the values for dx1, dx2 were found negative within D3DTextureToSurfaceScale but this did not occur always.
        . Hence the appearance of artifacts in connection to the above values could not be confirmed.
        Show
        pnarayanan Prahalad Kumar Narayanan added a comment - - edited The issue is tricky. Here are my inferences while inspecting this issue- . The output of image rendering (embedded in a Label) differs across multiple iterations-       . Iteration 1:             . The input BufferedImage is blitted onto an accelerated surface using D3DSwToSurfaceScale.             . Artifacts are not observed during this operation.             . However, D3DSurfaceDataProxy attempts to cache the buffered image on GPU.       . Iteration 2:             . In the second iteration, the buffered image gets cached by D3DSurfaceDataProxy using D3DSwToTextureBlit             . Once caching is complete, a valid D3DSurfaceData for cached BufferedImage is returned for the blitting operation.             . Thus, the logic for blitting gets changed to D3DTextureToSurfaceScale. It is here that the artifacts are observed. . While debugging the code, I found that the Blit logic further differs based on the Interpolation type- Nearest Neighbor, Bi-Linear, Bi-cubic . When experimented with different interpolation values, I observed that, the issue is visible with Nearest neighbor interpolation alone.       . With Bi-linear and Bi-cubic interpolations- D3DTextureToSurfaceTransform gets executed.       . Whereas with Nearest neighbor interpolation- D3DTextureToSurfaceScale gets executed. . Both the above methods of D3DBlitLoops- D3DTextureToSurfaceScale, D3DTextureToSurfaceTransform get realized with IsoBlit. Hence the backend logic at the native code is the same. It's the interpolation that results in such artifacts. . Changing the interpolation from Nearest Neighbor to Bilinear in D3DTextureToSurfaceScale fixes the issue. . In another observation, the values for dx1, dx2 were found negative within D3DTextureToSurfaceScale but this did not occur always. . Hence the appearance of artifacts in connection to the above values could not be confirmed.
        Hide
        pnarayanan Prahalad Kumar Narayanan added a comment - - edited
        . I modified the default interpolation at D3DTextureToSurfaceScale from nearest neighbor to bilinear interpoloation.
        . While this fixed the issue, it caused a regression failure at- sun/java2d/pipe/InterpolationQualityTest.java
        . The test failed because, it specifically tested and compared output images with Nearest Neighbor interpolation but the code change silently used bilinear instead in the D3DBlitLoops.

        . This brings us to a point wherein the fix isn't straight forward.
        . While user can set either Bilinear or Bicubic interpolation for all custom rendering done with Graphics2D object, it 's not possible (from end-user's program) to set interpolation while rendering swing components that contain images.

        . I will need time to discuss this issue in detail with the submitter and plan a course of action. Hence deferring this issue.
        Show
        pnarayanan Prahalad Kumar Narayanan added a comment - - edited . I modified the default interpolation at D3DTextureToSurfaceScale from nearest neighbor to bilinear interpoloation. . While this fixed the issue, it caused a regression failure at- sun/java2d/pipe/InterpolationQualityTest.java . The test failed because, it specifically tested and compared output images with Nearest Neighbor interpolation but the code change silently used bilinear instead in the D3DBlitLoops. . This brings us to a point wherein the fix isn't straight forward. . While user can set either Bilinear or Bicubic interpolation for all custom rendering done with Graphics2D object, it 's not possible (from end-user's program) to set interpolation while rendering swing components that contain images. . I will need time to discuss this issue in detail with the submitter and plan a course of action. Hence deferring this issue.
        pnarayanan Prahalad Kumar Narayanan made changes -
        Fix Version/s 11 [ 18723 ]
        Fix Version/s 10 [ 16302 ]
        psrivastava Praveen Srivastava made changes -
        Labels dcspks hidpi regression reproducer-yes sn_5 sn_6 webbug dcspks hidpi regression reproducer-yes sn_5 sn_6 sn_7 webbug
        Hide
        pnarayanan Prahalad Kumar Narayanan added a comment -
        . As updated in the comments, the root cause stems from the use of Nearest Neighbor interpolation (chosen as default when no interpolation hint is set) while rendering the swing components.
        . Nearest neighbor interpolation has been chosen as the default for performance reasons. Setting interpolation to Bilinear or Bicubic will incur huge performance loss for other swing components.

        . Since the default choice of interpolation cannot be changed, there are different work around options to avoid artifacts.
        . Unfortunately, the intent of the bug is not clear from the test-case present in the description.

        . If the submitter's intent is to avoid artifacts while rendering BufferedImages on D3D surfaces, the work around would be to explicitly set rendering hint before drawing an image.
              Graphics2D gfx2D = (Graphics2D) g;
              RenderingHints rHints = new RenderingHints(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR);
              gfx2D.addRenderingHints(rHints);

        . If the submitter's intent is to avoid artifacts while rendering ImageIcons within JLabel, the work around would be one of the two options-
              . Disable UI scaling with VM option: -Dsun.java2d.uiScale=1
              . Use MRI image (Multi-Resolution Image) instead of BufferedImage for ImageIcon. Note: A multi-resolution image is a collection of individual images at different scale factors.
        Show
        pnarayanan Prahalad Kumar Narayanan added a comment - . As updated in the comments, the root cause stems from the use of Nearest Neighbor interpolation (chosen as default when no interpolation hint is set) while rendering the swing components. . Nearest neighbor interpolation has been chosen as the default for performance reasons. Setting interpolation to Bilinear or Bicubic will incur huge performance loss for other swing components. . Since the default choice of interpolation cannot be changed, there are different work around options to avoid artifacts. . Unfortunately, the intent of the bug is not clear from the test-case present in the description. . If the submitter's intent is to avoid artifacts while rendering BufferedImages on D3D surfaces, the work around would be to explicitly set rendering hint before drawing an image.       Graphics2D gfx2D = (Graphics2D) g;       RenderingHints rHints = new RenderingHints(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BILINEAR);       gfx2D.addRenderingHints(rHints); . If the submitter's intent is to avoid artifacts while rendering ImageIcons within JLabel, the work around would be one of the two options-       . Disable UI scaling with VM option: -Dsun.java2d.uiScale=1       . Use MRI image (Multi-Resolution Image) instead of BufferedImage for ImageIcon. Note: A multi-resolution image is a collection of individual images at different scale factors.
        pnarayanan Prahalad Kumar Narayanan made changes -
        Status In Progress [ 3 ] Open [ 1 ]
        Understanding Cause Known [ 10000 ]
        pnarayanan Prahalad Kumar Narayanan made changes -
        Labels dcspks hidpi regression reproducer-yes sn_5 sn_6 sn_7 webbug dcspks hidpi regression reproducer-yes sn_5 sn_6 webbug
        Hide
        pnarayanan Prahalad Kumar Narayanan added a comment -
        . We discussed with the submitter of the issue and got to know the intent - Rendering BufferedImages smooth on scaled resolutions.
        . One of the workarounds suggested in the comments addresses this need. Hence closing the issue.
        Show
        pnarayanan Prahalad Kumar Narayanan added a comment - . We discussed with the submitter of the issue and got to know the intent - Rendering BufferedImages smooth on scaled resolutions. . One of the workarounds suggested in the comments addresses this need. Hence closing the issue.
        pnarayanan Prahalad Kumar Narayanan made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Understanding Cause Known [ 10000 ]
        Resolution Won't Fix [ 2 ]
        pnarayanan Prahalad Kumar Narayanan made changes -
        Fix Version/s 11 [ 18723 ]
        pnarayanan Prahalad Kumar Narayanan made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        serb Sergey Bylokhov added a comment -
        Providing a workaround for the user does not mean that the bug is fixed. We can take care of it later since it is not urgent now.
        Show
        serb Sergey Bylokhov added a comment - Providing a workaround for the user does not mean that the bug is fixed. We can take care of it later since it is not urgent now.
        serb Sergey Bylokhov made changes -
        Resolution Won't Fix [ 2 ]
        Status Closed [ 6 ] Open [ 1 ]
        vdyakov Victor Dyakov made changes -
        Fix Version/s 11 [ 18723 ]
        vdyakov Victor Dyakov made changes -
        Assignee Prahalad Kumar Narayanan [ pnarayanan ] Jayathirth D V [ jdv ]
        jdv Jayathirth D V made changes -
        Status Open [ 1 ] In Progress [ 3 ]

          People

          • Assignee:
            jdv Jayathirth D V
            Reporter:
            webbuggrp Webbug Group
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated: