Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8249592

Robot.mouseMove moves cursor to incorrect location when display scale varies

    Details

    • Subcomponent:
    • CPU:
      x86_64
    • OS:
      windows_10

      Description

      ADDITIONAL SYSTEM INFORMATION :
      Windows 10

      A DESCRIPTION OF THE PROBLEM :
      Using java.awt.Robot.mouseMove() to move the cursor to a location on a secondary monitor with a different scale factor than the primary monitor does not work correctly. It seems to be using a different coordinate system than the other swing/awt methods that deal with mouse position. The problem occurs regardless of which GraphicsDevice is passed into the Robot constructor.

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Setup:
      - java.exe must be launched in "DPI Unaware" mode to trigger this bug. It does *not* occur when java.exe is running in "DPI Aware" mode.
      - You can force the DPI mode to "Unaware" by setting the scaling mode to "System" in "Change DPI Settings" in the "Compatibility" tab of the windows properties dialog for java.exe (we must do this for our app to avoid more severe bugs that occur in "DPI Aware" mode)

      - Screen 1 (Primary, Left):
      -- Resolution: 3840 x 2160
      -- Scale Factor: 150%

      - Screen 2 (Right):
      -- Resolution: 1920 x 1200
      -- Scale Factor: 100%

      - When the mouse is on Screen 2, sampling the mouse location (either via MouseInfo or via an event listener) and feeding the position to Robot.mouseMove causes the cursor to move to a totally different location. Simple demo app is attached. It seems like Robot is using a different coordinate system than everything else.
      - Run attached test case. Clicking on either panel while the window is located on the right screen causes the cursor to be moved to the far right edge of the right monitor.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      Passing the current mouse screen location to Robot.mouseMove should not cause the cursor to move at all.
      ACTUAL -
      Passing the current mouse screen location to Robot.mouseMove causes the cursor to move dramatically.

      ---------- BEGIN SOURCE ----------
      import java.awt.Color;
      import java.awt.Dimension;
      import java.awt.GraphicsDevice;
      import java.awt.GraphicsEnvironment;
      import java.awt.GridLayout;
      import java.awt.MouseInfo;
      import java.awt.Point;
      import java.awt.Robot;
      import java.awt.event.MouseAdapter;
      import java.awt.event.MouseEvent;
      import java.util.Arrays;

      import javax.swing.BorderFactory;
      import javax.swing.JFrame;
      import javax.swing.JPanel;
      import javax.swing.SwingUtilities;

      public class RobotScalingTest {

      public static void main(String[] args) {

      GraphicsDevice[] screens = GraphicsEnvironment.getLocalGraphicsEnvironment().getScreenDevices();

      JFrame roboTestFrm = new JFrame("Robot Test");
      roboTestFrm.setDefaultCloseOperation(JFrame.DISPOSE_ON_CLOSE);
      roboTestFrm.setLayout(new GridLayout(1, screens.length));

      for (int screenNum=0; screenNum < screens.length; screenNum++) {
      GraphicsDevice device = screens[screenNum];

      JPanel pan = new JPanel();
      pan.setPreferredSize(new Dimension(200, 200));
      pan.setBorder(BorderFactory.createLineBorder(Color.GRAY, 5));
      pan.setBackground(Color.BLACK);

      pan.addMouseListener(new MouseAdapter() {
      @Override
      public void mouseReleased(MouseEvent e) {
      Point panelCoords = e.getPoint();

      System.out.println(Arrays.toString(device.getConfigurations()));

      Point screenCoordsViaConversion = new Point(panelCoords);
      SwingUtilities.convertPointToScreen(screenCoordsViaConversion, pan);

      Point screenCoordsViaMouseEvent = e.getLocationOnScreen();
      Point screenCoordsViaMouseInfo = MouseInfo.getPointerInfo().getLocation();

      // Note: These 3 points always seem to match / agree
      System.out.println("From MouseInfo: "+screenCoordsViaMouseInfo);
      System.out.println("From convertPointToScreen: "+screenCoordsViaConversion);
      System.out.println("From MouseEvent: "+screenCoordsViaMouseEvent);

      try {
      // The results of the "mouseMove" call appear to be the same regardless of which device is passed to the CTOR here
      Robot robo = new Robot(device);
      robo.mouseMove(screenCoordsViaConversion.x, screenCoordsViaConversion.y);
      } catch (Exception e1) {
      e1.printStackTrace();
      }

      }
      });

      roboTestFrm.add(pan);
      }

      roboTestFrm.pack();
      roboTestFrm.setVisible(true);
      }
      }
      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      We have not found a workaround.

      FREQUENCY : always


        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                serb Sergey Bylokhov
                Reporter:
                webbuggrp Webbug Group
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: