I was attempting to create a GLFW infinite mouse mode for Swing and here's my progress so far
package javax.swing.extras;
// AWT Imports
import java.awt.AWTException;
import java.awt.Container;
import java.awt.Cursor;
import java.awt.Point;
import java.awt.Robot;
import java.awt.Toolkit;
import java.awt.event.ComponentAdapter;
import java.awt.event.ComponentEvent;
import java.awt.event.MouseAdapter;
import java.awt.event.MouseEvent;
import java.awt.event.MouseMotionListener;
import java.awt.image.BufferedImage;
// Swing Imports
import javax.swing.JFrame;
/**
* An extension class that allows JFrames to have infinite input as seen in most
* 3D games.
*/
public class InfiniteMouse {
/** Image for the blank cursor */
private static final BufferedImage cursorImg = new BufferedImage(16, 16, BufferedImage.TYPE_INT_ARGB);
/** Blank cursor */
private static final Cursor blankCursor = Toolkit.getDefaultToolkit().createCustomCursor(cursorImg, new Point(0, 0),
"blank cursor");
/** Mouse X */
private int x;
/** Mouse Y */
private int y;
/** Previous Mouse X */
private int previousX;
/** Previous Mouse Y */
private int previousY;
/** Center X */
private int centerX;
/** Center Y */
private int centerY;
/**
* Create a InfiniteMouse extension for a JFrame
*
* @param f - The JFrame to add the extension
* @param c - The Container to detect mouse motion
* @return an instance of InfiniteMouse containing the x and y
* @throws AWTException if the platform configuration does not allowlow-level
* input control. This exception is always thrown
* whenGraphicsEnvironment.isHeadless() returns true
*/
public static InfiniteMouse addExtension(JFrame f, Container c) throws AWTException {
// Create a new instance
InfiniteMouse infiniteMouse = new InfiniteMouse();
// Create a final Robot class to later move the mouse to the center
final Robot robot = new Robot();
// Store the on-screen center x of the window
infiniteMouse.centerX = f.getX() + f.getWidth() / 2;
// Store the on-screen center y of the window
infiniteMouse.centerY = f.getY() + f.getHeight() / 2;
// Add a resize listener
f.addComponentListener(new ComponentAdapter() {
// If the frame is resized
public void componentResized(ComponentEvent e) {
// Update the X position
infiniteMouse.centerX = f.getX() + f.getWidth() / 2;
// Update the Y position
infiniteMouse.centerY = f.getY() + f.getHeight() / 2;
}
});
// Make the cursor invisible
f.setCursor(blankCursor);
// Add a mouse listener
c.addMouseMotionListener(new MouseMotionListener() {
// If the mouse is moved
public void mouseMoved(MouseEvent e) {
// Get the current X
int currentX = e.getX();
// Get the current Y
int currentY = e.getY();
System.out.println(infiniteMouse.x);
// Get the mouse X position delta
infiniteMouse.x += currentX - infiniteMouse.previousX;
// Get the mouse Y position delta
infiniteMouse.y += currentY - infiniteMouse.previousY;
// Remove the listener when moving to the center
c.removeMouseMotionListener(this);
// Move the mouse to the center
robot.mouseMove(infiniteMouse.centerX, infiniteMouse.centerY);
// Add the listener back
c.addMouseMotionListener(this);
// Update the previous X
infiniteMouse.previousX = currentX;
// Update the previous Y
infiniteMouse.previousY = currentY;
}
@Override
public void mouseDragged(MouseEvent e) {
}
});
// Return the newly created instance
return infiniteMouse;
}
/**
* Get the X position
*
* @return the X position
*/
public int getX() {
return x;
}
/**
* Get the Y position
*
* @return the Y position
*/
public int getY() {
return y;
}
}
Used in this code
package javax.swing.extras;
import java.awt.AWTException;
import java.awt.Color;
import java.awt.GridBagLayout;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JPanel;
public class Test {
public static void main(String[] args) throws AWTException {
JFrame jf = new JFrame();
JPanel panel1 = new JPanel();
jf.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
jf.setTitle("Window");
jf.setSize(800,600);
jf.setLocationRelativeTo(null);
jf.add(panel1);
panel1.grabFocus();
InfiniteMouse.addExtension(jf, panel1);
jf.setVisible(true);
}
}
However the issue I face with this code is that the mouse X being debugged in the console is really not increasing, and I suspect it to be the issue with these lines
c.removeMouseMotionListener(this);
robot.mouseMove(infiniteMouse.centerX, infiniteMouse.centerY);
c.addMouseMotionListener(this);
As it stays around the same value unless I tab out and move the JFrame around and move it back into the window. Well, the purpose of this line is to suspend the mouse motion listener so that I can move the mouse back and then resume the listener. Is there a working alternative to this method that I can use? or is it some other issue?
The following snippet in your code:
// Remove the listener when moving to the center
c.removeMouseMotionListener(this);
// Move the mouse to the center
robot.mouseMove(infiniteMouse.centerX, infiniteMouse.centerY);
// Add the listener back
c.addMouseMotionListener(this);
should have the same effect as the following one only:
// Move the mouse to the center
robot.mouseMove(infiniteMouse.centerX, infiniteMouse.centerY);
The Event Dispatch Thread (EDT) is a single thread and as far as I know all Swing events are posted to it. The mouseMoved method is called on the EDT. You are not going to get mouseMoved calls interleaved, but instead you get them sequentially. So, in the first snippet, you are essentially removing and re-adding the listener to the component on the same thread, inside the same fired event code. As far as I understand, the robot.mouseMove will eventually create another mouseMoved event on the EDT queue, but since EDT is a single thread and all events in its queue are executed sequentially, then the mouseMoved event generated by the robot.mouseMove will be dispatched at some point after the current mouseMoved call. But the listener always exists in the component (as long as we are outside mouseMoved). When the event generated by robot.mouseMove is dispatched, the listeners of the component will be called, which will include the listener of the first snippet (for the aforementioned reasons and code).
Some other minor considerations are:
// Get the mouse Y position delta
infiniteMouse.y += currentY - infiniteMouse.previousY;
// Get the mouse Y position delta
infiniteMouse.y += infiniteMouse.previousY - currentY;
Back to finding a solution, since robot.mouseMove is expected to generate mouse events, then one solution could be to discard them at some point of their dispatch. The most convenient way I guess is inside the mouse listener, the last destination of the event which we can control easily. Nevertheless, I couldn't find an easy (or non-hypothetic) way to distinguish between user and robot generated mouse move events inside the listener. I am also assuming that we cannot know when the robot.mouseMove will actually complete the native movement of the cursor (this is something I couldn't find guarantee of happening, inside the documentation). It could be for example the case that robot.mouseMove only posts a request to the OS to move the mouse and then immediately returns. So we can't be certain when that request completes. Also, we cannot block waiting for the mouse to reach the destination because we will lose information on what actually caused the mouse to move to destination (was it the user? Was it only the Robot? A combination of their movements? etc). Also, robot.waitForIdle cannot be called inside EDT (throws an exception), so that would require an extra thread, but its documentation states that:
Waits until all events currently on the event queue have been processed.
...which does not (at least for me) mean that the native events will be completed. Processing the events in the (Robot's) queue for me just means that the request to the OS was successful, but that does not mean the OS completed it yet. Then I realised that we can just cancel the Robot's events out proactively because we know the event that is going to be generated by the robot.mouseMove, since at any point we know the current location of the mouse, as well as where we want it to go. The delta resulting from these two locations can be subtracted from the InfiniteMouse location just before (or just after) the robot.mouseMove is called from our code. When trying to write this in code, I ended up using an even simpler concept: just assume the mouse is always (after each mouseMoved call) at the same location and don't maintain a previous location (we always know/assume it's on the location we specified). Of course this requires to call robot.mouseMove to that constant location on each mouseMoved event, so that the assumption holds:
import java.awt.AWTException;
import java.awt.BorderLayout;
import java.awt.Component;
import java.awt.Dimension;
import java.awt.FlowLayout;
import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.Point;
import java.awt.Robot;
import java.awt.event.ActionEvent;
import java.awt.event.KeyEvent;
import java.awt.event.MouseEvent;
import java.beans.PropertyChangeListener;
import java.beans.PropertyChangeSupport;
import java.util.Objects;
import javax.swing.AbstractAction;
import javax.swing.Action;
import javax.swing.BorderFactory;
import javax.swing.JButton;
import javax.swing.JComponent;
import javax.swing.JFrame;
import javax.swing.JLabel;
import javax.swing.JPanel;
import javax.swing.KeyStroke;
import javax.swing.SwingUtilities;
import javax.swing.event.MouseInputAdapter;
public class Main {
/** A model for a two dimensional point, with integer coordinates. */
public static interface IntXYModel {
/*public static final*/ String PROPERTY_XY = "xy";
void setXY(final Point xy);
Point getXY();
void addPropertyChangeListener(final PropertyChangeListener listener);
void removePropertyChangeListener(final PropertyChangeListener listener);
/*There is only one property in this model currently (PROPERTY_XY), but the model may be
subinterfaced to include more properties. In that case there needs to be a way to specify
which property we are interested in. Clients accepting plain IntXYModels do not have to know
if there exist more properties in them, in which case they should use the following
overloads (instead of the above methods which do not accept 'propertyName').*/
void addPropertyChangeListener(final String propretyName, final PropertyChangeListener listener);
void removePropertyChangeListener(final String propretyName, final PropertyChangeListener listener);
}
/** A basic implementation of a plain {@link IntXYModel}. */
public static class DefaultIntXYModel implements IntXYModel {
private final PropertyChangeSupport support;
private final Point xy;
public DefaultIntXYModel(final Point xy) {
this.xy = new Point(xy); //Defensive copy.
support = new PropertyChangeSupport(this);
}
@Override
public void setXY(final Point xy) {
if (!Objects.equals(xy, this.xy)) {
final Point oldXY = new Point(this.xy);
this.xy.setLocation(xy);
support.firePropertyChange(PROPERTY_XY, oldXY, xy);
}
}
@Override
public Point getXY() {
return new Point(xy); //Defensive copy.
}
@Override
public void addPropertyChangeListener(final PropertyChangeListener listener) {
if (listener != null)
support.addPropertyChangeListener(listener);
}
@Override
public void removePropertyChangeListener(final PropertyChangeListener listener) {
if (listener != null)
support.removePropertyChangeListener(listener);
}
@Override
public void addPropertyChangeListener(final String propretyName,
final PropertyChangeListener listener) {
if (listener != null)
support.addPropertyChangeListener(propretyName, listener);
}
@Override
public void removePropertyChangeListener(final String propretyName,
final PropertyChangeListener listener) {
if (listener != null)
support.removePropertyChangeListener(propretyName, listener);
}
}
/** A grid drawing panel. Essentially used as a visual clue to the user for the movement of the grid (with the mouse). */
public static class DrawPanel extends JPanel {
private IntXYModel currentLocationModel, strideModel;
/** A {@code PropertyChangeListener} which calls {@link #repaint() repaint}. Both our models need only this common listener. */
private final PropertyChangeListener repaintListener;
/**
* @param currentLocationModel A model for the location we are currently in the grid.
* @param strideModel A model for the spacing between drawn lines of the grid (essentially the cell size).
*/
public DrawPanel(final IntXYModel currentLocationModel,
final IntXYModel strideModel) {
this.currentLocationModel = Objects.requireNonNull(currentLocationModel);
this.strideModel = Objects.requireNonNull(strideModel);
repaintListener = evt -> repaint();
//We are specifying explicitly the PROPERTY_XY constant for the listener, because we are only interested in it. See IntXYModel comments.
currentLocationModel.addPropertyChangeListener(IntXYModel.PROPERTY_XY, repaintListener);
strideModel.addPropertyChangeListener(IntXYModel.PROPERTY_XY, repaintListener);
super.setBorder(BorderFactory.createLineBorder(super.getForeground(), 4));
}
public IntXYModel getCurrentLocationModel() {
return currentLocationModel;
}
public IntXYModel getStrideModel() {
return strideModel;
}
public void setCurrentLocationModel(final IntXYModel currentLocationModel) {
Objects.requireNonNull(currentLocationModel);
if (currentLocationModel != this.currentLocationModel) {
//We are specifying explicitly the PROPERTY_XY constant for the listener, because we are only interested in it. See IntXYModel comments.
this.currentLocationModel.removePropertyChangeListener(IntXYModel.PROPERTY_XY, repaintListener);
this.currentLocationModel = currentLocationModel;
currentLocationModel.addPropertyChangeListener(IntXYModel.PROPERTY_XY, repaintListener);
}
}
public void setStrideModel(final IntXYModel strideModel) {
Objects.requireNonNull(strideModel);
if (strideModel != this.strideModel) {
//We are specifying explicitly the PROPERTY_XY constant for the listener, because we are only interested in it. See IntXYModel comments.
this.strideModel.removePropertyChangeListener(IntXYModel.PROPERTY_XY, repaintListener);
this.strideModel = strideModel;
strideModel.addPropertyChangeListener(IntXYModel.PROPERTY_XY, repaintListener);
}
}
@Override
protected void paintComponent(final Graphics g) {
super.paintComponent(g);
//Standard metrics (panel size and center, as well as grid location and stride):
final int w = getWidth(), h = getHeight();
final int cx = w / 2, cy = h / 2;
final Point location = currentLocationModel.getXY(),
stride = strideModel.getXY();
final Graphics2D g2d = (Graphics2D) g.create();
try {
// //Draw a RadialGradientPaint to help the user navigate visually back to the origin (reduces EventQueue perceived experience a lot).
// final Paint originalPaint = g2d.getPaint();
// g2d.setPaint(new RadialGradientPaint(cx - location.x, h - cy + location.y, 1000, new float[]{0, 1}, new Color[]{Color.CYAN, Color.BLUE.brighter().brighter()}, MultipleGradientPaint.CycleMethod.REFLECT));
// g2d.fill(g2d.getClipBounds());
// g2d.setPaint(originalPaint);
//Draw a 'X' in the center of the panel (just so the user always knows where the mouse supposedly is):
final int fillSideSpace = 10;
g2d.drawLine(cx - fillSideSpace, cy - fillSideSpace, cx + fillSideSpace, cy + fillSideSpace);
g2d.drawLine(cx + fillSideSpace, cy - fillSideSpace, cx - fillSideSpace, cy + fillSideSpace);
//Draw grid:
final int verticalLinesForW = Math.max(1, (w + stride.x - 1) / stride.x),
horizontalLinesForH = Math.max(1, (h + stride.y - 1) / stride.y);
final int verticalLinesWhichForHalfW = (verticalLinesForW + 1) / 2,
horizontalLinesWhichForHalfH = (horizontalLinesForH + 1) / 2;
final int leftMostVisibleVerticalLineX = cx - location.x % stride.x - (verticalLinesWhichForHalfW - 1) * stride.x,
topMostVisibleHorizontalLineY = cy - location.y % stride.y - (horizontalLinesWhichForHalfH - 1) * stride.y;
//g2d.setStroke(new BasicStroke(6));
for (int i = -1; i <= verticalLinesForW; ++i)
g2d.drawLine(leftMostVisibleVerticalLineX + i * stride.x, 0, leftMostVisibleVerticalLineX + i * stride.x, h);
for (int i = -1; i <= horizontalLinesForH; ++i)
g2d.drawLine(0, h - topMostVisibleHorizontalLineY - i * stride.y, w, h - topMostVisibleHorizontalLineY - i * stride.y);
//Draw a String to indicate the grid origin (0,0):
//g2d.setColor(Color.GREEN.darker());
g2d.setFont(g.getFont().deriveFont(g.getFont().getSize2D() * 1.5f));
g2d.drawString("Origin", cx - location.x, h - cy + location.y);
}
finally {
g2d.dispose();
}
}
@Override
public Dimension getPreferredSize() {
final Dimension preferredSize = super.getPreferredSize();
if (isPreferredSizeSet())
return preferredSize;
return new Dimension(Math.max(900, preferredSize.width), Math.max(600, preferredSize.height)); //Give some room for the drawing...
}
}
/**
* Adds infinite mouse capability to a {@code JComponent}. The mouse capturing functionality is
* enabled when the user presses the mouse inside the component, and disabled when the user
* types the given {@code escapeKeyStroke}.
* @param c The component which will capture the mouse.
* @param locationModel The model of the location inside the 2D grid. This will be updated when
* the funcionality is enabled and the mouse is moved by the user.
* @param escapeKeyStroke A Key Binding will be created for this {@code KeyStroke} which, when
* typed, will disable mouse capturing functionality. It is applied to the
* {@linkplain JComponent#getInputMap() input map} ({@code WHEN_FOCUSED}) of the component.
* @return The {@code Action} created for the given {@code escapeKeyStroke}. Its enabled state
* serves as the mouse capturing functionality's enabled state.
* @throws AWTException In case of failing to create the {@link Robot} required to restore mouse
* position after each user's move.
*/
private static Action addInfiniteMouseMovement(final JComponent c,
final IntXYModel locationModel,
final KeyStroke escapeKeyStroke) throws AWTException {
final Robot robot = new Robot();
final Action escapeAction = new AbstractAction("Disable mouse capturing") {
@Override
public void actionPerformed(final ActionEvent e) {
setEnabled(false); /*Disabling this action effectively disables mouse capturing
support in the mouse listener which follows the creation of this action.*/
}
};
final MouseInputAdapter mouseListener = new MouseInputAdapter() {
/** The original mouse location, when the mouse capturing started. */
private Point mouseLocation = null;
@Override
public void mouseMoved(final MouseEvent e) {
if (escapeAction.isEnabled()) {
/*Always requesting focus is mandatory, since the escape action is registered
on WHEN_FOCUSED input map. If you remove the requestFocusInWindow() call, then
the escape action won't function!*/
e.getComponent().requestFocusInWindow();
/*Obtain mouse coordinate difference. We know the original mouse location when
the capturing started (see 'mousePressed' implementation), ie 'mouseLocation',
as well as the mouse location of the event. Thus subtracting them gives us the
relative movement of the mouse which we add to the grid. Both locations are on
the same coordinate system (ie screen coordinates). Note that the y axis is
inverted with respect to the component (we assume that the y coordinate in the
grid grows larger as we go upwards).*/
final Point currentMouseLocation = e.getLocationOnScreen();
final int dx = (mouseLocation.x - currentMouseLocation.x),
dy = (currentMouseLocation.y - mouseLocation.y);
//Move grid:
final Point xy = locationModel.getXY();
xy.x += dx;
xy.y += dy;
locationModel.setXY(xy);
/*Always move the mouse to original mouse location... This way the mouse is
reset, allowing for infinite mouse movement effect. Additionally the calculation
of 'dx' and 'dy' (in the next mouse event) will be valid.*/
robot.mouseMove(mouseLocation.x, mouseLocation.y);
}
}
@Override
public void mouseDragged(final MouseEvent e) {
mouseMoved(e); //Assume same behaviour on drag.
}
//When the user presses the mouse in the component, we start mouse capturing...
@Override
public void mousePressed(final MouseEvent e) {
if (!escapeAction.isEnabled()) {
escapeAction.setEnabled(true);
/*Initialize the original mouse location as the center of the component and, of
course move to it:*/
final Component source = e.getComponent();
mouseLocation = new Point(source.getLocationOnScreen());
mouseLocation.x += (source.getWidth() / 2);
mouseLocation.y += (source.getHeight() / 2);
robot.mouseMove(mouseLocation.x, mouseLocation.y);
}
}
};
c.addMouseListener(mouseListener);
c.addMouseMotionListener(mouseListener);
c.setFocusable(true); //Mandatory.
//Add escape Key Binding:
final Object key = new Object();
c.getInputMap().put(escapeKeyStroke, key);
c.getActionMap().put(key, escapeAction);
return escapeAction;
}
public static void main(final String[] args) {
SwingUtilities.invokeLater(() -> {
//Will be used as a key binding which stops mouse capturing:
final KeyStroke escape = KeyStroke.getKeyStroke(KeyEvent.VK_ESCAPE, 0);
//The following 3 objects will be properly initialized after adding listeners...
final IntXYModel locationModel = new DefaultIntXYModel(new Point(-1, -1));
final JLabel locationLabel = new JLabel(),
infoLabel = new JLabel();
final DrawPanel area = new DrawPanel(locationModel, new DefaultIntXYModel(new Point(300, 200)));
try {
final Action escapeAction = addInfiniteMouseMovement(area, locationModel, escape);
escapeAction.addPropertyChangeListener(e -> {
if (Objects.equals(e.getPropertyName(), "enabled"))
infoLabel.setText(
((Boolean) e.getNewValue())
? "Stop mouse capture key: " + escape
: "Left click inside the grid to capture the mouse!"
);
});
//We need to start the program without capturing the mouse (and also initialize 'infoLabel', because of the listener):
escapeAction.setEnabled(false);
locationModel.addPropertyChangeListener(IntXYModel.PROPERTY_XY, e -> {
final Point location = (Point) e.getNewValue();
locationLabel.setText(String.format("Location: (%+06d, %+06d)", location.x, location.y));
});
//Grid starts at location (0,0) supposedly (and also the text of 'locationLabel' is initialized, because of the listener):
locationModel.setXY(new Point());
//Just a button to reset the grid location to its origin:
final JButton resetGrid = new JButton("Reset grid");
resetGrid.addActionListener(e -> locationModel.setXY(new Point()));
//Create frame contents' panels:
final JPanel pageStart = new JPanel(new FlowLayout(FlowLayout.LEADING));
pageStart.add(resetGrid);
pageStart.add(infoLabel);
final JPanel contents = new JPanel(new BorderLayout());
contents.add(area, BorderLayout.CENTER);
contents.add(pageStart, BorderLayout.PAGE_START);
contents.add(locationLabel, BorderLayout.PAGE_END);
//Create and show frame:
final JFrame frame = new JFrame("Infinite mouse move 2D");
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.add(contents);
frame.pack();
frame.setLocationRelativeTo(null);
frame.setVisible(true);
}
catch (final AWTException awtException) {
awtException.printStackTrace();
}
});
}
}
The above code is an MVC design attempt to approach the problem, where grid is assumed to be the 2D map which the user sees in the drawing panel. At some point the deltas are added to the grid's location, moving the grid. I think this could be changed to degrees of sight in a 3D game for example in your case. Click inside the drawing panel to start mouse capturing and press ESCAPE to stop capturing. The code is documented to guide the reader through.