Automate mouse clicks with Mathematica

9

1

I need to automate mouse clicks with Mathematica. However, a similiar question was closed on this forum, without any answer.

Consider the following steps:

 Dynamic[MousePosition[]]

In this first step I dynamically show the mouse position.

 DynamicModule[{col = Green}, EventHandler[ Style["text", FontColor -> 
 Dynamic[col]], {"MouseClicked" :> (col = col /. {Red -> Green, Green -> Red})}]]

In this second step I create a "clickable" text: when you click it, it changes its color between green and red.

My objective here is twofold:

1) automatically move the cursor position - this has already been answered here. So, following the original answer I can write

Needs["JLink`"];
ReinstallJava[];
robotclass = JavaNew["java.awt.Robot"];
robotclass@mouseMove[90, 196];

2) automatically click on the clickable text (and, of course, I want to see its color changing). Here is the code I'm trying to use:

robotclass@mousePress[InputEvent.BUTTON1_MASK];
robotclass@mouseRelease[InputEvent.BUTTON1_MASK];

Although I believe I'm using the correct functions (which belong to the Java Robot Class), I'm getting the following error message:

Java::argx1: Method named mouseRelease defined in class java.awt.Robot was called with an incorrect number or type of arguments. The argument was InputEvent.BUTTON1_MASK.

Can anybody give me a hint to solve this problem?

Rod

Posted 2013-08-19T02:32:41.267

Reputation: 3 238

Answers

12

You can use the raw integer value 16, but the correct J/Link syntax for the symbolic name is InputEvent`BUTTON1UMASK, so your code would look like this:

(* Call LoadJavaClass when needing to refer to static members. *)
LoadJavaClass["java.awt.event.InputEvent"];

robotclass@mousePress[InputEvent`BUTTON1UMASK];
robotclass@mouseRelease[InputEvent`BUTTON1UMASK];

Enter JLink/tutorial/CallingJavaFromMathematica#15615 into the Documentation Center to see the section of the J/Link docs that talks about how to construct this name. BUTTON1_MASK is a static member of the InputEvent class, hence the `, and the _ character needs to be converted into a U to make it a legal Mathematica symbol name.

Todd Gayley

Posted 2013-08-19T02:32:41.267

Reputation: 4 078

It also works! +1 – Rod – 2013-08-19T10:27:49.057

hmm wonder why this syntax didn't occur to me. @RodLm feel free to change the accepted question – amr – 2013-08-19T14:37:39.593

5

InputEvent.BUTTON1_MASK is an attempt to get the dot product of InputEvent and BUTTON1_MASK, neither of which are matrices. You're using Java syntax in a Mathematica program.

I don't know what the idiomatic JLink approach is, but if you look here you can see that InputEvent.BUTTON1_MASK has the integer value 16 (the BUTTON1_MASK is just a variable for programmer convenience -- easier to deal with than an arbitrary integer).

amr

Posted 2013-08-19T02:32:41.267

Reputation: 5 317

I've tried other sintaxes as well... for instance, instead of InputEvent.BUTTON1_MASK I've also tried InputEvent[BUTTON1_MASK], mouseRelease["InputEvent.BUTTON1_MASK"], 'mouseRelease[InputEvent,BUTTON1_MASK]' and so on... but nothing seems to work... By the way, you could have put a comment instead of an answer... anyway, thank you. – Rod – 2013-08-19T03:42:29.580

@RodLm for some reason i had assumed you were familiar with Java. to be clearer: the mousePress function expects an integer. mousePress(InputEvent.BUTTON1_MASK) works in Java because InputEvent.BUTTON1_MASK is an integer in Java. what you do with this knowledge is something i leave to you :) – amr – 2013-08-19T03:55:23.827

+1! It works... You only need to use robotclass@mousePress[16] and it works perfectly. Thanks! – Rod – 2013-08-19T03:58:34.527