Is it possible to get the locations and dimensions of all boxes in a StandardForm expression renderation?

8

3

I can select all the boxes in a StandardForm rendered expression, like I've selected the left parenthesis in the picture above. Is it possible to get the positions and dimensions of all of these selectable boxes? Clearly the Mathematica rendering engine has this information, it's just a question of getting access to it somehow. Might J/Link provide this access?

I notice that you Accepted my answer. Were you able to complete the idea and produce a working implementation? If so would you consider posting it? If not you might hold off on the Accept in case someone else has a better idea. Thank you in any case. – Mr.Wizard – 2015-05-15T01:34:11.440

5

This is not yet a complete answer but I think it may set you on a viable path. We can style each element using StyleBox as follows:

boxes = RowBox[{"(",
FractionBox[
RowBox[{RowBox[{"(", RowBox[{"a", "+", "b"}], ")"}], SuperscriptBox["c", "d"]}],
SqrtBox["e"]], ")"}];

colors = ColorData[54, "ColorList"]

boxesNew =
Module[{i = 1},
boxes /. s_String :>
(StyleBox[s, FontColor -> #, Background -> #] & @ colors[[i++]])
];


The result:

boxesNew // DisplayForm


The object selected, revealing its parts:

We can create an Image with Rasterize:

img = Rasterize[boxesNew // DisplayForm, "Image"];


This image could then be processed to identify each area of solid color and these mapped back to the color list in colors. (The colors chosen are only for human readability; sequential RGB values might be used for many segments.)

3

Details depend of your specific needs but it seems that information you desire is at least partially accesible via FrontEndUndocumentedGetSelectionPacket.

RowBox[{"(",
FractionBox[
RowBox[{RowBox[{"(", RowBox[{"a", "+", "b"}], ")"}],
SuperscriptBox["c", "d"]}], SqrtBox["e"]], ")"}
] // RawBoxes


Dynamic[
FrontEndExecute @ FrontEndUndocumentedGetSelectionPacket[EvaluationNotebook[]],
UpdateInterval -> 1
]


VirtualSelectionRectangle seems to give {xmin, xmax}, {ymin, ymax} where x are pixel columns counted from left edge of the notebook and y coordinate determines pixel row counting from top edge (NOT from the beginning of the notebook).

This is really neat! But how would one programmatically apply this to every selectable part? This might make for a good self-Q&A that is more directly answered by UndocumentedGetSelectionPacket. – Mr.Wizard – 2016-08-30T00:05:03.827

@Mr.Wizard I suppose a loop with SelectionMove should do that. Not without pain but seems possible. – Kuba – 2016-08-30T05:54:51.813