How to "Copy as Unicode" from a Notebook?

63

44

I spent some time manually editing a post replacing Mathematica ASCII \[Alpha] with Unicode α. I did this by laboriously choosing Copy as LaTeX, pasting into the edit box, and then copying the Unicode symbol from the preview below. This made me realize I am lacking a "Copy as Unicode string" function in Mathematica.

How can I most easily copy an expression such as:

Mathematica graphics

In Unicode:

αβ + Mod[δΨ, 2 ⁢ρ^2]

Mr.Wizard

Posted 2012-02-01T22:11:44.943

Reputation: 259 163

Answers

30

Since a native method is not forthcoming, I shall post my file based circumvention, for Windows.

You will need to have this utility in the command path (it apparently is stock with Windows 7).

copyUnicode[expr_] := Run["clip <",
   Export["$Clipboard.temp", ToString[expr, InputForm],
          "Text", CharacterEncoding -> "Unicode"] ];

Usage:

expr = \[Alpha]\[Beta] + Mod[\[Delta]\[CapitalPsi], 2\[InvisibleTimes]\[Rho]^2];

copyUnicode[expr]

This leaves the following text in the Windows Clipboard:

αβ + Mod[δΨ, 2*ρ^2]

Here is a version of the function that holds (does not evaluate) the expression:

SetAttributes[copyUnicode, HoldFirst]

copyUnicode[expr_, form_: InputForm] := 
  Run["clip <", 
   Export["$Clipboard.temp", ToString[Unevaluated@expr, form], "Text", 
    CharacterEncoding -> "Unicode"]];

Now:

Plot[\[Alpha], {\[Alpha], 0, 10}] // copyUnicode

Puts in the Windows Clipboard:

Plot[α, {α, 0, 10}]

Mr.Wizard

Posted 2012-02-01T22:11:44.943

Reputation: 259 163

3I think clip is native in at least win-7. Perhaps you should note that this places the output in the system clipboard. The output of your copyUnicode function is just a 0 (at least, on my PC). You have to do a ctrl-v paste afterwards. A MMA Paste doesn't seem to work. – Sjoerd C. de Vries – 2012-02-02T21:10:52.090

1Very useful function! Why not make it available for any code sample? (Now it works only for single expression…) – xzczd – 2012-11-23T14:09:22.823

@xzczd I'll see if I can improve it. – Mr.Wizard – 2012-11-23T23:56:17.360

There's a built-in CopyToClipboard function, but it doesn't seem to work, at least on Linux.

– Mechanical snail – 2013-05-21T01:46:50.967

Does it possible to convert string to Plot[\[Alpha], {\[Alpha], 0, 10}] or Cell[BoxData["\<\"Plot[\\\\[Alpha], {\\\\[Alpha], 0, 10}]\"\>"], "Output"}] to Unicode Form? Because I use one function something like InputCell2SE@cellExpression//CopyToClipboard copy InputCell to SE which turns out Plot[\[Alpha], {\[Alpha], 0, 10}], and I used FrontEnd..."InputText" – HyperGroups – 2013-06-19T15:08:37.970

@HyperGroups I added a variation of the function that holds the expression. – Mr.Wizard – 2013-06-19T15:30:10.513

A native method may be forthcoming: see this meta discussion.

– shrx – 2013-06-19T17:05:57.250

@Mr.Wizard Maybe Leonid will come up with an OS independent solution, but if not, we may want to combine our approaches. I can probably simplify my approach to use pbcopy (similar to clip, I think). But I would say that a more modular design would be to let copyUnicode only accept String in the first place, and leave it to the calling function to provide the string. I.e., I wouldn't need to use HoldFirst on copyUnicode itself then. Anyway, it depends on how it's going to be used. Should I try to write a combined version of our Mac/PC solutions? – Jens – 2013-06-19T19:36:15.410

Hi, I post one question to describe my usage, need your attention. http://mathematica.stackexchange.com/questions/27289/convert-alpha-to-unicode-form

– HyperGroups – 2013-06-20T01:46:27.777

21

Declaration:

This method for Windows is based on the .NET code from Todd Gayley's this wonderful answer. My .NET knowledge is absolutely ZERO, all credit goes to Todd.

Code:

The main idea is to extract the "Input"-style code string, convert it to the UTF-16 little endian form, which is the standard byte order in Windows, feed the bytes to system clipboard by using .NET API.

The main functions are placed in a package:

Needs["NETLink`"]
InstallNET[];
BeginPackage["UniCodeCopy`"]

mmaUnicodeToUTF16LE::usage = 
        ToString[Row[{Style["mmaUnicodeToUTF16LE[_Integer]", Bold], " convert integer list obtained from ", Style[Row[{"ToCharacterCode[", Style["string", Italic], ", \"Unicode\"]"}], Bold], " to integer list consistent with the UTF‐16 Little Endian standard."}], StandardForm];
stringToUTF16LE::usage = 
        ToString[Row[{Style["stringToUTF16LE[_String]", Bold], " convert ", Style["string", Italic], " to integer list consistent with the UTF‐16 Little Endian standard."}], StandardForm];
WriteToClipboardUnicode::usage = 
        ToString[Row[{Style["WriteToClipboardUnicode[_String]", Bold], " write ", Style["string", Italic], " to Windows clipboard using the UTF‐16 Little Endian encoding."}], StandardForm];

Begin["`Private`"]

Clear[mmaUnicodeToUTF16LE]
mmaUnicodeToUTF16LE[bytecode_Integer] := 
                    PadRight[#, 2 Ceiling[Length@#/2]] &@
                     Join[Most[#] - Rest[# 2^8], #[[{-1}]]] &@
        DeleteCases[FixedPointList[BitShiftRight[#, 8] &, bytecode], 0]

Clear[stringToUTF16LE]
stringToUTF16LE[str_String] := 
                    mmaUnicodeToUTF16LE /@ ToCharacterCode[str, "Unicode"] // 
            Flatten // Join[#, {0, 0}] &

Clear[WriteToClipboardUnicode]
WriteToClipboardUnicode[str_String] :=
       Module[{bytecode, strm, dataObject},
          bytecode = stringToUTF16LE[str];
          NETLink`NETBlock[
                           strm = NETLink`NETNew["System.IO.MemoryStream", bytecode];
                           dataObject = NETLink`NETNew["System.Windows.Forms.DataObject"];
                           dataObject@NETLink`SetData["Text", strm];
                           NETLink`LoadNETType["System.Windows.Forms.Clipboard"];
                           System`Windows`Forms`Clipboard`SetDataObject[dataObject]
                          ]
              ]

End[]
EndPackage[]

Then execute the following code, which will generate a palette with a UniCode Copy button:

Button["UniCode Copy",
    Module[{codestr},
        timestamp = {};
        FrontEndExecute[FrontEndToken["CopySpecial", "InputText"]];
        codestr = 
            NotebookGet[ClipboardNotebook[]][[1, 1, 1]] // 
                StringReplace[#, {"\\\n" -> "", 
                            "\n" ~~ space : " " ... :> 
                                StringJoin["\n", ConstantArray[space, 4]]}] &;
        UniCodeCopy`WriteToClipboardUnicode[codestr]
        ],
    Method -> "Queued"
    ]

SelectionMove[EvaluationNotebook[], Previous, Cell];
FrontEndExecute[FrontEndToken["GeneratePalette"]];
FrontEndExecute[FrontEndToken["Clear"]];

To use it, select the Cells you want to copy as input text code, then press the button. The package can be put in a file and be auto-loaded when MMA start, the palette can be installed to system menu.

Examples:

Unicode expressions in Mathematica Notebook:

test unicode expressions

Text generated directly by the above code:

"αβγδϵζηθχϕϡΔΥϖϒϠíãéõÂÆŁØÝ∟△♆♇√÷∇■○▫✶✓¥¶†︵│︷中文汉字日文コンピュータ"

α β + Mod[δΨ, 2 ρ^2]

Silvia

Posted 2012-02-01T22:11:44.943

Reputation: 25 336

Very nice. Thanks! – Mr.Wizard – 2014-04-01T22:10:04.493

@Mr.Wizard It's my honour :) Needed this tool myself for a loong time. And there are still problems, such as some characters from private area get copied, e.g. a. Maybe a whitelist is needed. – Silvia – 2014-04-01T22:16:46.977

Arguably copy of characters such as \[Transpose] as  is acceptable as they can still be pasted into a Notebook, although it would be nice to also have the option to copy the expanded FullForm of such private letters. – Mr.Wizard – 2014-04-02T02:23:23.720

@Mr.Wizard One possible way came into my mind: Read the Boxes, identify the private characters (especially those used as functions like \[Transpose]), convert the corresponding parts into InputForm(how?), then make the copy. – Silvia – 2014-04-02T20:41:07.430

@Mr.Wizard Wow thank you for your bounty! :D – Silvia – 2014-04-07T23:59:35.303

I think you earned it. Apparently the foundation was presented by Todd Gayley but I never built upon it. A button like this is an actual solution, unlike having to wrap code blocks in copyUnicode (with or without a Hold attribute) as I have been doing. If it were platform and version independent I would have Accepted it, but I did not so as to leave room for future additions. – Mr.Wizard – 2014-04-08T01:03:00.533

Thanks. I believe a similar method can be realized with JLink, that will be platform independent. But again, I have zero knowledge on Java.. And why not build a button with your copyUnicode? – Silvia – 2014-04-08T01:08:42.310

I did that and it sorta worked, but I had trouble making it robust. So far this method hasn't failed. Anyway I like this method because it brings an entirely different approach that doesn't rely on exporting to a file, etc. Thanks again for sharing it. – Mr.Wizard – 2014-04-08T06:06:36.350

@Mr.Wizard You're welcome! I'm using it daily now :) I updated the code into a ready-to-use package, hope it makes thing easier. – Silvia – 2014-04-08T07:05:19.610

Why so cool you! – LCFactorization – 2016-02-02T06:13:53.540

I found \[Rule] cannot be converted into unicode. Does that mean there is no such a corresponding symbol there? – LCFactorization – 2016-02-14T09:11:25.117

1@LCFactorization I think the \[Rule] symbol in MMA, whose Unicode is U+F522, is in the Basic Multilingual Plane (U+E000–U+F8FF), one of the Private Use Area in Unicode standard, thus may not be printable in 3rd party software. – Silvia – 2016-02-14T16:03:48.803

thank you! It seems for this symbol, -> is a better way to represent. @xzczd suggests an online versiion converter, which is also very useful and deserves a try: http://steampiano.net/msc/ BTW, are you interested in solving tough nonlinear PDE system problems by NDSolve or similar solvers?

– LCFactorization – 2016-02-15T10:03:32.727

1@LCFactorization xzczd's web service looks very nice, thanks for sharing! Regarding the PDE, if you have any examples, please feel free to ping me in chat or email me :) – Silvia – 2016-02-18T14:20:44.387

15

Edit 2: A new version of the Mac solution with button is listed below

Fixed problem with pasting into textarea

In some applications on Mac, copying as Unicode from Mathematica already works without having to do any postprocessing. However, it doesn't work in textarea fields in web browsers.

Nevertheless, if you're willing to do a few additional mouse clicks, the Unicode forms can be brought into the browser by taking the detour through one of those applications that do support Mathematica's characters. One convenient choice is to open a new window in TextEdit and paste your original Mathematica code into it first. In the default RichText mode, TextEdit displays the special characters as Unicode glyphs. Then you can simply copy whatever you just pasted back from TextEdit and paste it into the browser. It seems that as soon as TextEdit is recognized as the provider of the Unicode text on the pasteboard, there is no conversion back to the Mathematica representation, so you get the correct appearance in the browser.

In the following, I use the same idea in order to automate the conversion. Instead of TextEdit, I go through a temporary file in RichText (RTF) format on the provider side. The Mac pasteboard does support RTF as a format for data on the pasteboard, but this isn't accepted by the browsers I've recently tried when attempting to paste into textarea.

Therefore, I have to add one additional step: using the textutil tool which is built into OS X, I convert the RTF file to a regular text file with Unicode characters first. Then I read the text file back in and put it on the pasteboard.

Of course, this means it's only going to work on Mac OS X because it uses Cocoa bindings in the built-in Python interpreter:

copyAsUnicode[t_] := 
 Module[{out = 
    FileNameJoin[{$TemporaryDirectory, 
      "MathematicaOutput" <> StringJoin[Map[ToString, DateList[]]]}]},
   Export[out <> ".rtf", t];
  Run["textutil -convert txt " <> out <> ".rtf -output " <> out <> 
    ".txt"]; 
  Run["printf \"from AppKit import *\n\
board=NSPasteboard.generalPasteboard()\n\
content=NSData.dataWithContentsOfFile_('" <> out <> 
    ".txt')\nboard.declareTypes_owner_([NSStringPboardType], None)\n\
board.setData_forType_(content, NSStringPboardType)\n\
\" | /usr/bin/python"];
  DeleteFile[{out <> ".txt", out <> ".rtf"}]]

The idea is to export to RTF and read the result to the clipboard outside of Mathematica. The function is invoked for example as copyAsUnicode["αβ+Mod[δΨ+ρ2]"]. This example itself was copied that way, too, i.e., I typed copyAsUnicode["copyAsUnicode[\"αβ+Mod[δΨ+ρ2]\"]"], which I again copied the same way... OK, I think you get the idea.

Of course the next step would be to make this into a Palette that acts on the NotebookSelection, but the above is the main step. Maybe someone else knows how to do something like this in other operating systems (I don't).

Installing this function as a button:

Responding to the comment, here I'm just listing the same function as above, but wrapped in Silvia's code to make it into a button. It requires no package loading because I inlined everything into the button code:

Button["UniCode Copy", Module[{codestr},
FrontEndExecute[FrontEndToken["CopySpecial", "InputText"]];
codestr = NotebookGet[ClipboardNotebook[]][[1, 1, 1]];
 Module[{out = 
    FileNameJoin[{$TemporaryDirectory, 
      "MathematicaOutput" <> StringJoin[Map[ToString, DateList[]]]}]},
   Export[out <> ".rtf", codestr];
  Run["textutil -convert txt " <> out <> ".rtf -output " <> out <> 
    ".txt"]; 
  Run["printf \"from AppKit import *\n\
board=NSPasteboard.generalPasteboard()\n\
content=NSData.dataWithContentsOfFile_('" <> out <> 
    ".txt')\nboard.declareTypes_owner_([NSStringPboardType], None)\n\
board.setData_forType_(content, NSStringPboardType)\n\
\" | /usr/bin/python"];
  DeleteFile[{out <> ".txt", out <> ".rtf"}]]],
Method -> "Queued"]
SelectionMove[EvaluationNotebook[], Previous, Cell];
FrontEndExecute[FrontEndToken["GeneratePalette"]];
FrontEndExecute[FrontEndToken["Clear"]];

Now you can also install this permanently by going to Palettes > Install Palette and selecting a name for the palette.

Jens

Posted 2012-02-01T22:11:44.943

Reputation: 93 191

@Jens, this is a nice workaround (and it still works). Any idea if there are plans for this to be integrated into a Palette as you suggested? – Rico Picone – 2014-05-13T18:30:17.857

@RicoPicone Thanks, I added the button code, hopefully that should put it on equal footing with the windows solution (except for potential shortcomings of the Mac Unicode handling, which I have no control over...). – Jens – 2014-05-13T19:33:48.567

@Jens, thanks for your work on this. I can't get it to work. I get the message (copy from clipboard option) that the clipboard contents are invalid. I'll try to save it as a file, next. – Rico Picone – 2014-05-14T14:32:51.007

Is it working if you use the copyAsUnicode function by itself? I'll get back to this later, but any additional info would help. Is the error message generated when you try to paste into some external application (which one)? I'm trying to understand if it's the button code or the original function that isn't working (it works for me). – Jens – 2014-05-14T15:17:35.817

On my Mac this answer works fine (including the copy button) in Mathematica versions 8 and 10. – Jens – 2014-09-20T22:13:28.667

For me both the button and the function work, on MMA 10, OSX 10.9.5. One thing is that an additional space seems to be added at the front of the string, which is only a minor inconvenience. (+1) – Jacob Akkerboom – 2014-09-30T13:12:58.063

This looks good, but since I cannot test it on Windows I will leave it to Mac users to upvote. Thank you for your answer. – Mr.Wizard – 2012-02-02T04:27:12.887

Hi, I posted one question with little difference from directly copy Input Expression to SE and need your attention. mathematica.stackexchange.com/questions/27289/… – HyperGroups – 2013-06-20T01:47:41.390

14

Here's how to add a permanent menu item and keyboard shortcut for Silvia's solution. (This can be easily adapted to Jens' solution for Mac OS.)

enter image description here

1. Add Package

Copy Silvia's first code block that starts with Needs["NETLink`"] to a file and save it as UniCodeCopy.m in one of the directories included in $Path.

2. Initialize the Front End

Save the following to the file $UserBaseDirectory/Autoload/FrontEnd/init.m (create it if needed):

Needs["UniCodeCopy`"]

CopySelectionAsUnicode[] :=
    Module[{selection, code},
           FrontEndExecute[FrontEndToken["CopySpecial", "InputText"]];
           selection = NotebookGet[ClipboardNotebook[]][[1, 1, 1]];
           code = StringReplace[selection, {"\\\n" -> "",
                                            "\n" ~~ space : " " ... :> StringJoin["\n", ConstantArray[space, 4]]}];
           UniCodeCopy`WriteToClipboardUnicode[code];]

FrontEndExecute[
    FrontEnd`AddMenuCommands["Copy",
                             {MenuItem["Copy as Unicode",
                                       FrontEndExecute[CopySelectionAsUnicode[]],
                                       (* Alt + U *)
                                       MenuKey["u", Modifiers -> {"Command"}],
                                       System`MenuEvaluator -> Automatic,
                                       Method -> "Queued"]}]]

For key bindings, the possible modifiers are "Shift", "Control", "Command" (Alt) and "Option" (Alt).

(It took me a few hours to figure this out. I hope this proves useful for other struggling novices.)

ens

Posted 2012-02-01T22:11:44.943

Reputation: 581

2Very nice! Thanks for sharing :) – Silvia – 2014-08-10T06:46:56.413

1Are you saying that $UserBaseDirectory/Autoload/FrontEnd/init.m should be created, because there is no such file on my Windows 8.1 computer? There is, however, the file $UserBaseDirectory/FrontEnd/init.m on my computer, but appending your code to the end of that file caused Mathematica 10.0.2.0 to crash during initialization. Thanks in advance for your assistance. – bbgodfrey – 2015-03-08T22:40:53.647

Yes, just create it. I've added that to my post. – ens – 2015-03-08T22:53:07.007

It works well. Thank you. – tanghe2014 – 2020-06-01T01:08:00.207

12

I use a small web application for when there are too many to convert by hand.

cormullion

Posted 2012-02-01T22:11:44.943

Reputation: 23 565

@Jens Hmm.. did not work for me - MacBook Pro OSX - all standard things. Copied from notebook - pasted into TextEdit - had all the \[Alpha] etc. Some special settings? – Vitaliy Kaurov – 2014-04-03T04:48:20.253

@VitaliyKaurov Works for me. Did you set the document format to Plain Text perhaps? Then it won't work of course. – Jens – 2014-04-03T05:37:33.793

Thanks @Jens - no, format is Rich Text. No worries - I'll dig a bit at it. Thanks! – Vitaliy Kaurov – 2014-04-03T06:42:10.313

Very handy. Now if we could just get this as a button in the editor.

– Mr.Wizard – 2013-06-19T16:09:42.467

On Mac, I can get the same result by simply pasting the Mathematica code into the TextEdit application and immediately copying the same passage from there. So there's no need to go to the web. – Jens – 2013-06-19T17:15:38.087

@Jens very true. One day everyone will use Macs... :) (typed on a iPad) – cormullion – 2013-06-19T17:27:09.660

11

Here's a version that doesn't require a temporary file.

Linux (needs xclip)

SetAttributes[copyUnicode, HoldAll];
copyUnicode[expr_] := With[{
      stream = OpenWrite["!xclip -in -selection clipboard", CharacterEncoding -> "UTF-8"]
   },
   WriteString[stream, ToString[Unevaluated@expr, InputForm]];
   Close@stream;
];

Example: executing the cell

Cell[BoxData[
 RowBox[{"copyUnicode", "[", 
  RowBox[{
   RowBox[{
    RowBox[{
     SuperscriptBox["x", "2"], "\[SmallCircle]", 
     RowBox[{"{", 
      RowBox[{
       RowBox[{"\[LeftFloor]", "\[Alpha]", "\[RightFloor]"}], ",", 
       "\"\<\[LeftFloor]\[Alpha]\[RightFloor]\>\""}], "}"}]}], 
    "\[PlusMinus]", 
    RowBox[{
     SqrtBox["5"], "\[CirclePlus]", "\[HappySmiley]"}]}], 
   "<=", 
   "\"\<\[Integral]\[PartialD]\[RightArrow]\[Union]\[Sum]\[Infinity]\
\[Element]\>\""}], "]"}]], "Input"]

gives x^2 ∘ {Floor[α], "⌊α⌋"} ± Sqrt[5] ⊕ ☺ <= "∫∂→⋃∑∞∈".

Windows

Not tested, but it should work if you use "!clip"instead of"!xclip -in -selection clipboard"`. You might have to change the encoding to UTF-16.

Caveats

  • Note that certain characters get ASCIIfied anyway in InputForm when not inside a string.
  • Mathematica uses non-standard private-use code points for some characters like U+211D , even when a standard code point exists, so the output will be wrong if the input contains such characters.

Mechanical snail

Posted 2012-02-01T22:11:44.943

Reputation: 2 302

Your caveats also apply to Mac OS X, so there's more work to be done before it becomes 100% reliable... (+1) – Jens – 2013-06-19T20:21:50.950

Another caveat to add: some special characters, such as \[UpEquilibrium], get converted incorrectly or not at all, no matter what I try. So I don't think it's possible to get a perfect translation to Unicode in general. – Jens – 2013-06-19T20:46:37.317

11

Here is a function that copies a Unicode string to the clipboard using JLink:

Needs["JLink`"];
InstallJava[];
LoadJavaClass["java.awt.Toolkit", AllowShortContext -> False];

uniclip[s_String] :=
  JavaBlock[
    java`awt`Toolkit`getDefaultToolkit[]@getSystemClipboard[]@setContents[#, #]& @
      JavaNew["java.awt.datatransfer.StringSelection", s]
  ];

Example usage:

expr = \[Alpha]\[Beta] + Mod[\[Delta]\[CapitalPsi], 2\[InvisibleTimes]\[Rho]^2];

uniclip @ ToString[expr, InputForm]

After evaluating these expressions, the clipboard contains αβ + Mod[δΨ, 2*ρ^2].


Update

In the comments, a request was made to include code to invoke this functionality from a context menu. I was unable to fulfill this request. If someone wants to take up the work, here is a packaged form of the function which will save them the trouble of dealing with the finicky auto-load requirements of anything that uses JLink`:

BeginPackage["Uniclip`", {"JLink`"}];

CopyUnicodeToClipboard::usage =
  "CopyUnicodeToClipboard[s] copies s to the clipboard using Unicode characters";

CopySelectionAsUnicode::usage =
  "CopySelectionAsUnicode[] copies the currently selected text to the clipboard using Unicode characters";

Begin["`Private`"];

retry:_CopyUnicodeToClipboard :=
  ( InstallJava[]
  ; LoadJavaClass["java.awt.Toolkit", AllowShortContext -> False]
  ; CopyUnicodeToClipboard[s_String] :=
      JavaBlock[
        java`awt`Toolkit`getDefaultToolkit[] @
           getSystemClipboard[]@
           setContents[#, #]& @
           JavaNew["java.awt.datatransfer.StringSelection", s]
      ]
  ; retry
  )

CopySelectionAsUnicode[] :=
  Module[{text}
  , FrontEndExecute[FrontEndToken["CopySpecial", "InputText"]]
  ; text = NotebookGet[ClipboardNotebook[]][[1, 1, 1]]
  ; CopyUnicodeToClipboard @ text
  ]

End[];

EndPackage[];

WARNING The function CopySelectionAsUnicode functions erratically when called from a menu item. It involves a re-entrant call to the front-end which introduces a lengthy delay.

The auto-load declaration to go into the kernel's init.m looks like this:

DeclarePackage["Uniclip`", {"CopyUnicodeToClipboard", "CopySelectionAsUnicode"}];

The package code must go into a file named Uniclip.m somewhere in the $Path.

The context menu declaration (for ContextMenus.tr) looks like this:

MenuItem["&Unicode",
    KernelExecute[ToExpression["Uniclip`CopySelectionAsUnicode[]"]],
    MenuEvaluator -> Automatic],

I had added it under all of the Copy As menu items for the Text, Input, Output, Plugin, CellBracket styles.

WReach

Posted 2012-02-01T22:11:44.943

Reputation: 62 787

1This is great! I suppose it should work on all platforms? – Silvia – 2014-09-07T20:43:28.163

Great - I see no problems on Mac OS X 10.7.5 with Mathematica 10. – Jens – 2014-09-07T21:33:35.200

@Silvia I can only vouch for it working on Windows, but by rights it should work on all platforms. – WReach – 2014-09-07T22:20:53.507

3I believe this is the first multiplatform solution. Provide complete self-contained package code to add this as a menu item and shortcut key (in the context menu if you can manage it), and I'll Accept it so long as no failures appear. Thanks! – Mr.Wizard – 2014-09-08T06:55:05.327

@WReach Are you disinterested in this or do you simply have other priorities? – Mr.Wizard – 2014-09-20T11:01:54.120

2@Mr.Wizard I spent a couple of hours today trying to fulfill the request, but failed. I kept hitting race conditions in re-entrant front-end calls while trying to hook up the context menu. This is because the only way I know how recover the text selection involves a front-end call, and the menu item invocation is already a front-end call. I can't really spend more time on it right now, so if anyone wants to incorporate uniclip into another answer that adds menu items and shortcut keys, feel free. – WReach – 2014-09-20T18:33:38.117

Thanks @WReach ! I appreciate the effort. :-) – Mr.Wizard – 2014-09-20T20:51:35.540

4

Here's a method that's really no different from the standard ones except it's generalized for copying cells and, theoretically, for Windows and Unix.

We'll use RunProcess because this exists now, and tweak the ProcessEnvironment to make it work for Mac:

FrontEnd`unicodeCopy[s_String,
   enc : _String | Automatic : Automatic,
   lang : _String | Automatic : Automatic
   ] :=
  With[{
    encoding = Replace[enc, Automatic :> $CharacterEncoding],
    language = Replace[lang, Automatic :> $Language]
    },
   If[#["ExitCode"] =!= 0,
      FrontEnd`unicodeCopy::copyerr = "error in copy process:\n ``";
      Message[FrontEnd`unicodeCopy::copyerr, #["StandardError"]];,
      #["StandardOuput"];
      ] &@
    RunProcess[
     Switch[$OperatingSystem,
      "Windows",
      "clip",
      "MacOSX",
      "pbcopy",
      "Unix",
      {"xsel", "-b"}
      ],
     All,
     ExportString[s,
      "String",
      CharacterEncoding -> encoding
      ],
     ProcessEnvironment -> <|
       "LANG" ->
        Replace[ToLowerCase@language,
          "english" -> "en_US"
          ] <> "." <>
         Replace[encoding, {
           "UTF8" -> "UTF-8",
           "Unicode" -> "UTF-16"
           }],
       "PATH" -> Environment["PATH"]
       |>
     ]
   ];

(I'm using FrontEnd` for when I dump to "MenuSetup.tr", just so it looks consistent)

Now we'll define a bunch of junk to generalize to make it work across an NB:

$unicodeTR =
  FrontEnd`FindFileOnPath["UnicodeCharacters.tr", 
    "PrivatePathsTextResources"] // FrontEndExecute;
$unicodeReplacements = #[[2]] -> 
     ToExpression[
      "\"" <> StringReplace[#[[1]], {"x" -> ":", 
         StartOfString ~~ "0" -> "\\"}] <> "\""] & /@
   Rest[
    StringSplit@StringSplit[Import[$unicodeTR, "String"], "\n"]
    ];

FrontEnd`UnicodeCopy[c : {__Cell}] :=
  Replace[
   MathLink`CallFrontEnd[
    ExportPacket[Cell[CellGroupData[c]], "InputText"]
    ], {
    {s_String, __} :>
     FrontEnd`unicodeCopy@
      StringReplace[s,$unicodeReplacements]
    }];
FrontEnd`UnicodeCopy[c_Cell] :=
  Replace[
   MathLink`CallFrontEnd[
    ExportPacket[c, "InputText"]
    ], {
    {s_String, __} :>
     FrontEnd`unicodeCopy@
      StringReplace[s,$unicodeReplacements]
    }];
FrontEnd`UnicodeCopy[b_BoxData | _TextData] :=

  FrontEnd`UnicodeCopy@Cell[b];
FrontEnd`UnicodeCopy[b_RowBox] :=

  FrontEnd`UnicodeCopy@Cell[BoxData@b];
FrontEnd`UnicodeCopy[
   nb : _NotebookObject | _FrontEnd`NotebookObject | \
_FrontEnd`SelectedNotebook | _FrontEnd`InputNotebook | \
_FrontEnd`ButtonNotebook | _FrontEnd`EvaluationNotebook | \
_FrontEnd`MessagesNotebook | _FrontEnd`HelpBrowserNotebook | \
_FrontEnd`ClipboardNotebook
   ] :=
  FrontEnd`UnicodeCopy[Evaluate@NotebookRead@nb];
FrontEnd`UnicodeCopy[
   nb : _InputNotebook | _EvaluationNotebook | _ClipboardNotebook | \
_ButtonNotebook | _MessagesNotebook | _HelpBrowserNotebook] :=

  FrontEnd`UnicodeCopy@Evaluate@nb;
FrontEnd`UnicodeCopy[Optional[Automatic, Automatic]] :=

  FrontEnd`UnicodeCopy@InputNotebook[];
FrontEnd`UnicodeCopy[s_String] :=
  FrontEnd`unicodeCopy[s];
FrontEnd`UnicodeCopy[e_] :=

  FrontEnd`UnicodeCopy[Evaluate@ToString[Unevaluated[e], InputForm]];
FrontEnd`UnicodeCopy~SetAttributes~HoldFirst

Then I'll add this to my MenuSetup using the framework I wrote up here and whose up-to-date implementation lives here

FEMenuSetupAdd[
 {"Edit", "Copy As", 5},
 "Unicode" -> KernelExecute[ToExpression["FrontEnd`UnicodeCopy[]"]],
 MenuEvaluator -> Automatic,
 System`MenuKey["C", System`Modifiers -> {"Control"}]
 ]

This now lives under the "Copy As" menu:

copy as

And it can be used by function:

FrontEnd`UnicodeCopy["asdasd\[Alpha]\[Beta]\[Gamma]"]

asdasdαβγ

FrontEnd`UnicodeCopy[Cell["asdasd\[Alpha]\[Beta]\[Gamma]"]]

asdasdαβγ

Or by Control-C:

FrontEnd`UnicodeCopy[Cell["asdasdαβγ"]]

(fixed, now, by doing a more selective replacement from UnicodeCharacters.tr)

One issue here is that I was a bit over zealous with some replacements when copying from cells and boxes (has to do with what comes out of the ExportPacket):

(Command-C): "asdasd\\[Alpha]\\[Beta]\\[Gamma]" 
(Control-C): "asdasd\α\β\γ" 

And just to include the original example:

\[Alpha]\[Beta] + 
 Mod[\[Delta]\[CapitalPsi], 2\[InvisibleTimes]\[Rho]^2]

becomes:

αβ + 
 Mod[δΨ, 2⁢ρ^2]

(note how, unfortunately, the added spacing from the cell copy is preserved)

b3m2a1

Posted 2012-02-01T22:11:44.943

Reputation: 42 610

That looks like a lot of work! Thank you. The spacing thing may even be useful. :-) – Mr.Wizard – 2017-06-17T14:08:14.140

3

Edit: Updated code

As a generalization to the excellent answer by ens, Silvia's solution also can be added as a palette to the menu as follows. First, create and save the UniCodeCopy.m package, as described by ens. Then create as a separate notebook, perhaps named Unicode Copy Source.nb,

NotebookPut[
    Notebook[{Cell[BoxData[ButtonBox["\"UniCode Copy\"", Appearance -> Automatic,
        ButtonFunction :> Module[{codestr},                  
        AppendTo[$Path, FileNameJoin[{$UserDocumentsDirectory, 
            "Mathematica/MyPackages"}]]; 
        Needs["UniCodeCopy`"];                  
        FrontEndExecute[FrontEndToken["CopySpecial", "InputText"]]; 
        codestr = (StringReplace[#1, {"\\\n" -> "", "\n" ~~ space : " " ... :> 
            StringJoin["\n", ConstantArray[space, 4]]}] & )
            [NotebookGet[ClipboardNotebook[]][[1, 1, 1]]]; 
            UniCodeCopy`WriteToClipboardUnicode[codestr]], 
            Evaluator -> Automatic, ImageSize -> 200, Method -> "Queued"]], 
            NotebookDefault]}, WindowSize -> {Fit, Fit}, 
            WindowMargins -> {{Automatic, 522}, {Automatic, 41}}, 
            WindowFrame -> "Palette", WindowElements -> {}, 
            StyleDefinitions -> "Palette.nb"], WindowTitle -> "UniCodeCopy"];
NotebookSave[%, FileNameJoin[{$UserDocumentsDirectory, 
    "Mathematica/MyPackages/UniCodeCopy.nb"}]]

Note that the third and fourth lines of code add the location of UniCodeCopy.m (in this case, my Mathematica/MyPackages directory) to $Path, if the location is not already there. (If it is there, these two lines of code can be omitted.) Executing Unicode Copy Source.nb creates the small palette, UnicodeCopy.nb, saved in Mathematica/MyPackages. Finally, use the Install Palette ... command in the Palette menu to register UnicodeCopy.nb as a palette. After Mathematica is restarted, it will display UnicodeCopy in the Palette menu.

bbgodfrey

Posted 2012-02-01T22:11:44.943

Reputation: 55 236

2

Under windows with Nircmd available, one can add the code below (a modification of https://mathematica.meta.stackexchange.com/a/155/9754) to an initialization cell of
$UserBaseDirectory <> "\\Autoload\\FrontEnd\\init.m" to get Alt Gr+p as a shortcut for copying as unicode and having 4 spaces added at the beginning of each line.

FrontEndExecute[FrontEnd`AddMenuCommands["Copy", {MenuItem["Copy for MSE",
      FrontEndExecute[Module[{opts, content},
            If[MatchQ[NotebookRead[SelectedNotebook[]], {} | $Failed | NotebookRead[$Failed]], Beep[],

                opts = AbsoluteOptions[$FrontEndSession, "ExportTypesetOptions"];
                SetOptions[$FrontEndSession, "ExportTypesetOptions" -> {"PageWidth" -> 800, "CellBreaks" -> "\nCeLlBrEaK\n"}];
                FrontEndTokenExecute[SelectedNotebook[], "CopySpecial", "InputText"];
                content = NotebookGet[ClipboardNotebook[]];
                SetOptions[$FrontEndSession, opts];

                RunProcess[{"C:\\nircmd\\nircmdc.exe", "clipboard", "readfile", "\"" <> 
                          Export["C:\\nircmd\\mathCopy.txt", #, CharacterEncoding -> "Unicode"] <> "\""}] &[

                   Fold[StringReplace, StringJoin[{"    ", Cases[content, Cell[c_String, ___] :> c, All]}],
                      {{"" | "" -> "I", "" -> "E", "" | "" -> "<->", "\n" -> "\n    "},
                        {"    In[" ~~ NumberString ~~ "]:= " -> "    ",
                          "    CeLlBrEaK\n" | StartOfString ~~ "    Out[" ~~ NumberString ~~
                              "]= " ~~ e : Shortest[__] ~~ f : "\n    CeLlBrEaK" | EndOfString :>
                            "    (* " <> StringReplace[e, "\n    " -> "\n       "] <> " *)" <> StringTake[f, UpTo[5]],
                          "CeLlBrEaK" -> ""}}]]]]],
       MenuKey["p", Modifiers -> {"Control", "Command"}], System`MenuEvaluator -> Automatic, Method -> "Queued"]}]]

MeMyselfI

Posted 2012-02-01T22:11:44.943

Reputation: 1 064