How do I comminucate between dm-script code and python code, both executed in Digital Micrograph?

---

In my specific case I have complicated dialogs. Sice the dialogs are not supported in python but in dm-script, I wrote my dialogs in dm-script. The problem now is to pass from the dialog to the python code.

Consider the following example:

import DigitalMicrograph as DM

script = """string field_value = "";

class ButtonDialog : UIFrame{
    void field_changed(object self, TagGroup field){
        field_value = field.DLGGetStringValue();
    }
    
    object init(object self){
        TagGroup dlg, dlg_items, field;
        
        dlg = DLGCreateDialog("Type in a number", dlg_items);
        
        dlg.DLGAddElement(DLGCreateLabel("Number:"));
        
        field = DLGCreateIntegerField(0, 10, "field_changed");
        dlg.DLGAddElement(field);
        
        self.super.init(dlg);
        return self;
    }
}

object dialog = alloc(ButtonDialog).init();
dialog.pose();
"""

DM.ExecuteScriptString(script)

# how do I get this?
field_value = ""

To synchronize data between dm-script and python while both are running in the same instance (and thread in this example, can be modified for different threads too) one can use the persistent tags.

The dm-script is setting the persistent tags. python can then read the persistent tags again.

---

Update: Python module

Because of the limitations mentioned below and the increadibly growing and unreadable code I decided to write my own python module for this. With execdmscript one can execute dm-script code with variable synchronization very easily from Digital Micrograph.

Check out the following example:

from execdmscript import exec_dmscript

# some script to execute
dmscript = """
number input;
number s = GetNumber("Enter a number between " + min + " and " + max + ".", init, input);"
"""

# variables that will be defined for the dm-scripts (and readable later on in python)
sv = {"min": 1, "max": 10, "init": 2}
# variables the dm-script defines and that should be readable in the python file
rv = {"input": "number", "s": "number"}

with exec_dmscript(dmscript, readvars=rv, setvars=sv) as script:
    if script["s"]:
        print(script["input"])
    else:
        print("User pressed cancel.")

This hides away all the dm-script related saving and problems (mentioned below). It allows to use list and dicts and basic types like bool, str, int and float. All types and values can be used in the pythonic way with no need to care about dm-script types. The dm-script can be moved to a separete file to clean up the code even more.

Note that for debugging there is a exec_dmscript(debug=True, debug_file="path/to/file") switch that will save the code to the given debug_file. This file can then be executed in GMS manually which shows the errors.

Disclaimer: As mentioned, I am the author of the execdmscript module. Still I think this is the best solution for this case. In fact this question was the reason I wrote the module.

---

Basic datatypes

For basic datatypes like string, and number one can create and add the code by hand. The following example shows the idea:

import DigitalMicrograph as DM
import time

sync_id = "sync-{}".format(int(time.time() * 100))

dmscript = """
number input;
number s = GetNumber("Enter a number between {min} and {max}.", {init}, input);

TagGroup p = GetPersistentTagGroup();
p.TagGroupSetTagAsBoolean("{id}-success", s);
p.TagGroupSetTagAsLong("{id}-input", input);
"""

DM.ExecuteScriptString(dmscript.format(min=1, max=10, init=2, id=sync_id))

# cannot save TagGroups to variables, check below
s, success = DM.GetPersistentTagGroup().GetTagAsBoolean("{}-success".format(sync_id))

if s and success:
    # cannot save TagGroups to variables, check below
    s, input = DM.GetPersistentTagGroup().GetTagAsLong("{}-input".format(sync_id))
    
    if s:
        print(input)
    else:
        print("Did not find 'input' in persistent tags.")
elif not s:
    print("Did not find 'success' in persistent tags.")
else:
    print("User clicked cancel.")

# cannot save TagGroups to variables, check below
# delete tag, otherwise the persistent tags gets filled with garbage
DM.GetPersistentTagGroup().DeleteTagWithLabel("{}-success".format(sync_id))
DM.GetPersistentTagGroup().DeleteTagWithLabel("{}-input".format(sync_id))

As one can see there some downsides of this method. There is a lot of code creating the setup and the synchronization for just entering one number. This makes the code messy. Also one may want to save the dm-script code in a separate file wich adds python file opening. Debugging the dm-script code is difficult in this code too. And lastly there are limitations on TagGroup travelling on python side (check out below).

---

Limitations

Update: The python module mentioned below can deal with TagGrous and TagLists because it saves them linearized and with separate keys and types.

Note that TagGroups are very difficult to synchronize. TagGroups must not be assigned to variables. As soon as they are, they are not usable anymore. This can be illustrated by the following, very easy code:

import DigitalMicrograph as DM

# create group for the example
group1 = DM.NewTagGroup()
group1.SetTagAsString("test-string", "test content")
print("Group 1 valid:", group1.IsValid(), "(type:", type(group1), ")")

# save to persistent tags
DM.GetPersistentTagGroup().SetTagAsTagGroup("test", group1)

# get the group without assigning to a variable, works
s, group2 = DM.GetPersistentTagGroup().GetTagAsTagGroup("test")
print("Group 2 success:", s, ", valid:", group2.IsValid(), "(type:", type(group2), ")")

# assign one parent group to a variable, doesn't work
tags = DM.GetPersistentTagGroup()
s, group3 = tags.GetTagAsTagGroup("test")
print("Group 3 success:", s, ", valid:", group3.IsValid(), "(type:", type(group3), ")")

This code will produce the output

Group 1 valid: True (type: <class 'DigitalMicrograph.Py_TagGroup'> )
Group 2 success: True , valid: True (type: <class 'DigitalMicrograph.Py_TagGroup'> )
Group 3 success: True , valid: False (type: <class 'DigitalMicrograph.Py_TagGroup'> )

This shows that you cannot assign TagGroups to python variables.

---

Asynchronous ideas

This can be extended for asynchronous applications. The dm-script and the python implementation can set threads that observe the persistent tags. (Note that you can mark TagGroups as modified by using TagGroupMarkAsChanged()), perferrably different tags. Then both can add commands and/or data to the tag. The other "instance" can read and process them.

This has to be coded manually. This is (currently) not included in the execdmscript module.