Insert custom button on Insert/Edit Link dialog box tinymce 6.7.0?

This: Insert custom button on Insert/Edit Link dialog? is not working anymore with an error on this.oldOpen.apply the only available methods under editor.windowManager are:

alert()
close()
confirm()
open()
openUrl()

Able to add a custom element inside tinymce dialog box.

Changing an existing dialog opened by the link menu button

You can listen for the editor event ExecCommand and when its name is mceLink (being the command fired when opening the link dialog), you can fetch the dialog iframe using specific selectors and mutate its dom to add a custom button performing your own logic.

This is a full demo adding a B button next to the url text field:

https://fiddle.tiny.cloud/NhtB7PMWnW/2

tinymce.init({
  /*...*/
  setup: function (editor) {        
    /*...*/

    //when the event ExecCommand is fired on the editor
    editor.on('ExecCommand', function(e) {
      //if the command refers to the link dialog opening
      if (e.command === 'mceLink') {                
        //wait for the DOM to update before changing the dialog content
        setTimeout(function() {
          //retrieve the dialog content
          const iframe = document.querySelector('.tox-dialog__body-content');
          //dig down to the first input text field (being the URL)
          const urlTextField = iframe.querySelector('.tox-control-wrap .tox-textfield');                    

          //sets its width to 90% to make room for the extra button next to it
          urlTextField.style.width = '90%';

          //create a new button
          let customButton = document.createElement('button');
          customButton.textContent = 'B';
          customButton.classList.add('tox-button');
          customButton.style.marginLeft = '1px';
          customButton.onclick = function() {
            //this is the click event handler for the new button
            console.log('Custom button was clicked!');
          };
          
          //insert the new button next to the url text field
          urlTextField.parentNode.insertBefore(customButton, urlTextField.nextElementSibling);        
        }, 1);
      }
    });  
    /*...*/
  }
});

Definining your own custom dialog having buttons

When definining a dialog you are going to open with editor.windowManager.open(...) you have the option to use to different types of buttons:

• Custom dialog footer buttons
• Custom dialog buttons

Here I'm showing how to open a dialog having both kind of buttons:

function openMyCustomDialog(editor) {
    editor.windowManager.open({
        title: 'My Custom Dialog',
        body: {
            type: 'panel',
            //items composing the dialog
            items: [
                {
                    type: 'input',
                    name: 'myInput',
                    label: 'My Input'
                },
                //this is a dialog button
                //https://www.tiny.cloud/docs/tinymce/6/dialog-components/#button
                {
                  type: 'button',
                  text: 'CUSTOM DIALOG BUTTON',
                  buttonType: 'primary',
                  name: 'myCustomDialogButton',
                  enabled: true,
                  borderless: false
                }
            ]
        },
        //buttons composing the footer bar
        buttons: [
            //this is a footer button
            //https://www.tiny.cloud/docs/tinymce/6/dialog-footer-buttons/
            {
                type: 'custom',
                name: 'myCustomFooterButton',
                text: 'CUSTOM FOOTER BUTTON',                                
            },
            {
                type: 'cancel',
                text: 'Cancel',
                align: 'end'
            }
        ],

        //event triggering when action is taken on the dialog
        onAction: (dialogApi, details) => {
            //if the action target was myCustomFooterButton         
            if (details.name === 'myCustomFooterButton') {
                const msg = 'custom FOOTER button pressed!';
                console.log(msg);
                dialogApi.setData({ myInput: msg });
            }
            //if the action target was myCustomDialogButton         
            else if(details.name === 'myCustomDialogButton') {
                const msg = 'custom DIALOG button pressed!';
                console.log(msg);
                dialogApi.setData({ myInput: msg });
            }
        }
        
    });
}

The click on any of both buttons will fire a call to the onAction dialog event passing its name in the details argument.

This is a working example on fiddle.tiny.cloud

https://fiddle.tiny.cloud/cPIkEAXqKj/11

The demo shared above also wraps in a layout bar the group containing the input field and the dialog button so that they will lay on a single line:

items: [
  //this is a row container (that will hold an input and a button next to it on the same line)
  //https://www.tiny.cloud/docs/tinymce/6/dialog-components/#bar
  //https://www.tiny.cloud/docs/tinymce/6/dialog-components/#layout-components
  {
    type: 'bar',
    items: [
      {
        type: 'input',
        name: 'myInput',
        label: '',
        maximized: true
      },
      //this is a dialog button
      //https://www.tiny.cloud/docs/tinymce/6/dialog-components/#button
      {
        type: 'button',
        text: 'CUSTOM DIALOG BUTTON',
        icon: 'bold',
        buttonType: 'primary',
        name: 'myCustomDialogButton',
        enabled: true,
        borderless: false,                    
      }
    ]
  }                
]

Here you can read more about layout components:

https://www.tiny.cloud/docs/tinymce/6/dialog-components/#layout-components https://www.tiny.cloud/docs/tinymce/6/dialog-components/#bar

You can find a live example covering buttons, from the official documentaion here:

https://www.tiny.cloud/docs/tinymce/6/dialog-examples/#interactive-example-using-redial