perlglobnode-redbsd

Confounding differing behavior of the perl module `File::Glob`'s subroutine `bsd_glob` in Node-RED on Sonoma

I updated my mac mini (that is a part of my entertainment center) to Sonoma the other day. It was long overdue. I think it was still on High Sierra before. I have a Node-RED instance on that computer (also lagging behind on updates at version v1.2.7, but I suspect that this is unrelated to the version - and it would be nice if someone with the latest version could run my MRE and confirm it's an issue in the latest NR...).

My script that identifies the correct sqlite file of the Reminders app database (as well as other scripts that do similar things) no longer works correctly after the update. I have boiled down the problem with differing behaviors of the bsd_glob subroutine that is a part of the File::Glob ':glob' perl module.

The thing is, it works as expected on the command line (even when called from other scripts (like Apple Scripts and SLURM scripts I call it from at work)). But after the update to Sonoma, it no longer behaves the same from the Node-RED exec node, which returns no file matches.

Here is an MRE example, run from...

The command line returns 3 files:

[golrath:~] robleach% perl -e 'use File::Glob ":glob";print(bsd_glob("/Users/robleach/Library/Group Containers/group.com.apple.reminders/Container_v1/Stores/Data-*-*.sqlite",GLOB_CSH),"\n")'
/Users/robleach/Library/Group Containers/group.com.apple.reminders/Container_v1/Stores/Data-A0D24954-1339-494B-9E7D-9D0911C096F8.sqlite/Users/robleach/Library/Group Containers/group.com.apple.reminders/Container_v1/Stores/Data-B826F8F1-0153-4318-8666-F1C8EBBFCE96.sqlite/Users/robleach/Library/Group Containers/group.com.apple.reminders/Container_v1/Stores/Data-D8328338-F382-4E34-8C39-E1DC35D60A32.sqlite

And via a Node-RED exec node, it returns 0 files:

exec config

exec result

My first thought was that the perl executable, module, or even csh versions must have differed, but I confirmed they're all the same between the command line and Node-RED:

The command line:

[golrath:~] robleach% perl -e 'use File::Glob ":glob";print("$File::Glob::VERSION\n")'
1.37
[golrath:~] robleach% csh --version
tcsh 6.21.00 (Astron) 2019-05-08 (x86_64-apple-darwin) options wide,nls,dl,bye,al,kan,sm,rh,color,filec
[golrath:~] robleach% tcsh --version
tcsh 6.21.00 (Astron) 2019-05-08 (x86_64-apple-darwin) options wide,nls,dl,bye,al,kan,sm,rh,color,filec
[golrath:~] robleach% perl --version
[golrath:~] robleach% which perl
/Users/robleach/perl5/perlbrew/perls/perl-5.36.1/bin/perl
[golrath:~] robleach% ls -halF /usr/local/bin/perl
lrwxr-xr-x  1 robleach  admin    57B Jul 22 09:13 /usr/local/bin/perl@ -> /Users/robleach/perl5/perlbrew/perls/perl-5.36.1/bin/perl

The only difference is the perl soft-link in the path. Could that be what's causing the difference in behavior? I can't imagine how it could...

enter image description here

enter image description here

enter image description here

Solution

  • @ikegami (and user Corion on perlmonks) identified the problem that governed the difference in behavior. I'm not sure how they figured that out (perhaps a hunch?), but I confirmed the theory by simply doing an ls in Node-RED, which resulted in an Operation not permitted error (that I wasn't getting from the command line [probably because the Terminal app was granted "Full Disk Access" =in the Privacy and Security settings]).

    The reason the problem eluded me is because the call to File::Glob::bsd_glob silently fails, i.e. no Operation not permitted error is produced.

    The error starts occurring at this point in the path: /Users/robleach/Library/Group Containers/group.com.apple.reminders (and everything underneath it).

    I had also posted about this in the Apple Developer forum and I got a very informative response from a developer there who linked me to an explainer about Apple's custom permissions structure (which you might need to have a [free] apple developer account to view) that sits on top of the typical bsd permissions architecture. To quote that post:

    Modern versions of macOS have four different file system permission mechanisms:

    • Traditional BSD permissions
    • Access control lists (ACLs)
    • App Sandbox
    • Mandatory access control (MAC)

    The first two were introduced a long time ago and rarely trip folks up. The second two are newer, more complex, and specific to macOS, and thus are the source of some confusion. This post is my attempt to clear that up.

    The post goes into some details about those last 2 that are worth a read, but suffice it to say, the solution was to find the parent process and grant disk access to it.

    And of course, the process I granted access to that fixed the issue for me was /usr/local/bin/node. I didn't need to reboot or restart any processed. Once I granted access, the commands run via Node-RED just started working without error.

    I granted full disk access to node, though it's rather heavy-handed, and I intend to change it later today when I get a chance to the specific directories I need. Although I don't know if or how I could restrict it to just Node-RED (probably not). So any/every Node.js script can now access system files, so do this at your own risk.

    And to just complete this answer, here's how to grant full access to Node.js in MacOS Sonoma:

    1. Open the System Settings app (Apple Menu > System Settings)
    2. Click Privacy & Security
    3. Click "Full Disk Access"
    4. Click the "+" button at the bottom left (unlock if necessary)
    5. Type command-shif-g
    6. Enter "/usr/local/bin/node"
    7. Click "usr > local > bin > node"
    8. Click Open

    After having done that, you should immediately be able to access files in Node-RED that were previously either silently failing or resulting in an Operation not permitted error.

    I believe that traditional read/write bsd permissions would still restrict attempts to access sensitive system files, but I have not yet confirmed that.