javascriptpythonwebassemblypyodide

Select and read a file from user's filesystem

I need to:

The code to read currently replaced by dummy code: inp_str = 'ACGTACGT'. The actual use case involves complicated processing of the text file, but the minimal working example simply converts the input to lowercase.

<!doctype html>
<html>
  <head>
      <script src="https://cdn.jsdelivr.net/pyodide/v0.22.1/full/pyodide.js"></script>
  </head>
  <body>
    Analyze input <br>
    <script type="text/javascript">
      async function main(){
          let pyodide = await loadPyodide();
          let txt = pyodide.runPython(`
    # Need to replace the line below with code for the user to select
    # the file from the user's filesystem and read 
    # its contents line-by-line:
    inp_str = 'ACGTACGT'

    out_str = inp_str.lower()
    with open('/out.txt', 'w') as fh:
        print(out_str, file=fh)
    with open('/out.txt', 'rt') as fh:
        out = fh.read()
    out
`);

          const blob = new Blob([txt], {type : 'application/text'});
          let url = window.URL.createObjectURL(blob);
          
          var downloadLink = document.createElement("a");
          downloadLink.href = url;
          downloadLink.text = "Download output";
          downloadLink.download = "out.txt";
          document.body.appendChild(downloadLink);
      }
      main();
    </script>
  </body>
</html>

We have external users that may not be advanced computer users. We can specify they need to use Google Chrome browser, but not specific releases like Chrome Canary. We cannot ask them to manually enable the File System API.

Based on the suggestion by TachyonicBytes, I tried the code below. Now I got the error below. I also cannot see something like select file button, or any obvious code for it:

Uncaught (in promise) DOMException: Failed to execute 'showDirectoryPicker' on 'Window': Must be handling a user gesture to show a file picker.
    at main (file:///Users/foo/bar/upload_nativefs.html:10:35)
    at file:///Users/foo/bar/upload_nativefs.html:26:7

This is line 10 referred to in the error message:

const dirHandle = await showDirectoryPicker();

And the full page is pasted below:

<!doctype html>
<html>
  <head>
      <script src="https://cdn.jsdelivr.net/pyodide/v0.22.1/full/pyodide.js"></script>
  </head>
  <body>
    Analyze input <br>
    <script type="text/javascript">
      async function main(){
          const dirHandle = await showDirectoryPicker();
          if ((await dirHandle.queryPermission({ mode: "readwrite" })) !== "granted") {
              if (
                  (await dirHandle.requestPermission({ mode: "readwrite" })) !== "granted"
              ) {
                  throw Error("Unable to read and write directory");
              }
          }    
          let pyodide = await loadPyodide();
          const nativefs = await pyodide.mountNativeFS("/mount_dir", dirHandle);
          
          pyodide.runPython(`
  import os
  print(os.listdir('/mount_dir'))
`);
      }
      main();
    </script>
  </body>
</html>
Solution

  • So, I am adding another answer, because I think I fully solved it.

    You error comes from the fact that accessing the filesystem with await showDirectoryPicker(); has to be invoked by the user (cannot be invoked directly). So, the solution is to use a user event (such as clicking a button) for the user to trigger the function. I added the full solution here.

    Basically, I replaced the main call with the button.

    <!doctype html>
<html>
  <head>
      <script src="https://cdn.jsdelivr.net/pyodide/v0.22.1/full/pyodide.js"></script>
  </head>
  <body>
    <button>Analyze input</button>
    <script type="text/javascript">
      async function main(){
          const dirHandle = await showDirectoryPicker();
          if ((await dirHandle.queryPermission({ mode: "readwrite" })) !== "granted") {
              if (
                  (await dirHandle.requestPermission({ mode: "readwrite" })) !== "granted"
              ) {
                  throw Error("Unable to read and write directory");
              }
          }
          let pyodide = await loadPyodide();
          const nativefs = await pyodide.mountNativeFS("/mount_dir", dirHandle);

          pyodide.runPython(`
            import os
            print(os.listdir('/mount_dir'))
          `);
      }

      const button = document.querySelector('button');
      button.addEventListener('click', main);
    </script>
  </body>
</html>

    On a fairly recent Chromium, after you click the button, you will get the system file picker in order to select a directory to mount. Afterward, you will have some pop-ups explaining what is happening. Something like this:

    enter image description here

    After you accept, the end result will look like this:

    enter image description here

    , meaning your browser has access to the filesystem, and you can use the other python functions to work with it.

    Edit:

    Now that I understand the problem a little better, I made the code get the file selection picker. Unfortunately, it seems that the filesystem api makes it hard to get the directory from the file, so there is still a call to the directory picker. I made did some error handling in case the file is not in the selected directory, but for now, the ergonomics are lacking in that respect.

    The javascript looks like this:

          // Check if handle exists inside directory our directory handle
      async function inDirectory(directoryHandle, fileHandle) {
        return await directoryHandle.resolve(fileHandle) === null;
      }

      async function main() {
        const dirHandle = await showDirectoryPicker();
        const [fileHandle] = await showOpenFilePicker();
        const bad = await inDirectory(dirHandle, fileHandle);
        if (bad) {
          throw Error("The file is not in the mounted directory");
          return;
        }

        const fileData = await fileHandle.getFile();
        console.log(fileData.name);
          if ((await fileHandle.queryPermission({ mode: "readwrite" })) !== "granted") {
              if (
                  (await fileHandle.requestPermission({ mode: "readwrite" })) !== "granted"
              ) {
                  throw Error("Unable to read and write directory");
              }
          }
          let pyodide = await loadPyodide();
          const nativefs = await pyodide.mountNativeFS("/mount_dir", dirHandle);

          pyodide.runPython(`
            import os
            print(os.listdir('/mount_dir'))
            print(open('/mount_dir/${fileData.name}').read())
          `);
      }

      const button = document.querySelector('button');
      button.addEventListener('click', main);

    I also updated the repo with it. If this is still lacking in some respect, please detail the question even further.

    Edit 2:

    Well, the code ended up being much simpler. I did away with the directory entirely, now it asks you to pick a file, it reads the contents into JS memory, it constructs the Python conversion function and then it just calls the function and puts the result in Blob.

    The code to construct the download link appears after the Analyze button has been pressed.

    This is the code, as usually, updated in the repo as well:

          async function main() {
        // Get the file contents into JS
        const [fileHandle] = await showOpenFilePicker();
        const fileData = await fileHandle.getFile();
        const contents = await fileData.text();

        // Create the Python convert toy function
        let pyodide = await loadPyodide();
        let convert = pyodide.runPython(`
from pyodide.ffi import to_js
def convert(contents):
    return to_js(contents.lower())
convert
      `);

        let result = convert(contents);
        console.log(result);

        const blob = new Blob([result], {type : 'application/text'});

        let url = window.URL.createObjectURL(blob);

        var downloadLink = document.createElement("a");
        downloadLink.href = url;
        downloadLink.text = "Download output";
        downloadLink.download = "out.txt";
        document.body.appendChild(downloadLink);

      }
      const button = document.querySelector('button');
      button.addEventListener('click', main);

    The code ended up much, much cleaner.