I am trying to use pyodide with lxml and urllib3, for some reasons I don't understand, when I try to use urllib3
in a class supposed to be a Resolver for lxml etree I get an error NameError: name 'urllib3' is not defined
.
Example code is online at https://martin-honnen.github.io/pyodide-tests/simple-requests-test-case3.html (note that all output goes to the browser console so use F12 to see the output).
Code is doing <script src="https://cdn.jsdelivr.net/pyodide/v0.26.0/full/pyodide.js"></script>
and then
const python = `
import js
import urllib3
import lxml
from lxml import etree as ET
url = base_url + 'foo-transform-module.xsl'
js.console.log(urllib3.request('GET', url).status)
class TestResolver(ET.Resolver):
def resolve(self, url, id, context):
print("Resolving URL '%s'" % url)
if url.startswith('http'):
return self.resolve_file(urllib3.request('GET', url), context)
else:
return False
parser = ET.XMLParser(no_network=False)
parser.resolvers.add(TestResolver())
tree = ET.parse(url, parser)
tree.getroot().tag
`;
async function main() {
let pyodide = await loadPyodide();
await pyodide.loadPackagesFromImports(python);
const locals = new Map();
locals.set('base_url', window.location.href.replace(/[^/]+?$/, ''));
console.log(await pyodide.runPythonAsync(python, { locals : locals }));
};
main();
Full error in console is
Uncaught (in promise) PythonError: Traceback (most recent call last):
File "/lib/python312.zip/_pyodide/_base.py", line 574, in eval_code_async
await CodeRunner(
File "/lib/python312.zip/_pyodide/_base.py", line 394, in run_async
coroutine = eval(self.code, globals, locals)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<exec>", line 28, in <module>
File "src/lxml/etree.pyx", line 3570, in lxml.etree.parse
File "src/lxml/parser.pxi", line 1952, in lxml.etree._parseDocument
File "src/lxml/parser.pxi", line 1978, in lxml.etree._parseDocumentFromURL
File "src/lxml/parser.pxi", line 1881, in lxml.etree._parseDocFromFile
File "src/lxml/parser.pxi", line 1200, in lxml.etree._BaseParser._parseDocFromFile
File "src/lxml/parser.pxi", line 633, in lxml.etree._ParserContext._handleParseResultDoc
File "src/lxml/parser.pxi", line 739, in lxml.etree._handleParseResult
File "src/lxml/etree.pyx", line 329, in lxml.etree._ExceptionContext._raise_if_stored
File "src/lxml/parser.pxi", line 462, in lxml.etree._local_resolver
File "src/lxml/docloader.pxi", line 150, in lxml.etree._ResolverRegistry.resolve
File "<exec>", line 20, in resolve
NameError: name 'urllib3' is not defined
at new_error (pyodide.asm.js:10:9965)
at pyodide.asm.wasm:0x16dbeb
at pyodide.asm.wasm:0x177339
at _PyEM_TrampolineCall_JS (pyodide.asm.js:10:125866)
at pyodide.asm.wasm:0x1c2db7
at pyodide.asm.wasm:0x2c7b17
at pyodide.asm.wasm:0x20a78c
at pyodide.asm.wasm:0x1c34a4
at pyodide.asm.wasm:0x1c37b3
at pyodide.asm.wasm:0x1c3831
at pyodide.asm.wasm:0x29e865
at pyodide.asm.wasm:0x2a4e5c
at pyodide.asm.wasm:0x1c3971
at pyodide.asm.wasm:0x1c35da
at pyodide.asm.wasm:0x17699d
at callPyObjectKwargs (pyodide.asm.js:10:64068)
at Module.callPyObjectMaybePromising (pyodide.asm.js:10:65316)
at wrapper (pyodide.asm.js:10:27006)
at onGlobalMessage (pyodide.asm.js:10:101760)
Is there some flaw in my code? How do I get the imported module to be known in the code of the class?
I deleted old answer because it was wrong.
It seems all problem makes locals
- maybe it removes original urllib3
I found Pyoide FAQ: How can I execute code in a custom namespace?.
It shows some examples with globals
.
It seems this works for me:
let my_namespace = pyodide.toPy({'base_url': window.location.href.replace(/[^/]+?$/, '')});
await pyodide.runPythonAsync(python, {globals: my_namespace});
Information for other users: problem was send by OP to Pyodide as issue
but there is already similar issue
Document weird behavior of runPython locals parameter · Issue #4673 · pyodide/pyodide