I am trying to create this python binding to a electronjs app I create, where the python part is used to analyze data, and the result (a python dictionary) is passed to the web browser to be visualized. It all worked like a charm for small amount of data, but when I tried to pass a large result dictionary (~200MB), the window was created, but closed without warning in the terminal (I couldn't check devtool), although there seem to be a subprocess kept running on the backend. This problem occurred on both windows and Ubuntu machines, and I would really appreciate your help.
here is my code:
def view(data):
config = data
settings = {
"debug": True,
"log_severity": cef.LOGSEVERITY_INFO,
"log_file": "debug.log",
}
cef.Initialize(settings=settings)
browser_setting = { "file_access_from_file_urls_allowed":True,\
"universal_access_from_file_urls_allowed": True,\
"web_security_disabled":True}
browser = cef.CreateBrowserSync(url='file://' + os.path.realpath("index_cefpython.html"),
window_title="Javascript Bindings", settings = browser_setting)
browser.SetClientHandler(LoadHandler(config))
bindings = cef.JavascriptBindings()
browser.SetJavascriptBindings(bindings)
cef.MessageLoop()
del browser
cef.Shutdown()
return
class LoadHandler(object):
def __init__(self, config):
self.config = config
def OnLoadEnd(self, browser, **_):
browser.ExecuteFunction("defineData", self.config)
on the JS side, I have:
<script type="text/javascript">
function defineData(datainput){
console.log("start")
data = datainput;
Main();
}
</script>
and here is all the messages printed out in the terminal:
[0312/104311.439:INFO:cefpython_app.cpp(199)] [Browser process] OnBeforeChildProcessLaunch() command line: "<>\Anaconda3\lib\site-packages\cefpython3\subprocess" --type=gpu-process --no-sandbox --locales-dir-path="<>\Anaconda3\lib\site-packages\cefpython3\locales" --log-file=debug.log --log-severity=info --resources-dir-path="<>\Anaconda3\lib\site-packages\cefpython3" --lang=en-US --disable-gpu-shader-disk-cache /prefetch:2
[0312/104311.440:INFO:cef_log.cpp(8)] [Browser process] Switch already set, ignoring: disable-gpu-shader-disk-cache
[0312/104311.440:INFO:cefpython_app.cpp(199)] [Browser process] OnBeforeChildProcessLaunch() command line: "<>\Anaconda3\lib\site-packages\cefpython3\subprocess" --type=gpu-process --no-sandbox --locales-dir-path="C:\Users\Xiangyun\Anaconda3\lib\site-packages\cefpython3\locales" --log-file=debug.log --log-severity=info --resources-dir-path=" <>\Anaconda3\lib\site-packages\cefpython3" --lang=en-US --disable-gpu-shader-disk-cache --gpu-preferences=KAAAAAAAAACAA4CAAQAAAAAAAAAAAGAAEAAAAAAAAAAAAAAAAAAAACgAAAAEAAAAIAAAAAAAAAAoAAAAAAAAADAAAAAAAAAAOAAAAAAAAAAQAAAAAAAAAAAAAAAKAAAAEAAAAAAAAAAAAAAACwAAABAAAAAAAAAAAQAAAAoAAAAQAAAAAAAAAAEAAAALAAAA --locales-dir-path="<>\Anaconda3\lib\site-packages\cefpython3\locales" --log-file=debug.log --log-severity=info --resources-dir-path="<>\Anaconda3\lib\site-packages\cefpython3" --lang=en-US /prefetch:2
[0312/104311.473:INFO:cef_log.cpp(8)] [Browser process] CreateBrowserSync() called
DevTools listening on ws://127.0.0.1:59232/devtools/browser/abaff316-3b5c-4647-8af3-c7c521146d08 [0312/104311.473:INFO:cef_log.cpp(8)] [Browser process] navigateUrl: file://///<>/index_cefpython.html
[0312/104311.475:INFO:cef_log.cpp(8)] [Browser process] CefBrowser::CreateBrowserSync()
[0312/104311.493:INFO:cef_log.cpp(8)] [Browser process] GetPyBrowser(): create new PyBrowser, browserId=1
[0312/104311.516:INFO:cefpython_app.cpp(199)] [Browser process] OnBeforeChildProcessLaunch() command line: "<>\Anaconda3\lib\site-packages\cefpython3\subprocess" --type=renderer --no-sandbox --service-pipe-token=C31AF08C64F1883299C21D068EF5263C --lang=en-US --locales-dir-path=<>\Anaconda3\lib\site-packages\cefpython3\locales" --log-file=debug.log --log-severity=info --resources-dir-path="<>\Anaconda3\lib\site-packages\cefpython3" --disable-gpu-shader-disk-cache /prefetch:1
[0312/104311.528:INFO:cef_log.cpp(8)] [Browser process] GetPyFrame(): underlying frame does not yet exist: browserId = 1, frameId = -4
[0312/104311.529:INFO:cef_log.cpp(8)] [Browser process] GetPyFrame(): underlying frame does not yet exist: browserId = 1, frameId = -4
[0312/104311.531:INFO:cef_log.cpp(8)] [Browser process] CefBrowser::CreateBrowserSync() succeeded
[0312/104311.532:INFO:cef_log.cpp(8)] [Browser process] CefBrowser window handle = 595526
[0312/104311.533:INFO:cef_log.cpp(8)] [Browser process] GetPyFrame(): underlying frame does not yet exist: browserId = 1, frameId = -4
[0312/104311.534:INFO:cef_log.cpp(8)] [Browser process] SendProcessMessage(): message=DoJavascriptBindings, arguments size=1
[0312/104311.534:INFO:cef_log.cpp(8)] [Browser process] MessageLoop()
[0312/104311.612:INFO:client_handler.cpp(40)] [Browser process] OnProcessMessageReceived(): OnContextCreated
[0312/104311.613:INFO:cef_log.cpp(8)] [Browser process] V8ContextHandler_OnContextCreated()
[0312/104311.614:INFO:cef_log.cpp(8)] [Browser process] GetPyFrame(): create new PyFrame, frameId=2
Javascript bindinsgs should not be used to transfer large data. To send big data use http requests instead. Basically either use AJAX requests and run an internal in-app web server or use Resource Handler which allows to handle requests without running a web server.
See the Tutorial document > "Javascript integration" > "Communication using http requests" section (scroll down a bit after clicking link):
https://github.com/cztomczak/cefpython/blob/master/docs/Tutorial.md#javascript-integration
You could try to send data in smaller parts using javacript bindings, but using requests will be better performant.
Check how your app memory spikes when sending the data. Use process monitor on Linux and process manager on Windows.
To find out what failed when sending 200 MB of data, debug app to get stack trace. If using Linux then download libcef.so with debug symbols from Releases page tagged e.g. v66-upstream. Replace original libcef.so in the cefpython3 package with that one. GDB commands can be found here: