TL;DR :-) fetching https urls is working in both python and R, but not when python code is run from R.
While running some code in documentation of package RKeras , I have repeatedly run into the fatal error : "unknown url type: https" The problem originates in Keras which is implemented in Python.
I dug into the problem and found that this was happening when Keras was calling the function urlretrieve in the module six to retrieve data from https url.
I then tested the code in iPython and found it to be working perfectly.
<!-- language: lang-python -->
from six.moves.urllib.request import urlretrieve
urlretrieve(url="https://www.google.com")
Then I try doing the same in R and it fails
<!-- language: lang-r -->
library(reticulate)
py_run_string('from six.moves.urllib.request import urlretrieve')
py_run_string("urlretrieve(url='https://www.google.com')")
However, same thing works in R with plain http
<!-- language:lang-r -->
py_run_string("urlretrieve(url='http://www.google.com')")
For the record, https works fine within my R with packages like httr.
I am totally out of my depth here. What could be happening?
Here is some output about my env
R:
sessionInfo()
# R version 3.4.4 (2018-03-15)
# Platform: x86_64-pc-linux-gnu (64-bit)
# Running under: Ubuntu 16.04.4 LTS
# other attached packages:
# [1] reticulate_1.6 kerasR_0.8.0
And Python (as seen from R)
py_config()
version: 3.5.2 |Anaconda custom (64-bit)|
(default, Jul 2 2016, 17:53:06) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
Will much appreciate your time and effort.
Edit: Some more debug info in R:
py_run_string("from OpenSSL import SSL")
# ImportError: /xxx/python3.5/lib-dynload/_ssl.so: undefined symbol: SSLv2_method
For the time being, I am posting a partial solution to my particular problem. This is not going to solve the general problem with Reticulate and SSL, but is a decent temporary workaround to help anyone using KerasR or Keras and finding it is not able to download models and databases
Keras / KerasR uses a cache to avoid future downloads of the same object. In my case, it is ~/.keras/
So copy the URL of failed download from the console, download the object with the browser and save to keras' cache directory.
cache for models : ~/.keras/models/
cache for datasets : ~/.keras/datasets/
(where ~ is my home directory)
Of course, this is a workaround and I am still looking forward to someone posting a proper solution that works system-wide.