I'm spinning up a Windows Container image (provided my MS here) containing the Self-Hosted Integration Runtime to be able to use ADF in a on-premise situation. It ran smoothly until I needed to use Parquet files.
When I pointed the output to a .parquet I got a Data Factory task failure pointing the absence of Java in the Integration Runtime container.
ErrorCode=JreNotFound,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Java Runtime Environment cannot be found on the Self-hosted Integration Runtime machine. It is required for parsing or writing to Parquet/ORC files. Make sure Java Runtime Environment has been installed on the Self-hosted Integration Runtime machine.,Source=Microsoft.DataTransfer.Common,''Type=System.DllNotFoundException,Message=Unable to load DLL 'jvm.dll': The specified module could not be found.
I took the path of modifying the build.ps1 file to install and configure the dependencies during container image creation. These are the steps taken:
Install Microsoft Visual C++ 2010 Service Pack 1 (here)
Instal JDK provided from Microsoft OpenJDK 17.0.6 LTS - 64bits MSI (here)
Manually set JAVA_HOME environment variable: setx -m JAVA_HOME "C:\Program Files\Microsoft\jdk-17.0.6.10-hotspot" (As for as I got SHIR will look in the registry for JRE location and in case it is not found it will look JAVA_HOME env var).
Java seems to be working fine, since when I run java -version it returns me the following output.
openjdk version "17.0.6" 2023-01-17 LTS
OpenJDK Runtime Environment Microsoft-7209853 (build 17.0.6+10-LTS)
OpenJDK 64-Bit Server VM Microsoft-7209853 (build 17.0.6+10-LTS, mixed mode, sharing)
Everything seems to be Ok but I keep getting the error I mentioned above. I tried to install JRE7, JRE8, configure registry keys, but nothing seems to work.
We are experiencing the exact same issue since our production environments have been upgraded from 5.24x to 5.26x. Looking at the release notes for the Integration Runtime upgrades found here: release notes
You can see that there are known issues regarding "finding" dll files. They dont specify which files have issues though.
We attempted upgrading to the newest version 5.28x, but that still gives us the same issues.
We are currently downgrading to 5.27.8441.3 download link. I will update my response if that solved the issue.
UPDATE: We have attempted the current version: 5.28.8473.1 a previous version: 5.27.8441.3 another previous version: 5.26.8431.1.
all not fixing the issue. I have raised a ticket with Support, and am currently downgrading to our known working version: 5.24.8369.1.
I will update again if this version is actually still working.
UPDATE2: @FlodenH our release pipeline is still running (+- 10 more minutes) but someone told me it works for him again using this older version.
You can download any version by changing this url:
so for the working version it should be: link
UPDATE3: I can confirm that version: 5.24.8369.1 works. Download link can be found above. A ticket was raised with Microsoft and they are investigating it on the newer version, but for now this version is the only one that works for parquet convertion.