Trying to run DL4J using NVIDIA cuDNN library (Windows10):
https://deeplearning4j.konduit.ai/multi-project/explanation/configuration/backends/cudnn
In my pom I added:
<properties>
<java.version>16</java.version>
<dl4j-master.version>1.0.0-M2.1</dl4j-master.version>
<nd4j.backend>nd4j-cuda-11.6</nd4j.backend>
<jcommon.version>1.0.24</jcommon.version>
<jfreechart.version>1.0.13</jfreechart.version>
<javacv.version>1.5.5</javacv.version>
<logback.version>1.2.6</logback.version>
<maven.compiler.source>16</maven.compiler.source>
<maven.compiler.target>16</maven.compiler.target>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>${nd4j.backend}</artifactId>
<version>${dl4j-master.version}</version>
</dependency>
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>${nd4j.backend}</artifactId>
<version>${dl4j-master.version}</version>
<classifier>windows-x86_64-cudnn</classifier>
</dependency>
<!-- https://mvnrepository.com/artifact/org.bytedeco/cuda-platform-redist -->
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>cuda-platform-redist</artifactId>
<version>11.6-8.3-1.5.7</version>
</dependency>
<dependency>
<groupId>org.datavec</groupId>
<artifactId>datavec-api</artifactId>
<version>${dl4j-master.version}</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-core</artifactId>
<version>${dl4j-master.version}</version>
</dependency>
...
I create uber-jar:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.4.2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
<configuration>
<archive>
<manifest>
<mainClass>com.somename.Launcher</mainClass>
</manifest>
</archive>
<descriptorRefs>
<descriptorRef>jar-with-dependencies</descriptorRef>
</descriptorRefs>
</configuration>
</plugin>
</plugins>
</build>
After running jar, I got the error: java.lang.RuntimeException: Error loading ND4J Compressors via service loader: No compressors were found. This usually occurs when running ND4J UI from an uber-jar, which was built incorrectly (without services resource files being included)
10:39:06.143 [main] INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [JCublasBackend] backend
10:39:15.316 [main] INFO org.nd4j.nativeblas.NativeOpsHolder - Number of threads used for linear algebra: 32
10:39:15.363 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Backend used: [CUDA]; OS: [Windows Server 2019]
10:39:15.363 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Cores: [8]; Memory: [4,0GB];
10:39:15.364 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Blas vendor: [CUBLAS]
10:39:15.390 [main] INFO org.nd4j.linalg.jcublas.JCublasBackend - ND4J CUDA build version: 11.6.55
10:39:15.391 [main] INFO org.nd4j.linalg.jcublas.JCublasBackend - CUDA device 0: [Tesla T4]; cc: [7.5]; Total memory: [16105865216]
10:39:15.391 [main] INFO org.nd4j.linalg.jcublas.JCublasBackend - Backend build information:
MSVC: 192930146
STD version: 201402L
DEFAULT_ENGINE: samediff::ENGINE_CUDA
HAVE_FLATBUFFERS
HAVE_CUDNN
10:39:32.606 [main] ERROR org.nd4j.linalg.compression.BasicNDArrayCompressor - Error loading ND4J Compressors via service loader: No compressors were found. This usually occurs when running ND4J UI from an uber-jar, which was built incorrectly (without services resource files being included)
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.nd4j.linalg.factory.Nd4j.getCompressor(Nd4j.java:5288)
at org.nd4j.linalg.api.ndarray.BaseNDArray.getDouble(BaseNDArray.java:1785)
at org.nd4j.linalg.api.ndarray.BaseNDArray.getInt(BaseNDArray.java:1753)
at org.deeplearning4j.datasets.datavec.RecordReaderMultiDataSetIterator.convertWritablesBatched(RecordReaderMultiDataSetIterator.java:413)
at org.deeplearning4j.datasets.datavec.RecordReaderMultiDataSetIterator.convertFeaturesOrLabels(RecordReaderMultiDataSetIterator.java:353)
at org.deeplearning4j.datasets.datavec.RecordReaderMultiDataSetIterator.nextMultiDataSet(RecordReaderMultiDataSetIterator.java:326)
at org.deeplearning4j.datasets.datavec.RecordReaderMultiDataSetIterator.next(RecordReaderMultiDataSetIterator.java:206)
at org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator.next(RecordReaderDataSetIterator.java:346)
at org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator.next(RecordReaderDataSetIterator.java:421)
at org.deeplearning4j.datasets.datavec.RecordReaderDataSetIterator.next(RecordReaderDataSetIterator.java:53)
at com.packagename.service.FAClassifierLearning.main(FAClassifierLearning.java:69)
Caused by: java.lang.RuntimeException: Error loading ND4J Compressors via service loader: No compressors were found. This usually occurs when running ND4J UI from an uber-jar, which was built incorrectly (without services resource files being included)
at org.nd4j.linalg.compression.BasicNDArrayCompressor.loadCompressors(BasicNDArrayCompressor.java:66)
at org.nd4j.linalg.compression.BasicNDArrayCompressor.<init>(BasicNDArrayCompressor.java:46)
at org.nd4j.linalg.compression.BasicNDArrayCompressor.<clinit>(BasicNDArrayCompressor.java:39)
... 11 more
What did I miss? How can these compressors be added?
Could you try using the maven-shade-plugin as an alternative?
Shade would look like this:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.2.1</version>
<configuration>
<shadedArtifactAttached>true</shadedArtifactAttached>
<createDependencyReducedPom>false</createDependencyReducedPom>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>reference.conf</resource>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer" />
</transformers>
</configuration>
</execution>
</executions>
</plugin>
If you want to continue using assembly, =look in the jar you'll find that certain resources are missing. Every backend has properties files: https://github.com/deeplearning4j/deeplearning4j/tree/master/nd4j/nd4j-backends/nd4j-backend-impls/nd4j-cuda/src/main/resources
META-INF/resources files are needed as well.
You'll see those here including the relevant compressor file: https://github.com/deeplearning4j/deeplearning4j/tree/74ca3775d9a01dfcd1b1058e36e33e3c1fbfa93a/nd4j/nd4j-backends/nd4j-backend-impls/nd4j-cuda/src/main/resources/META-INF/services
Your assembly declaration appears to miss some files.
We mainly use uber jar not assembly in our day to day but I found another stack overflow post here: Merging META-INF/services files with Maven Assembly plugin
Either way please confirm if your zip file is missing these files and ensure they're added.