A Java application landscape is currently running on Wildfly32@VMWare6.5 under Java21. The application has been extensively tested and is stable in the test stack. Now it is to be deployed in an Openshift cluster.
Several containers were built consisting of wildfly-33.0.2.Final-jdk21.tar.
sh-5.1$ cat /etc/system-release
Red Hat Enterprise Linux release 9.5 (Plow)
sh-5.1$ java --version
openjdk 21.0.4 2024-07-16 LTS
OpenJDK Runtime Environment Temurin-21.0.4+7 (build 21.0.4+7-LTS)
OpenJDK 64-Bit Server VM Temurin-21.0.4+7 (build 21.0.4+7-LTS, mixed mode, sharing)
sh-5.1$ /opt/jboss/wildfly/bin/jboss-cli.sh --connect --controller=localhost:9990 --command="version”
JBoss Admin Command-line Interface
JBOSS_HOME: /opt/jboss/wildfly
Release: 25.0.2.Final
Product: WildFly 33.0.2.Final
JAVA_HOME: /opt/java/openjdk
java.version: 21.0.4
java.vm.vendor: Eclipse Adoptium
java.vm.version: 21.0.4+7-LTS
os.name: Linux
os.version: 5.14.0-427.49.1.el9_4.x86_64
The applications are copied into the container during the Docker build
FROM dxc.com/basys3-base:v1 AS builder
USER jboss
ENV JBOSS_ROOT=/opt/jboss
ENV JBOSS_HOME=/opt/jboss/wildfly
ENV DEPLOY_DIR=$JBOSS_HOME/standalone/deployments/
COPY benutzungsdienst.war $DEPLOY_DIR/benutzungsdienst.war
RUN touch $DEPLOY_DIR/benutzungsdienst.war.dodeploy
WORKDIR $JBOSS_HOME
EXPOSE 8080 9990
CMD [“/opt/jboss/wildfly/bin/standalone.sh”, “-b”, “0.0.0.0”, “-bmanagement”, “0.0.0.0”]
and then accordingly in the openshift cluster
oc version
Client version: 4.17.10
Kustomize version: v5.0.4-0.20230601165947-6ce0bf390ce3
Kubernetes version: v1.30.7
with such a deployment YAML deployed
apiVersion: apps/v1
kind: Deployment
metadata:
name: xxx-deployment
spec:
replicas: 1
selector:
matchLabels:
app: basys3film
template:
metadata:
labels:
app: basys3film
spec:
containers:
- name: basys3film-container
image: xxx.xxx.xxx.xxx:5000/xxx:v1
imagePullPolicy: Always
ports:
- containerPort: 8080
- containerPort: 9990
securityContext:
runAsUser: 1000650000
allowPrivilegeEscalation: false
runAsNonRoot: true
capabilities:
drop: [“ALL”]
seccompProfile:
type: RuntimeDefault
env:
- name: TZ
value: “Europe/Berlin”
- name: WILDFLY_LOG_FILE_NAME
value: “server.film.log”
volumeMounts:
- name: wildfly-dat
mountPath: /opt/jboss/wildfly/standalone/data
- name: wildfly-tmp
mountPath: /opt/jboss/wildfly/standalone/tmp
- name: wildfly-log
mountPath: /opt/jboss/wildfly/standalone/log
volumes:
- name: wildfly-dat
persistentVolumeClaim:
claimName: pvc-dat
- name: wildfly-log
persistentVolumeClaim:
claimName: pvc-log
- name: wildfly-tmp
persistentVolumeClaim:
claimName: pvc-tmp
Three persistent volume claims are mounted in the containers that look something like this
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pvc-tmp
spec:
storageClassName: crc-csi-hostpath-provisioner
accessModes:
- ReadWriteMany
resources:
requests:
storage: “1Gi”
The startup of the Wildflys is clean, the applications are deployed without errors. The problem is that errors can occur very sporadically during the test cases, and then more frequently:
Caused by: java.lang.ClassNotFoundException: com.xxx.yyy.benutzung.dao.datatable.DateUtils from [Module “deployment.benutzungsdienst.war” from Service Module Loader]
Caused by: java.lang.ClassNotFoundException: com.xxx.yyyy.benutzung.utils.Utils from [Module “deployment.benutzungsdienst.war” from Service Module Loader]
Caused by: java.lang.ClassNotFoundException: com.xxx.yyyy.zzz.controller.mag.mag0102.MAG010202Bean from [Module \“deployment.film.war\” from Service Module Loader]"}}
Caused by: org.jboss.as.server.deployment.DeploymentUnitProcessingException: java.lang.ClassNotFoundException: com.xxx.yyyy.zzz.controller.mag.mag0102.MAG010202Bean from [Module \“deployment.basys3film.war\” from Service Module Loader]
If such a Java class cannot be found by Wildfly, it is still present in the file system and can be tracked down with a find -name.
It seems to me as if this error image is only present after I have made the PVC mounts. If I remove the mount and deploy again, I can no longer recreate the error pattern.
I have read that such errors can occur when using symlinks and have also tried the Java switch -Djboss.vfs.forceCanonical=true, but this didn't help either.
Do you have any ideas on how to prevent such behavior?
Thanks a lot and kind regards, Juergen
The issue might be caused by storing WildFly content repository ($JBOSS_HOME/standalone/data) in a persistent volume.
As explained in WildFly documentation , when a deployment is deployed in WildFly (which you are doing in your Dockerfile when you COPY benutzungsdienst.war), the actual binaries (and its classes) are stored in the content repository.
I don't know what your tests are doing but it is possible that you end up with binaries from the content repository in the persistent volume that no longer match the classes that are deployed (as the PV has precedence over the container file system).
Try to remove the PV for wildfly-dat to prevent these random errors.
The other PV for the log and tmp directories are fine.