Jenkins Cyclic plugin dependency on mina-sshd-api-common and mina-sshd-api-core

A bit of history on how I encountered this problem. I had a CICD server running Jenkins for my Symfony projects on a Ubuntu 18.04 VPS. I had Composer v1.0 running and there were several messages every time I ran a command to upgrade to Composer v2.0. So I ran a composer self update

$ composer self-update 2.5.5

I had PHP version 7.4.15 running on the VPS and Composer v2.0 required PHP 8 so I had to update PHP from the ondrej repositories. At this time I realised that Ubuntu 18.04 Bionic Beaver had reached end of life in May 2023 and the only was to upgrade my distribution. I had some initial hiccups with upgrading Chrome, Docker and Jenkins. With some googling I managed to upgrade Chrome and Docker. For Jenkins I had to use the new repository sigining keys.

After I upgraded the distribution to Ubuntu 20.04 I noticed that Jenkins failed to start. Do I did a

$sudo journalctl -u jenkins.service 

and the log had the following entries

Running with Java 8 from /usr/lib/jvm/java-8-openjdk-amd64/jre, which is older than minimum required version (Java 11)
Supported Java versions are : [11, 17,21]

So I installed a newer version of Java

$sudo update-java-alternatives --set java-17-openjdk-amd64

I restarted Jenkins and this time I found the following entries in the log

Dec 26 16:50:04 ub-s-2vcpu-4gb-blr1-01-02 jenkins[823753]: 2023-12-26 11:20:04.319+0000 [id=30]        INFO        hudson.PluginManager#loadDetachedPlugins: Upgrading Jenkins. The last running version was 2.235.
2. This Jenkins is version 2.426.2.
Dec 26 16:50:04 ub-s-2vcpu-4gb-blr1-01-02 jenkins[823753]: 2023-12-26 11:20:04.397+0000 [id=30]        INFO        hudson.PluginManager#loadDetachedPlugins: Upgraded Jenkins from version 2.235.2 to version 2.426
.2. Loaded detached plugins (and dependencies): []
Dec 26 16:50:04 ub-s-2vcpu-4gb-blr1-01-02 jenkins[823753]: 2023-12-26 11:20:04.634+0000 [id=31]        INFO        jenkins.InitReactorRunner$1#onAttained: Started initialization
Dec 26 16:50:04 ub-s-2vcpu-4gb-blr1-01-02 jenkins[823753]: 2023-12-26 11:20:04.768+0000 [id=31]        SEVERE        hudson.PluginManager$1$3$2$1#reactOnCycle: found cycle in plugin dependencies: (root=Plugin:ss
hd, deactivating all involved) Plugin:sshd -> Plugin:mina-sshd-api-core -> Plugin:ssh-credentials -> Plugin:credentials -> Plugin:sshd
Dec 26 16:50:04 ub-s-2vcpu-4gb-blr1-01-02 jenkins[823753]: 2023-12-26 11:20:04.777+0000 [id=31]        SEVERE        hudson.PluginManager$1$3$2$1#reactOnCycle: found cycle in plugin dependencies: (root=Plugin:ss
hd, deactivating all involved) Plugin:sshd -> Plugin:mina-sshd-api-core -> Plugin:ssh-credentials -> Plugin:credentials -> Plugin:sshd

There was a dependency on the apache mina ssd plugin. With some googling I found that there was an issue with the current version of the mina plugin and the only way is to install the working versions as per the following posts

1. Jenkins reports a cycle dependcy error with plugins after restart and will not start now
2. Mina ssh plugin 2.8.0-30.vf9df64641cb_d has a cyclic dependency, breaks Jenkins plugin loading

I am unable to access the jenkins CLI to downgrade the plugins. So I deleted the current version of the plugins and the directory located at /var/lib/jenkins/plugins/ and downloaded the older versions and copied them to the plugins directory. I renamed the .hpi files to .jpi and restarted jenkins. This time I get the following message in the logs

Dec 26 17:20:14 ub-s-2vcpu-4gb-blr1-01-02 jenkins[826241]: 2023-12-26 11:50:14.271+0000 [id=30]        SEVERE        jenkins.InitReactorRunner$1#onTaskFailed: Failed Loading plugin SSH server v3.303.vefc7119b_ec
23 (sshd)
Dec 26 17:20:14 ub-s-2vcpu-4gb-blr1-01-02 jenkins[826241]: java.io.IOException: Failed to load: SSH server (sshd 3.303.vefc7119b_ec23)
Dec 26 17:20:14 ub-s-2vcpu-4gb-blr1-01-02 jenkins[826241]:  - Update required: Mina SSHD API :: Common (mina-sshd-api-common 2.8.0-21.v493b_6b_db_22c6) to be updated to 2.10.0-69.v28e3e36d18eb_ or higher
Dec 26 17:20:14 ub-s-2vcpu-4gb-blr1-01-02 jenkins[826241]:  - Update required: Mina SSHD API :: Core (mina-sshd-api-core 2.8.0-21.v493b_6b_db_22c6) to be updated to 2.10.0-69.v28e3e36d18eb_ or higher

If I switch back to mina-sshd-common and core to 2.10.. I get the cyclic plugin dependency error if I downgrade to 2.8 it looks for 2.10.. Looks like I have reached a dead end. Is there a way to resolve this and get Jenkins up and running?

Resolved this and got jenkins to startup. Apparently I figured out by trial and error that we should upgrade the plugins when Jenkins is upgraded to a newer version.I had upgraded my version to 2.426.2 but had done nothing to the plugins. They were all from the old installation. So Jenkins wasn't even allowing access to the Pluig-in manager. Manually removing and upgraading each plugin is quite cumbersome. So I downloaded the Jenkinsci Plugin Installation Manager Tool to my home directory.

I then listed the current version of all the plugins using this tool by running the following command from my HOME directory $ sudo java -jar jenkins-plugin-manager-2.12.14.jar --war /usr/share/java/jenkins.war --plugin-download-directory /var/lib/jenkins/plugins/ --plugin-file /var/lib/jenkins/plugins.txt --list >> InstalledPlugins.txt

So the InstalledPlugins.txt is written to the HOME directory. The contents of my InstalledPlugins.txt is as follows. (Yours could be different based on the plugins you've installed)

Resulting plugin list:
ace-editor 1.1
analysis-core 1.96
analysis-model-api 8.1.3
ant 1.11
antisamy-markup-formatter 159.v25b_c67cd35fb_
apache-httpcomponents-client-4-api 4.5.14-150.v7a_b_9d17134a_5
authorize-project 1.4.0
bootstrap4-api 4.6.0-1
bootstrap5-api 5.3.0-1
bouncycastle-api 2.28
branch-api 2.5.6
build-timeout 1.20
caffeine-api 3.1.6-115.vb_8b_b_328e59d8
checks-api 2.0.0
checkstyle 4.0.0
cloudbees-folder 6.14
clover 4.11.1
cloverphp 0.5
command-launcher 100.v2f6722292ee8
commons-lang3-api 3.12.0-36.vd97de6465d5b_
commons-text-api 1.10.0-36.vc008c8fcda_7b_
conditional-buildstep 1.4.1
covcomplplot 1.1.1
crap4j 0.9
credentials 2.3.11
credentials-binding 1.23
cvs 2.11
data-tables-api 1.10.21-2
delivery-pipeline-plugin 1.4.2
discord-notifier 206.vee9f4569ee63
display-url-api 2.3.7
dry 3.0.0
dtkit-api 2.1.2
durable-task 1.34
echarts-api 5.4.0-5
email-ext 2.82
embeddable-build-status 2.0.3
envinject 2.3.0
envinject-api 1.8
external-monitor-job 1.7
font-awesome-api 6.4.0-1
forensics-api 0.7.0
git 4.3.0
git-client 3.3.0
git-server 1.9
github 1.30.0
github-api 1.114.3
github-branch-source 2.8.2
gradle 1.36
handlebars 1.1.1
htmlpublisher 1.23
instance-identity 173.va_37c494ec4e5
ionicons-api 56.v1b_1c8c49374e
jackson2-api 2.15.2-350.v0c2f3f8fc595
jakarta-activation-api 2.0.1-3
jakarta-mail-api 2.0.1-3
javadoc 1.5
javax-activation-api 1.2.0-6
javax-mail-api 1.6.2-9
jaxb 2.3.8-1
jdepend 1.3.0
jdk-tool 66.vd8fa_64ee91b_d
jquery 1.12.4-1
jquery-detached 1.2.1
jquery3-api 3.7.0-1
jsch 0.1.55.2
junit 1207.va_09d5100410f
ldap 1.24
lockable-resources 2.8
mailer 457.v3f72cb_e015e5
mapdb-api 1.0.9.0
matrix-auth 3.1.8
matrix-project 789.v57a_725b_63c79
maven-plugin 3.8
mina-sshd-api-common 2.10.0-69.v28e3e36d18eb_
mina-sshd-api-core 2.10.0-69.v28e3e36d18eb_
momentjs 1.1.1
okhttp-api 3.14.9
pam-auth 1.6
parameterized-trigger 2.39
pipeline-build-step 2.12
pipeline-github-lib 1.0
pipeline-graph-analysis 1.10
pipeline-input-step 2.11
pipeline-milestone-step 1.3.1
pipeline-model-api 1.7.0
pipeline-model-definition 1.7.0
pipeline-model-extensions 1.7.0
pipeline-rest-api 2.13
pipeline-stage-step 2.5
pipeline-stage-tags-metadata 1.7.0
pipeline-stage-view 2.13
plain-credentials 1.7
plot 2.1.7
plugin-util-api 3.3.0
pmd 4.0.0
popper-api 1.16.1-1
resource-disposer 0.14
role-strategy 3.1.1
run-condition 1.3
scm-api 672.v64378a_b_20c60
script-security 1251.vfe552ed55f8d
snakeyaml-api 2.2-111.vc6598e30cc65
ssh 2.6.1
ssh-credentials 1.18.1
ssh-slaves 1.31.2
sshd 3.303.vefc7119b_ec23
structs 324.va_f5d6774f3a_d
subversion 2.13.1
timestamper 1.11.3
token-macro 2.13
trilead-api 2.84.v72119de229b_7
violations 0.7.11
warnings 5.0.1
warnings-ng 8.2.0
windows-slaves 1.6
workflow-aggregator 2.6
workflow-api 1213.v646def1087f9
workflow-basic-steps 2.20
workflow-cps 2.80
workflow-cps-global-lib 2.16
workflow-durable-task-step 2.35
workflow-job 2.40
workflow-multibranch 2.21
workflow-scm-step 2.11
workflow-step-api 639.v6eca_cd8c04a_a_
workflow-support 839.v35e2736cfd5c
ws-cleanup 0.38
xunit 2.3.9
xvfb 1.2

Make a copy of this file

$ cp InstalledPLugins.txt plugins.txt

Edit the plugins.txt and replace all version numbers with "latest" as follows

ace-editor:latest
analysis-core:latest
analysis-model-api:latest
ant:latest

This means we want the Plugin Installation Manager Tool to install the latest versions for each of these plugins. Copy this file to the Jenkins director

$ cp plugins.txt /var/lib/jenkins/.

Now use the Plugin Installation Manager Tool to upgrade the plugins as per the plugins.txt

$ sudo java -jar jenkins-plugin-manager-2.12.14.jar --war /usr/share/java/jenkins.war --plugin-download-directory /var/lib/jenkins/plugins/ --plugin-file /var/lib/jenkins/plugins.txt

This will install the latest version of plugins. In my case some of the plugins threw 404 errors as they were eithier depreciated or no longer maintained. Once the upgrade is complete start jenkins

$ sudo service jenkins start

Vist the URL and you will see the new version of Jenkins. In my case this is the version that showed up.