I have successfully upgraded Gitlab to 12.1.6 on Ubuntu 16.04 and checked that all was working. After making sure Ubuntu was fully up to date I checked again: Gitlab worked.
I then used the do-release-upgrade command to update to Ubuntu 18.04. After restart, everything seems to work ok, but Gitlab refuses to start.
I get the following errors:
fail: alertmanager: runsv not running
fail: gitaly: runsv not running
fail: gitlab-exporter: runsv not running
fail: gitlab-workhorse: runsv not running
fail: grafana: runsv not running
fail: logrotate: runsv not running
fail: nginx: runsv not running
fail: node-exporter: runsv not running
fail: postgres-exporter: runsv not running
fail: postgresql: runsv not running
fail: prometheus: runsv not running
fail: redis: runsv not running
fail: redis-exporter: runsv not running
fail: sidekiq: runsv not running
fail: unicorn: runsv not running
I tried:
gitlab-ctl reconfigure --> runs with success
I installed runit with success, rebooted the machine but the errors remain
I found a similar issue here: on Stackoverflow, followed the instructions (yum --> apt), still no success
and here on Gitlab. This advised to run
sudo systemctl restart gitlab-runsvdir
sudo gitlab-ctl restart
But the first command never finishes
I found this: on Gitlab which states to run
sudo gitlab-rake gitlab:env:info --trace
Output:
** Invoke gitlab:env:info (first_time)
** Invoke gitlab_environment (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute gitlab_environment
** Execute gitlab:env:info
System information
System: Ubuntu 18.04
Current User: git
Using RVM: no
Ruby Version: 2.6.3p62
Gem Version: 2.7.9
Bundler Version:1.17.3
Rake Version: 12.3.3
Redis Version: 3.2.12
Git Version: 2.24.1
Sidekiq Version:5.2.7
Go Version: unknown
rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/pg-1.1.4/lib/pg.rb:56:in `initialize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/pg-1.1.4/lib/pg.rb:56:in `new'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/pg-1.1.4/lib/pg.rb:56:in `connect'
which suggests that the Postgresql server is not running. I have no idea how to start it. Any ideas?
UPDATE: Since then I found an answer that said for some reason on the reboot command the runsv process gets stuck. To solve this you have to:
I wanted to post the same above question. This happens to me when i do the monthly upgrade/updates to our GitLab. I tried your solution before, but to me they didn't solve the problem.
I present my solution, but use these tips with caution, as I still sometimes struggle with this problem in different ways.
I do a combination of these command, the order of these steps are really important(!):
Reference: In this issue Stan Hu's answer helped me: https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/3744
Now yesterday I messed up and didn't do the first step, which could cause my problems, because somehow then for 10 hours the postgresql server was down and it couldn't make it back up again.
If i would have stick to the order, maybe it wouldn't have happened, because it didn't happen in the past while I upgraded/updated GitLab several times.
This was the connection error yesterday, same issue as questioner's
Automatic restart this morning
This morning magically the server at 7:13 am, was accepting connections again. (althought i tried to restart it yesterday, it didn't make any difference then) Still GitLab wasn't reachable online from web browser as of 7:30 am. One hour later, and a few runsvdir restarting/starting commands after (added it to the commands order), the GitLab is working. I have no idea why.
