I’m encountering an issue in GitLab where any attempt to make significant changes, such as deleting a project or adjusting repository permissions, results in a:
'500 - We're sorry. Something went wrong on our end.' Error
Upon further investigation, I discovered that the underlying error is the following:
HTTP_V2::BlockedUrlError - URL is blocked: Requests to the local network are not allowed
After researching the issue, I learned that enabling the 'Allow requests to the local network from web hooks and services' option in the 'Outbound requests' settings should resolve the problem.
However, when I attempt to enable this setting in the GitLab UI, I encounter the same '500' error, creating a chicken-egg problem.
My Configuration::
GITLAB_OMNIBUS_CONFIG: | external_url ‘http:url.com’ nginx[‘listen_https’] = false nginx[‘real_ip_header’] = ‘X-Forwarded-For’ nginx[‘real_ip_recusrive’] = ‘on’
I’ve spent so much time on this issue, I really do not know what to do. Any help is highly appreciated. Thank you very much.
I've had the exact same problem and managed to resolve it.
If you look in you log and see mention of an OpenSSL::Cipher::CipherError
or something similar, there is a chance that the application settings encryption is out of sync with your current secret keys. This results in Gitlab being unable to de/encrypt your application settings. Which results in a 500 error every time you try to update them.
I don't know how it happened to my test setup, but for me that was the underlying issue. If you see the same, here's how to fix that.
If you are not too squeamish, you can just throw away the application settings completely (it will create new settings using the current keys from the /etc/gitlab/gitlab-secrets.json
- Gitlab::Application.secrets
in the console):
gitlab-rails console
(it takes a while to start, wait until the header and the "irb(main)"-prompt appear)ApplicationSetting.first.delete
ApplicationSetting.first
should output -> nil
exit
)gitlab-ctl restart
After a while it should come back online (you don't even have to restart the container) and then you should be able to update your settings.
If you've customized other things (the ApplicationSetting
does contain all +- 600 settings ;-) ) you'll have to restore those.
In any case, I can now update my "Outbound requests"-settings without issue.
Edit:
Per the comments (for encryption problems besides Application settings):
If you encounter a similar error when deleting a project, you need to reset the encryption for the project by resetting the tokens for runners. This can be done in the Gitlab-Rails console as well:
Project.find_by_full_path('root/my-project').update(runners_token: nil, runners_token_encrypted:nil)
The source also describes a second way via the Database Console.