capistrano3

Capistrano - HTTP Basic: Access denied


I have a an older Rails application that its GitLab's Access Token had to be renewed. After renewing Access Token I now get an error when running capistrano deployment. The new Access Token was created with the following (Role: developer; Scopes: read_repository, write_repository). I am able to successfully git clone to my development box using the new Access Token. I can successfully run git clone and git remote update on the staging server using the new Access Token.

Versions Being Used

Rails -v 4.2.5.1
Ruby -v 2.3.6
capistrano (3.4.0)
capistrano-bundler (1.1.4)
capistrano-file-permissions (1.0.0)
capistrano-rails (1.1.6)
capistrano-rbenv (2.2.0)
sshkit (1.8.1)

I am using the below line in my Rails deploy.rb file

set :repo_url, 'https://name:token@git.edu/cals-it-acs/cals-db-sync/application.git'

The deployment output and errors

$ bundle exec cap staging deploy --trace
** Invoke staging (first_time)
** Execute staging
** Invoke load:defaults (first_time)
** Execute load:defaults
** Invoke bundler:map_bins (first_time)
** Execute bundler:map_bins
** Invoke rbenv:validate (first_time)
** Execute rbenv:validate
DEBUG [29548db2] Running /usr/bin/env [ -d $HOME/.rbenv/versions/2.3.6 ] as cals_db_sync@dev.edu
DEBUG [29548db2] Command: [ -d $HOME/.rbenv/versions/2.3.6 ]
DEBUG [29548db2] Finished in 1.404 seconds with exit status 0 (successful).
** Invoke rbenv:map_bins (first_time)
** Execute rbenv:map_bins
** Invoke deploy (first_time)
** Execute deploy
** Invoke deploy:starting (first_time)
** Execute deploy:starting
** Invoke deploy:check (first_time)
** Execute deploy:check
** Invoke git:check (first_time)
** Invoke git:wrapper (first_time)
** Execute git:wrapper
INFO [9b294597] Running /usr/bin/env mkdir -p /home/cals_db_sync/tmp/cals_db_sync/ as cals_db_sync@dev.edu
DEBUG [9b294597] Command: ( export PATH="/home/cals_db_sync/.rbenv/shims:/home/cals_db_sync/.rbenv/bin:$PATH" RBENV_ROOT="$HOME/.rbenv" RBENV_VERSION="2.3.6" ; /usr/bin/env mkdir -p /home/cals_db_sync/tmp/cals_db_sync/ )
INFO [9b294597] Finished in 0.015 seconds with exit status 0 (successful).
DEBUG Uploading /home/cals_db_sync/tmp/cals_db_sync/git-ssh.sh 0.0%
INFO Uploading /home/cals_db_sync/tmp/cals_db_sync/git-ssh.sh 100.0%
INFO [eecc4434] Running /usr/bin/env chmod +x /home/cals_db_sync/tmp/cals_db_sync/git-ssh.sh as cals_db_sync@dev.edu
DEBUG [eecc4434] Command: ( export PATH="/home/cals_db_sync/.rbenv/shims:/home/cals_db_sync/.rbenv/bin:$PATH" RBENV_ROOT="$HOME/.rbenv" RBENV_VERSION="2.3.6" ; /usr/bin/env chmod +x /home/cals_db_sync/tmp/cals_db_sync/git-ssh.sh )
INFO [eecc4434] Finished in 0.010 seconds with exit status 0 (successful).
** Execute git:check
INFO [3553be1e] Running /usr/bin/env git ls-remote --heads https://name:token@git.edu/cals-it-acs/cals-db-sync/application.git as cals_db_sync@dev.edu
DEBUG [3553be1e] Command: ( export PATH="/home/cals_db_sync/.rbenv/shims:/home/cals_db_sync/.rbenv/bin:$PATH" RBENV_ROOT="$HOME/.rbenv" RBENV_VERSION="2.3.6" GIT_ASKPASS="/bin/echo" GIT_SSH="/home/cals_db_sync/tmp/cals_db_sync/git-ssh.sh" ; /usr/bin/env git ls-remote --heads https://name:token@git.edu/cals-it-acs/cals-db-sync/application.git )
DEBUG [3553be1e]    aa0eee168566fb7357a69208c70ffebe6873881d    refs/heads/3_char_logins
0a1e2e41464c87d370f4a2d998c53cee8ed1b8d8    refs/heads/cleanup_manually_entered_job_rcds_instead_of_all
1462cd26e99535677a3861a0cb553a53633ae72c    refs/heads/conn_to_sfs
b9863ededd8434497afdef828a9a9478a4d04689    refs/heads/develop
3876a8aa7fc14f9bc394f4fe2f623339ebd10d62    refs/heads/master
INFO [3553be1e] Finished in 0.469 seconds with exit status 0 (successful).
** Invoke deploy:check:directories (first_time)
** Execute deploy:check:directories
INFO [226bbf71] Running /usr/bin/env mkdir -p /home/cals_db_sync/app/cals_db_sync/shared /home/cals_db_sync/app/cals_db_sync/releases as cals_db_sync@dev.edu
DEBUG [226bbf71] Command: ( export PATH="/home/cals_db_sync/.rbenv/shims:/home/cals_db_sync/.rbenv/bin:$PATH" RBENV_ROOT="$HOME/.rbenv" RBENV_VERSION="2.3.6" ; /usr/bin/env mkdir -p /home/cals_db_sync/app/cals_db_sync/shared /home/cals_db_sync/app/cals_db_sync/releases )
INFO [226bbf71] Finished in 0.010 seconds with exit status 0 (successful).
** Invoke deploy:check:linked_dirs (first_time)
** Execute deploy:check:linked_dirs
INFO [ea7bd0ce] Running /usr/bin/env mkdir -p /home/cals_db_sync/app/cals_db_sync/shared/log /home/cals_db_sync/app/cals_db_sync/shared/tmp/pids /home/cals_db_sync/app/cals_db_sync/shared/tmp/cache /home/cals_db_sync/app/cals_db_sync/shared/tmp/sockets /home/cals_db_sync/app/cals_db_sync/shared/vendor/bundle /home/cals_db_sync/app/cals_db_sync/shared/public/system as cals_db_sync@dev.edu
DEBUG [ea7bd0ce] Command: ( export PATH="/home/cals_db_sync/.rbenv/shims:/home/cals_db_sync/.rbenv/bin:$PATH" RBENV_ROOT="$HOME/.rbenv" RBENV_VERSION="2.3.6" ; /usr/bin/env mkdir -p /home/cals_db_sync/app/cals_db_sync/shared/log /home/cals_db_sync/app/cals_db_sync/shared/tmp/pids /home/cals_db_sync/app/cals_db_sync/shared/tmp/cache /home/cals_db_sync/app/cals_db_sync/shared/tmp/sockets /home/cals_db_sync/app/cals_db_sync/shared/vendor/bundle /home/cals_db_sync/app/cals_db_sync/shared/public/system )
INFO [ea7bd0ce] Finished in 0.010 seconds with exit status 0 (successful).
** Invoke deploy:check:make_linked_dirs (first_time)
** Execute deploy:check:make_linked_dirs
** Invoke deploy:check:linked_files (first_time)
** Execute deploy:check:linked_files
** Invoke deploy:set_previous_revision (first_time)
** Execute deploy:set_previous_revision
DEBUG [9da1440c] Running /usr/bin/env [ -f /home/cals_db_sync/app/cals_db_sync/current/REVISION ] as cals_db_sync@dev.edu
DEBUG [9da1440c] Command: [ -f /home/cals_db_sync/app/cals_db_sync/current/REVISION ]
DEBUG [9da1440c] Finished in 0.006 seconds with exit status 0 (successful).
DEBUG [e2441a9a] Running /usr/bin/env cat /home/cals_db_sync/app/cals_db_sync/current/REVISION 2>/dev/null as cals_db_sync@dev.edu
DEBUG [e2441a9a] Command: ( export PATH="/home/cals_db_sync/.rbenv/shims:/home/cals_db_sync/.rbenv/bin:$PATH" RBENV_ROOT="$HOME/.rbenv" RBENV_VERSION="2.3.6" ; /usr/bin/env cat /home/cals_db_sync/app/cals_db_sync/current/REVISION 2>/dev/null )
DEBUG [e2441a9a]    3876a8a
DEBUG [e2441a9a] Finished in 0.011 seconds with exit status 0 (successful).
** Invoke deploy:started (first_time)
** Execute deploy:started
** Invoke deploy:updating (first_time)
** Invoke deploy:new_release_path (first_time)
** Execute deploy:new_release_path
** Execute deploy:updating
** Invoke git:create_release (first_time)
** Invoke git:update (first_time)
** Invoke git:clone (first_time)
** Invoke git:wrapper 
** Execute git:clone
DEBUG [9c85054a] Running /usr/bin/env [ -f /home/cals_db_sync/app/cals_db_sync/repo/HEAD ] as cals_db_sync@dev.edu
DEBUG [9c85054a] Command: [ -f /home/cals_db_sync/app/cals_db_sync/repo/HEAD ]
DEBUG [9c85054a] Finished in 0.006 seconds with exit status 0 (successful).
INFO The repository mirror is at /home/cals_db_sync/app/cals_db_sync/repo
** Execute git:update
DEBUG [ca905765] Running /usr/bin/env if test ! -d /home/cals_db_sync/app/cals_db_sync/repo; then echo "Directory does not exist '/home/cals_db_sync/app/cals_db_sync/repo'" 1>&2; false; fi as cals_db_sync@dev.edu
DEBUG [ca905765] Command: if test ! -d /home/cals_db_sync/app/cals_db_sync/repo; then echo "Directory does not exist '/home/cals_db_sync/app/cals_db_sync/repo'" 1>&2; false; fi
DEBUG [ca905765] Finished in 0.006 seconds with exit status 0 (successful).
INFO [eb2d73b6] Running /usr/bin/env git remote update as cals_db_sync@dev.edu
DEBUG [eb2d73b6] Command: cd /home/cals_db_sync/app/cals_db_sync/repo && ( export PATH="/home/cals_db_sync/.rbenv/shims:/home/cals_db_sync/.rbenv/bin:$PATH" RBENV_ROOT="$HOME/.rbenv" RBENV_VERSION="2.3.6" GIT_ASKPASS="/bin/echo" GIT_SSH="/home/cals_db_sync/tmp/cals_db_sync/git-ssh.sh" ; /usr/bin/env git remote update )
DEBUG [eb2d73b6]    Fetching origin
DEBUG [eb2d73b6]    remote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://git.edu/help/topics/git/troubleshooting_git#error-on-git-fetch-http-basic-access-denied
DEBUG [eb2d73b6]    fatal: Authentication failed for 'https://git.edu/cals-it-acs/cals-db-sync/application.git/'
DEBUG [eb2d73b6]    error: Could not fetch origin
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as cals_db_sync@dev.edu: git exit status: 1
git stdout: Fetching origin
git stderr: remote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://git.edu/help/topics/git/troubleshooting_git#error-on-git-fetch-http-basic-access-denied
fatal: Authentication failed for 'https://git.edu/cals-it-acs/cals-db-sync/application.git/'
error: Could not fetch origin
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/sshkit-1.8.1/lib/sshkit/runners/parallel.rb:16:in `rescue in block (2 levels) in execute'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/sshkit-1.8.1/lib/sshkit/runners/parallel.rb:12:in `block (2 levels) in execute'
SSHKit::Command::Failed: git exit status: 1
git stdout: Fetching origin
git stderr: remote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://git.edu/help/topics/git/troubleshooting_git#error-on-git-fetch-http-basic-access-denied
fatal: Authentication failed for 'https://git.edu/cals-it-acs/cals-db-sync/application.git/'
error: Could not fetch origin
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/sshkit-1.8.1/lib/sshkit/command.rb:100:in `exit_status='
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/sshkit-1.8.1/lib/sshkit/backends/netssh.rb:127:in `execute_command'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/sshkit-1.8.1/lib/sshkit/backends/abstract.rb:121:in `block in create_command_and_execute'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/sshkit-1.8.1/lib/sshkit/backends/abstract.rb:121:in `tap'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/sshkit-1.8.1/lib/sshkit/backends/abstract.rb:121:in `create_command_and_execute'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/sshkit-1.8.1/lib/sshkit/backends/abstract.rb:53:in `execute'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/capistrano-3.4.0/lib/capistrano/git.rb:11:in `git'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/capistrano-3.4.0/lib/capistrano/git.rb:29:in `update'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/capistrano-3.4.0/lib/capistrano/tasks/git.rake:53:in `block (5 levels) in <top (required)>'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/sshkit-1.8.1/lib/sshkit/backends/abstract.rb:72:in `with'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/capistrano-3.4.0/lib/capistrano/tasks/git.rake:52:in `block (4 levels) in <top (required)>'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/sshkit-1.8.1/lib/sshkit/backends/abstract.rb:64:in `within'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/capistrano-3.4.0/lib/capistrano/tasks/git.rake:51:in `block (3 levels) in <top (required)>'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/sshkit-1.8.1/lib/sshkit/backends/abstract.rb:15:in `instance_exec'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/sshkit-1.8.1/lib/sshkit/backends/abstract.rb:15:in `run'
/home/cgarndt/.rbenv/versions/2.3.6/lib/ruby/gems/2.3.0/gems/sshkit-1.8.1/lib/sshkit/runners/parallel.rb:13:in `block (2 levels) in execute'
Tasks: TOP => git:create_release => git:update
The deploy has failed with an error: Exception while executing as cals_db_sync@dev.edu: git exit status: 1
git stdout: Fetching origin
git stderr: remote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://git.edu/help/topics/git/troubleshooting_git#error-on-git-fetch-http-basic-access-denied
fatal: Authentication failed for 'https://git.edu/cals-it-acs/cals-db-sync/application.git/'
error: Could not fetch origin
** Invoke deploy:failed (first_time)
** Execute deploy:failed

Solution

  • The fix was to do the following:

    Update Ruby Version

    rbenv install 2.4.10

    rbenv local 2.4.10

    Reinstall Bundler

    gem install bundler -v '1.17.3'

    Update Capistrano to version 3.11.0 and dependent gems.

    I also added the below to Capistrano's deploy.rb file but I don't really think it was needed.

    set :app_deploy_user, '<user-used-to-write-files-to-server>'