Trying to replicate my GitHub repository to AWS CodeCommit.
After some troubles the AWS IAM role and machine was successfully configured to fetch from github and push to AWS CodeCommit:
origin https://********:x-oauth-basic@github.com/org/infra.git (fetch)
origin https://********@git-codecommit.ca-central-1.amazonaws.com/v1/repos/infra (push)
while this works well for smaller repositories the main app repository refused to push:
$ git push --mirror
UnrecognizedFormatException: Too many reference update commands
Enumerating objects: 352143, done.
Counting objects: 100% (352143/352143), done.
Delta compression using up to 2 threads
Compressing objects: 100% (60932/60932), done.
send-pack: unexpected disconnect while reading sideband packet
fatal: the remote end hung up unexpectedly
Tried many git config options but no luck.
Googling the error got me to the page Troubleshooting Git clients and AWS CodeCommit that suggests:
Possible fixes: Try pushing branches and tags individually with
git push --all
andgit push --tags
. If you have too many tags, split the tags into multiple pushes. For more information, see Quotas.
We don't use tags and git push --all
worked well. Though the git push --mirror
still didn't work.
The Quotas in AWS CodeCommit says:
Number of references in a single push: Maximum of 4,000, including create, delete, and update. There is no limit on the overall number of references in the repository.
...well, there is more than 5,000 closed pull requests in the repository and that's where the big number of ref objects come from:
$ git show-ref | wc -l
5657
Finally the solution was to push refs in multiple pushes:
$ git push origin "refs/pull/1*"
...
* [new reference] refs/pull/1313/head -> refs/pull/1313/head
...
* [new reference] refs/pull/1999/head -> refs/pull/1999/head
$ git push origin "refs/pull/2*"
...
* [new reference] refs/pull/2000/head -> refs/pull/2000/head
...
* [new reference] refs/pull/2999/head -> refs/pull/2999/head
...
$ git push origin "refs/pull/3*"
...
* [new reference] refs/pull/3000/head -> refs/pull/3000/head
...
* [new reference] refs/pull/3999/head -> refs/pull/3999/head
...after pushing these the git push --mirror
finally worked.