In a large monorepo (.git directory 10GB), I have observed the following behavior.
# When outputing to a file, it is very fast:
time git log -2 > /tmp/X
real 0m0.076s
user 0m0.007s
sys 0m0.046s
# When outputting to the tty, it is more than 10x slower:
time git log -2
# ... Output omitted
real 0m0.830s
user 0m0.078s
sys 0m0.586s
Here are all of my log-related configs:
$ git config -l | grep log
log.mailmap=true
core.logallrefupdates=true
What causes the difference in performance, and how can I get the same performance when git log is outputting to a tty?
More observations:
In a smaller repo, git log -2
is fast even when outputting to tty, so the problem seems correlated with a large repo.
real 0m0.057s
user 0m0.008s
sys 0m0.022s
Cat on the output file is fast, so it is not just writing to stdout that is slow.
time cat /tmp/X
real 0m0.015s
user 0m0.001s
sys 0m0.005s
Forcing a tty with expect
's unbuffer
also slows down git log
:
time unbuffer git log -2 > /tmp/X
real 0m1.007s
user 0m0.104s
sys 0m0.669s
By comparison, unbuffer
itself has ~240 ms overhead:
time unbuffer cat /tmp/X
real 0m0.239s
user 0m0.033s
sys 0m0.111s
As written in your answer, I think you have identified the cause for your slowness: a very high number of references in your local repository slows down git log
when it tries to print the decoration for each commit.
Check what kind of references make the bulk of these. You can compare the following numbers:
wc -l .git/packed-refs # total refs
git branch | wc -l # local branches
git branch -r | wc -l # remote branches
git tag --list | wc -l # tags
If you find out you have a huge amount of remote branches: try running git fetch --prune
to get rid of remote refs (refs in refs/remotes/origin/*
) that have been deleted upstream.
If the 3 numbers do not add up to the total amount of lines in packed-refs
, check what other refs could bring the count higher.
packed-refs
: it is just a text file, one line per ref,git for-each-ref
, for example:git for-each-ref --format="%(refname)" | cut -d/ -f1-2 | sort | uniq -c
cut -d/ -f1-2
says "use /
as a field separator, only keep the first 2 fields", you can narrow on the highest numbers, and get more details by raising the second number in -f1-x
# example: if you find out you have refs looking like `refs/history/*`:
git for-each ref --format=... refs/history | ... | cut -d/ -f1-3