performancechef-infrachef-recipechef-solo

why is chef-client synchronization slow for local cookbooks


Background

I'm running chef-client --local-mode with cookbook path set to ./berks-cookbooks these cookbooks where already downloaded by berkshelf before running chef. (chef sdk 4.0.0, berkshelf 3.2.3)

Before running my scripts, chef has to run "Synchronizing Cookbooks" and this step takes 2 to 3 minutes. I understand that chef is downloading dependant cookbooks in this time and checking their hashes.

my question

If I'm using berkshelf and already have the cookbooks saved locally, why does chef spend so long downloading Cookbooks?

Anyone know a way to speed it up?


Solution

  • I wondered about this as well and finally did a strace on the chef-client process to see what it was doing. For every cookbook available to the chef client when you run in local mode, it reads through files in all other cookbooks that are available (and to make matters worse it seems to be doing it more than once for every cookbook).

    This means the time to synchronize cookbooks in local mode will grow with the square of the number of cookbooks available.

    One thing you can do to minimize this time is to remove all unnecessary cookbooks before the run - ie. synchronize your cookbooks manually before hand and remove the ones that will not be used.

    As I haven't checked the code, I don't know why it does this, but the real fix would be to change the code to not reread the same files over and over again.