multithreadinggem5

How many threads to use when building gem5?

When building gem5, the user can use the -j flag to set the number of threads to use.

In the documentation on building gem5, this note is included:

Note: I’m using -j9 here to execute the build on 9 of my 8 cores on my machine. You should choose an appropriate number for your machine, usually cores+1

Why is using cores+1 threads "appropriate?

I'm assuming that it has something to do with the fact that if one thread is blocked, there should be another thread to switch to. But at this point, why not have cores+2 number of threads (assuming that the payoff is greater than the cost of creating the threads?

In short,
Is there any sort of research on the "optimal" number of threads to use when running multi-threaded processes?

Solution

  • Why is using cores+1 threads "appropriate?

    Well ... a rule of thumb is that when you have N cores then running N+1 threads is going to keep all of your threads busy without wasting too much extra memory (etc) on threads that cannot run because all of the cores are busy.

    However, the "theory" (i.e. modelling) on which this rule of thumb is based makes assumptions that are not necessarily realistic. Plug in different assumptions and you will get a different formula.

    In practice ... N+1 should be treated with a grain of salt. The real optimal thread count for a particular build can only be determined experimentally on your build platform. And the optimal thread count will vary depending on many factors which are difficult to quantify. It is probably more effort to do that than it is worth to find the optimal setting.

    My advice:

    1. Don't over-think it.
    2. If N+1 gives you good results, use it. If not, try a larger or smaller number.

    If someone tells you categorically that N+1 or N*2 or something else is the "optimal" thread count, they probably don't have a sound basis for that. Treat what they say with healthy skepticism! They are entitled to their opinion ... but that is all that it is.

    Is there any sort of research on the "optimal" number of threads to use when running multi-threaded processes?

    The problem is too (mathematically) complicated to come up with a model that can give you good a priori predictions for a previously unseen multi-threaded application.