In the article https://en.m.wikipedia.org/wiki/Mutual_exclusion#Software_solutions it is given that
These algorithms do not work if out-of-order execution is used on the platform that executes them. Programmers have to specify strict ordering on the memory operations within a thread.
But those algorithms are simple C programs and if we can't be sure of them working as expected in ooo systems how can we be sure that our other programs will work correctly? What do these programs do that fails in ooo situation? Basically when do ooo programs not work, so we can be careful in their usage? Can we trust ooo processors to do what we code?
There are no problems with out-of-order execution within a program (more precisely within a thread). This can only lead to problems if there is concurrency (two or more threads run in parallel), for example with a software mutex.
Barriers with mfence (or lfence and sfence in special cases) can help on x86 plattforms. They instruct the processor that no out-of-order execution is to take place at this point. These are assembler instructions, so in C you have to write
asm volatile("mfence");
or use the corresponding instrinct.
Another problem could be that the compiler arranges the instructions differently than they are in the program or makes other optimizations (for example, does not write the values in the memory at all but keeps them in a register). To prevent this, the keyword volatile must be used at the variables you use for the software mutex.