Benchmark C++ vs Java, Unrealistic results

I did a simple test, I know C++ is often faster but weirdly the results of this test is unrealistic to my eyes. So, I think there is something about C++ that I don't know.

C++ code is:

#include <stdio.h>
#include <windows.h>
   
unsigned long long s(unsigned long long n)
{
    unsigned long long s = 0;

    for (unsigned long long i = 0; i < n; i++)
        s += i;

    return s;
}

int main()
{
    LARGE_INTEGER freq, start, end;
    QueryPerformanceFrequency(&freq);
    QueryPerformanceCounter(&start);
    
    printf("%llu\n", s(1000000000));

    QueryPerformanceCounter(&end);
    double d = (double) (end.QuadPart - start.QuadPart) / freq.QuadPart * 1000.0;

    printf("Delta: %f\n", d);

    return 0;
}

Java code is:

public class JavaApplication5 {

    public static long s(long n) {
        long s = 0;

        for (long i = 0; i < n; i++) {
            s += i;
        }

        return s;
    }

    public static void main(String[] args) {
        
        long start = System.nanoTime();
        
        System.out.println(s(1000000000));
        
        long end = System.nanoTime();
        
        System.out.println((end - start)/1000000);
    }
}

^{C++ compiler: gcc 4.4.0 and Java: jdk 1.6.0}

Java: 2795 ms

C++ : 0.013517 ms

It says C++ is 206777 times faster than Java! No way! What is wrong in my test?

Update: No matter you put the System.out.println or printf between the measurement or not. The difference is huge.

Solution

Show the compiler options you used. And your REAL code (#include <stdio> isn't your real code).

Your C++ compiler is much smarter than your Java compiler (this is true on average and in your case, but not every C++ compiler is smarter than every Java compiler), and it precomputed the result. The only thing you're timing is the printf call.

On most of the tasks Java is used for, it performs about as well as C++.

VM languages (Java, C#) have additional costs related to JIT compilation, but also benefit from more efficient memory allocation and inlining across shared libraries. And C++ is much much faster at accessing OS syscalls. Beyond that, C++ memory layouts can be carefully tuned for cache behavior; you don't get that level of control in managed languages.

Which of these factors has more influence is completely application-specific. Anyone making a blanket statement that "C++ is faster in general than Java" or "Java is faster in general than C++" is an idiot. Averages don't matter. Performance on YOUR application matters.

And here is my proof, that gcc is precomputing the answer.

On this code:

#include <stdio.h>
#include <windows.h>

unsigned long long s(unsigned long long n)
{
    unsigned long long s = 0;

    for (unsigned long long i = 0; i < n; i++)
        s += i;

    return s;
}

int main( int argc, char** argv )
{
    LARGE_INTEGER freq, start, end;
    QueryPerformanceFrequency(&freq);
    QueryPerformanceCounter(&start);

    printf("%llu\n", s(1000000000));

    QueryPerformanceCounter(&end);
    double d = (double) (end.QuadPart - start.QuadPart) / freq.QuadPart * 1000.0;

    printf("Delta: %f\n", d);

    QueryPerformanceCounter(&start);

    printf("%llu\n", s(atol(argv[1])));

    QueryPerformanceCounter(&end);
    d = (double) (end.QuadPart - start.QuadPart) / freq.QuadPart * 1000.0;

    printf("Delta: %f\n", d);
    return 0;
}

With gcc-4.3.4, using the command-line ./g++-4 -omasoud-gcc.exe -O3 masoud.cpp:

bash-3.2# ./masoud-gcc 1000000000
499999999500000000
Delta: 0.845755
499999999500000000
Delta: 1114.105866

By comparison, MSVC++ 16.00.40219.01 for x64 (2010 SP1), command-line cl /Ox masoud.cpp:

> masoud 1000000000
499999999500000000
Delta: 229.684364
499999999500000000
Delta: 354.275606

VC++ isn't precomputing the answer, but 64-bit code does execute the loop more than three times faster. This is the speed that Java ought to approach.

More fun facts: gcc precomputes the answer faster than the code it generates to calculate it out. Compile time for gcc:

real    0m0.886s
user    0m0.248s
sys     0m0.185s