cxeon-phicilk

Reading files to shared memory


I am reading a binary file that I want to offload directly to the Xeon Phi through Cilk and shared memory.

As we are reading fairly much data at once each time and binary data the preferred option is to use fread.

So if I make a very simple example it would go like this

#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>

_Cilk_shared uint8_t* _Cilk_shared buf;

int main(int argc, char **argv) {
  printf("Argv is %s\n", argv[1]);
  FILE* infile = fopen(argv[1], "rb");
  buf = (_Cilk_shared uint8_t*) _Offload_shared_malloc(2073600);
  int len = fread(buf, 1, 2073600, infile);
  if(ferror(infile)) {
    perror("ferror");
  }
  printf("Len is %d and first value of buf is %d\n", len, *buf);
  return 0;
}

The example is very simplified from the real code but enough to examplify the behavior.

This code would then return

ferror: Bad address
Len is 0 and first value of buf is 0

However if we switch out the fread for a fgets (not very suitable for reading binary data, specially with the return value) things work great.

That is we switch fgets((char *) buf, 2073600, infile); and then drop the len from the print out we get

first value of buf is 46

Which fits with what we need and I can run _Offload_cilk on a function with buf as an argument and do work on it.

Is there something I am missing or is fread just not supported? I've tried to find as much info on this from both intel and other sites on the internet but I have sadly been unable to.

----EDIT----

After more research into this it seems that running fread on the shared memory with a value higher than 524287 (524287 is 19 bits exactly) fread gets the error from above. At 524287 or lower things work, and you can run as many fread as you want and read all the data.

I am utterly unable to find any reason written anywhere for this.


Solution

  • Can you try to insert something like this before the fread calls?

    memset(buf, 0, 2073600); // after including string.h
    

    This trick worked for me, but I don't know why (lazy allocation?).

    FYI, you can also post a MIC question on this forum.