hadoopdistributed-cache

Geting exception while using DistributedCache in Hadoop


I have used below mentioned code for Distributed Caching in main class.

job.addCacheFile(new URI(args[2]));

Below mentioned code is in reduce class.

@Override
        protected void setup(Reducer<LongWritable, Text, Text, Text>.Context context) throws IOException, InterruptedException {
            super.setup(context);
            URI[] paths = context.getCacheFiles();
            if (paths.length > 0) {
                loadDeliveryStatusCodes(paths[0].toString());
            }
        }
        private void loadDeliveryStatusCodes(String file) {
            String strRead;
            BufferedReader reader = null;
            try {
                reader = new BufferedReader(new FileReader("./some"));
                while ((strRead = reader.readLine()) != null) {
                    String splitarray[] = strRead.split(",");
                    deliveryCodesMap.put(splitarray[0].trim(), splitarray[1].trim());
                }
            } catch (FileNotFoundException e) {
                e.printStackTrace();
            } catch (IOException e) {
                e.printStackTrace();
            } finally {
                try {
                    reader.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }

I am getting below mentioned exception.

@@@@@@@@/user/DeliveryStatusCodes.txt 1
java.io.FileNotFoundException: ./some (No such file or directory)
    at java.io.FileInputStream.open0(Native Method)
    at java.io.FileInputStream.open(FileInputStream.java:195)
    at java.io.FileInputStream.<init>(FileInputStream.java:138)
    at java.io.FileInputStream.<init>(FileInputStream.java:93)
    at java.io.FileReader.<init>(FileReader.java:58)
    at com.hadoop.intellipaat.UserSMSDeliveryJob$USERNameSMSStatusCodeMapper.loadDeliveryStatusCodes(UserSMSDeliveryJob.java:95)

Your help will save my day. Thanks.


Solution

  • It looks like you're missing the #some from end of your args[2].At the moment you have /user/DeliveryStatusCodes.txt, but you should have

    /user/DeliveryStatusCodes.txt#some
    

    Without it, FileReader is literally looking for ./some, which of course won't exist.

    Or you can skip the alias (it's optional) and write

    new FileReader("/user/DeliveryStatusCodes.txt")