hadooppowerbihdfswebhdfs

Unable Connecting Power BI to Hadoop HDFS failed to get contents


When I'm trying to connect Power BI to Hadoop webhdfs, i get this error

DataSource.Error: HDFS failed to get contents from 'http://xxx.xx.x.x:50070/webhdfs/v1/myFolder/20200626150740_PERSONAL_IDS'. Status code: 503, description: 'Service Unavailable'.

When I access the URL, error like this occurs:

{
  RemoteException: {
    exception: "UnsupportedOperationException",
    javaClassName: "java.lang.UnsupportedOperationException",
    message: "op=NULL is not supported"
  }
}

Anyone can help this out?


Solution

  • An operation (?op=...) is required when you use webhdfs REST API with http(s): scheme.

    For example, when I type this in my browser:

    https://<my.namenode.host>:9871/webhdfs/v1/user?op=LISTSTATUS
    

    ... I get back the list of hdfs directory /user content (similar to what I'll get with hadoop fs -ls /user plus some additional info):

    // 20200709155934
    // https://<my.namenode.host>:9871/webhdfs/v1/user?op=LISTSTATUS
    
    {
      "FileStatuses": {
        "FileStatus": [
          {
            "accessTime": 0,
            "blockSize": 0,
            "childrenNum": 2,
            "fileId": 56899,
            "group": "supergroup",
            "length": 0,
            "modificationTime": 1592790690971,
            "owner": "hdfs",
            "pathSuffix": "hdfs",
            "permission": "700",
            "replication": 0,
            "storagePolicy": 0,
            "type": "DIRECTORY"
          },
          . . . . 
          {
            "accessTime": 0,
            "blockSize": 0,
            "childrenNum": 2,
            "fileId": 16390,
            "group": "hive",
            "length": 0,
            "modificationTime": 1591715444764,
            "owner": "hive",
            "pathSuffix": "hive",
            "permission": "1775",
            "replication": 0,
            "storagePolicy": 0,
            "type": "DIRECTORY"
          },
          . . . . 
          {
            "accessTime": 0,
            "blockSize": 0,
            "childrenNum": 2,
            "fileId": 16454,
            "group": "mazaneicha",
            "length": 0,
            "modificationTime": 1591035918913,
            "owner": "mazaneicha",
            "pathSuffix": "mazaneicha",
            "permission": "720",
            "replication": 0,
            "storagePolicy": 0,
            "type": "DIRECTORY"
          },
          . . . .
        ]
      }
    }
    

    You can find the complete list of REST API operations online.