ibm-cloudanalytics-engine

How can I replace 'system' libraries on IBM Analytics Engine?

To help debug an issue with a yarn application, I need to modify some of the system code on IAE to provide more debug output.

I have retrieved this jar file from the cluster to my local machine:

/usr/hdp/current/hadoop-client/hadoop-aws.jar

I've modified the bytecode to log more information when the an exception is thrown on checkOpen():

public class S3AOutputStream extends OutputStream {
  ...
  void checkOpen() throws IOException {
    if (closed.get()) {

      // some log4j statements added to the bytecode here ...

      throw new IOException("Output Stream closed");
    }
  }
  ...
}

However, I'm unable to save the library with my changes back to the cluster because I don't have root access.

How can I deploy my modified jar files to the cluster? Assume that I need to install the libraries on the name node and compute nodes.

Solution

  • This is not currently possible with IBM Analytics Engine.

    Please raise a support ticket describing your issue.