Debugging ASM-generated bytecode with JDB (or similar)

So I have some malfuctioning code to debug where SOMEthing throws an NPE and I'd like to step through some generated methods in order to try and find out why.

Except stepping blindly is not really useful.

Thread-4[1] list
Source file not found: Foo.java
Thread-4[1] locals
Local variable information not available.  Compile with -g to generate variable information

The code was generated, so of course there is no .java file available for JDB.

And since I don't compile it with javac, there's no specifying any -g flags either.

Can I tell JDB to show me the bytecode, instead (which he obviously has, because otherwise java would have had nothing to execute)?

Can I tell ASM to generate locals information as if it were compiled with javac -g?

Or is there a useful debugger out there that can do what I am looking for?

Generating local variable information is rather easy. Emit the right visitLocalVariable invocations on the target method visitor, declaring name, type and scope of local variables. This will generate the LocalVariableTable attribute in the class file.

When it comes to source level debugging, the tools will simply look for the SourceFile attribute on the class to get the name of a text file to load and display. You can generate it by calling visitSource(fileName, null) on the target class visitor (ClassWriter). The relation between the specified text file and the byte code instructions can be declared via invocations of visitLineNumber on the target method visitor. For ordinary source code, you only have to invoke it when the associated line changes. But for a byte code representation, it would change for every instruction, which may result in a rather large class file so you should definitely make the generation of these debug information optional.

Now, you only need to produce the text file. You may wrap the target ClassWriter in a TraceClassVisitor before passing it to your code generator, to produce a human readable form while generating the code. But we have to extend the Textifier provided by ASM, as we need to track the line number of the buffered text and also want to suppress the generation of output for our line number information itself, which would clutter the source with two additional lines per instruction.

public class LineNumberTextifier extends Textifier {
    private final LineNumberTextifier root;
    private boolean selfCall;
    public LineNumberTextifier() { super(ASM5); root = this; }
    private LineNumberTextifier(LineNumberTextifier root) { super(ASM5); this.root = root; }
    int currentLineNumber() { return count(super.text)+1; }
    private static int count(List<?> text) {
        int no = 0;
        for(Object o: text)
            if(o instanceof List) no+=count((List<?>)o);
            else {
                String s = (String)o;
                for(int ix=s.indexOf('\n'); ix>=0; ix=s.indexOf('\n', ix+1)) no++;
            }
        return no;
    }
    void updateLineInfo(MethodVisitor target) {
        selfCall = true;
        Label l = new Label();
        target.visitLabel(l);
        target.visitLineNumber(currentLineNumber(), l);
        selfCall = false;
    }
    // do not generate source for our own artifacts
    @Override public void visitLabel(Label label) {
        if(!root.selfCall) super.visitLabel(label);
    }
    @Override public void visitLineNumber(int line, Label start) {}
    @Override public void visitSource(String file, String debug) {}
    @Override protected Textifier createTextifier() {
        return new LineNumberTextifier(root);
    }
}

Then, you may generate the class file and the source file together like this:

Path targetPath = …
String clName = "TestClass", srcName = clName+".jasm", binName = clName+".class";
Path srcFile = targetPath.resolve(srcName), binFile = targetPath.resolve(binName);
ClassWriter actualCW = new ClassWriter(0);
try(PrintWriter sourceWriter = new PrintWriter(Files.newBufferedWriter(srcFile))) {
    LineNumberTextifier lno = new LineNumberTextifier();
    TraceClassVisitor classWriter = new TraceClassVisitor(actualCW, lno, sourceWriter);
    classWriter.visit(V1_8, ACC_PUBLIC, clName, null, "java/lang/Object", null);
    MethodVisitor constructor
        = classWriter.visitMethod(ACC_PRIVATE, "<init>", "()V", null, null);
    constructor.visitVarInsn(ALOAD, 0);
    constructor.visitMethodInsn(
        INVOKESPECIAL, "java/lang/Object", "<init>", "()V", false);
    constructor.visitInsn(RETURN);
    constructor.visitMaxs(1, 1);
    constructor.visitEnd();
    MethodVisitor main = classWriter.visitMethod(
        ACC_PUBLIC|ACC_STATIC, "main", "([Ljava/lang/String;)V", null, null);
    Label start = new Label(), end = new Label();
    main.visitLabel(start);
    lno.updateLineInfo(main);
    main.visitFieldInsn(GETSTATIC, "java/lang/System", "out", "Ljava/io/PrintStream;");
    lno.updateLineInfo(main);
    main.visitLdcInsn("hello world");
    lno.updateLineInfo(main);
    main.visitMethodInsn(
        INVOKEVIRTUAL, "java/io/PrintStream", "println", "(Ljava/lang/String;)V", false);
    lno.updateLineInfo(main);
    main.visitInsn(RETURN);
    main.visitLabel(end);
    main.visitLocalVariable("arg", "[Ljava/lang/String;", null, start, end, 0);
    main.visitMaxs(2, 1);
    main.visitEnd();
    classWriter.visitSource(srcName, null);
    classWriter.visitEnd(); // writes the buffered text
}
Files.write(binFile, actualCW.toByteArray());

The “source” file it produces looks like

// class version 52.0 (52)
// access flags 0x1
public class TestClass {


  // access flags 0x2
  private <init>()V
    ALOAD 0
    INVOKESPECIAL java/lang/Object.<init> ()V
    RETURN
    MAXSTACK = 1
    MAXLOCALS = 1

  // access flags 0x9
  public static main([Ljava/lang/String;)V
   L0
    GETSTATIC java/lang/System.out : Ljava/io/PrintStream;
    LDC "hello world"
    INVOKEVIRTUAL java/io/PrintStream.println (Ljava/lang/String;)V
    RETURN
   L1
    LOCALVARIABLE arg [Ljava/lang/String; L0 L1 0
    MAXSTACK = 2
    MAXLOCALS = 1
}

and javap reports

  Compiled from "TestClass.jasm"
public class TestClass
  minor version: 0
  major version: 52
  flags: ACC_PUBLIC
{
  public static void main(java.lang.String[]);
    descriptor: ([Ljava/lang/String;)V
    flags: ACC_PUBLIC, ACC_STATIC
    Code:
      stack=2, locals=1, args_size=1
         0: getstatic     #16                 // Field java/lang/System.out:Ljava/io/PrintStream;
         3: ldc           #18                 // String hello world
         5: invokevirtual #24                 // Method java/io/PrintStream.println:(Ljava/lang/String;)V
         8: return
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0       9     0   arg   [Ljava/lang/String;
      LineNumberTable:
        line 17: 0
        line 18: 3
        line 19: 5
        line 20: 8
}
SourceFile: "TestClass.jasm"

The example generator placed both files into the same directory, which is already sufficient for jdb to use it. It should also work with IDE debuggers when you place the files into the class path resp. source path of a project.

Initializing jdb ...
> stop in TestClass.main
Deferring breakpoint TestClass.main.
It will be set after the class is loaded.
> run TestClass
run  TestClass
Set uncaught java.lang.Throwable
Set deferred uncaught java.lang.Throwable
>
VM Started: Set deferred breakpoint TestClass.main

Breakpoint hit: "thread=main", TestClass.main(), line=17 bci=0
17        GETSTATIC java/lang/System.out : Ljava/io/PrintStream;

main[1] locals
Method arguments:
arg = instance of java.lang.String[0] (id=433)
Local variables:
main[1] step
>
Step completed: "thread=main", TestClass.main(), line=18 bci=3
18        LDC "hello world"

main[1] step
>
Step completed: "thread=main", TestClass.main(), line=19 bci=5
19        INVOKEVIRTUAL java/io/PrintStream.println (Ljava/lang/String;)V

main[1] step
> hello world

Step completed: "thread=main", TestClass.main(), line=20 bci=8
20        RETURN

main[1] step
>
The application exited

As said, this also works with IDEs when you put the two files into the class and source paths of a project. I just verified this with Eclipse: