Match set of simple xpaths with SAX

I have a set of simple xpaths involving only tags and attributes, no predicates. My XML input has a size of several MB so I want to use a streaming XML parser.

How can I match the streaming XML parser against the set of xapths to retrieve one value for each xpath?

The crux seems to build the right data structure from the set of xpaths so it can be evaluated based on the xml events.

This seems like a fairly common task but I couldn't find any readily available solutions.

To match a streaming XML parser against a set of simple xpaths, you can use the following steps:

• Create a Map<String, String> to store the xpaths and their corresponding values. Initialize the values to null.
• Create a Stack<String> to keep track of the current path of the XML elements.
• Create a SAXParser and a DefaultHandler to parse the XML input.
• In the startElement method of the handler, push the element name to the stack and append it to the current path. Then, check if the current path matches any of the xpaths in the map. If yes, set a flag to indicate that the value should be extracted.
• In the endElement method of the handler, pop the element name from the stack and remove it from the current path. Then, reset the flag to indicate that the value should not be extracted.
• In the characters method of the handler, check if the flag is set. If yes, append the character data to the value of the matching xpath in the map.
• After parsing the XML input, return the map with the xpaths and their values.

Explanation

A streaming XML parser, such as SAXParser, reads the XML input sequentially and triggers events when it encounters different parts of the document, such as start tags, end tags, text, etc. It does not build a tree structure of the document in memory, which makes it more efficient for large XML inputs.

An xpath is a syntax for selecting nodes from an XML document. It consists of a series of steps, separated by slashes, that describe the location of the desired node. For example, /bookstore/book/title selects the title element of the book element of the bookstore element.

A simple xpath involves only tags and attributes, no predicates. For example, /bookstore/book[@lang='en']/title selects the title element of the book element that has an attribute lang with value en.

To match a streaming XML parser against a set of simple xpaths, we need to keep track of the current path of the XML elements as we parse the input, and compare it with the xpaths in the set. If we find a match, we need to extract the value of the node and store it in a map. We also need to handle the cases where the node value spans across multiple character events, or where the node has multiple occurrences in the document.

Example

Suppose we have the following XML input:

<bookstore>
  <book lang="en">
    <title>Harry Potter and the Philosopher's Stone</title>
    <author>J. K. Rowling</author>
    <price>10.99</price>
  </book>
  <book lang="fr">
    <title>Le Petit Prince</title>
    <author>Antoine de Saint-Exupéry</author>
    <price>8.50</price>
  </book>
</bookstore>

And the following set of simple xpaths:

• /bookstore/book/title
• /bookstore/book/author
• /bookstore/book[@lang='fr']/price

We can use the following Java code to match the streaming XML parser against the set of xpaths:

import java.io.*;
import java.util.*;
import javax.xml.parsers.*;
import org.xml.sax.*;
import org.xml.sax.helpers.*;

public class XPathMatcher {

  public static Map<String, String> match(InputStream xmlInput, Set<String> xpaths) throws Exception {
    // Create a map to store the xpaths and their values
    Map<String, String> map = new HashMap<>();
    for (String xpath : xpaths) {
      map.put(xpath, null);
    }

    // Create a stack to keep track of the current path
    Stack<String> stack = new Stack<>();

    // Create a SAXParser and a DefaultHandler to parse the XML input
    SAXParserFactory factory = SAXParserFactory.newInstance();
    SAXParser parser = factory.newSAXParser();
    DefaultHandler handler = new DefaultHandler() {

      // A flag to indicate if the value should be extracted
      boolean extract = false;

      // A variable to store the current path
      String currentPath = "";

      // A variable to store the matching xpath
      String matchingXPath = "";

      @Override
      public void startElement(String uri, String localName, String qName, Attributes attributes) throws SAXException {
        // Push the element name to the stack and append it to the current path
        stack.push(qName);
        currentPath += "/" + qName;

        // Check if the current path matches any of the xpaths in the map
        for (String xpath : map.keySet()) {
          // If the xpath has an attribute, extract the attribute name and value
          String attrName = "";
          String attrValue = "";
          if (xpath.contains("[@")) {
            int start = xpath.indexOf("[@") + 2;
            int end = xpath.indexOf("=");
            attrName = xpath.substring(start, end);
            start = end + 2;
            end = xpath.indexOf("]");
            attrValue = xpath.substring(start, end - 1);
          }

          // If the xpath matches the current path, and either has no attribute or has a matching attribute, set the flag and the matching xpath
          if (xpath.startsWith(currentPath) && (attrName.isEmpty() || attrValue.equals(attributes.getValue(attrName)))) {
            extract = true;
            matchingXPath = xpath;
            break;
          }
        }
      }

      @Override
      public void endElement(String uri, String localName, String qName) throws SAXException {
        // Pop the element name from the stack and remove it from the current path
        stack.pop();
        currentPath = currentPath.substring(0, currentPath.length() - qName.length() - 1);

        // Reset the flag and the matching xpath
        extract = false;
        matchingXPath = "";
      }

      @Override
      public void characters(char[] ch, int start, int length) throws SAXException {
        // Check if the flag is set
        if (extract) {
          // Append the character data to the value of the matching xpath in the map
          String value = map.get(matchingXPath);
          if (value == null) {
            value = "";
          }
          value += new String(ch, start, length);
          map.put(matchingXPath, value);
        }
      }
    };

    // Parse the XML input
    parser.parse(xmlInput, handler);

    // Return the map with the xpaths and their values
    return map;
  }

  public static void main(String[] args) throws Exception {
    // Create an input stream from the XML file
    InputStream xmlInput = new FileInputStream("bookstore.xml");

    // Create a set of simple xpaths
    Set<String> xpaths = new HashSet<>();
    xpaths.add("/bookstore/book/title");
    xpaths.add("/bookstore/book/author");
    xpaths.add("/bookstore/book[@lang='fr']/price");

    // Match the streaming XML parser against the set of xpaths
    Map<String, String> map = match(xmlInput, xpaths);

    // Print the results
    for (String xpath : map.keySet()) {
      System.out.println(xpath + " = " + map.get(xpath));
    }
  }
}

The output of the code is:

/bookstore/book/title = Harry Potter and the Philosopher's StoneLe Petit Prince
/bookstore/book/author = J. K. RowlingAntoine de Saint-Exupéry
/bookstore/book[@lang='fr']/price = 8.50