I'm trying to use Python to extract data from Tesseract's hocr output file. We're limited to tesseact version 3.04, so no image_to_data function or tsv output is available. I have been able to do it with beautifulsoup and in R, but that's neither are available in the environment in which it needs to be deployed. I am just trying to extract the word and confidence "x_wconf." An example output file is below, for which I'd be happy to just return lists of [90, 87, 89, 89] and ['the', '(quick)', '[brown]', '{fox}', 'jumps!'].
lxml is the only available xml parser outside of the elementtree in the environment so I'm a bit at a loss for how to proceed.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title></title>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<meta name='ocr-system' content='tesseract 3.05.00dev' />
<meta name='ocr-capabilities' content='ocr_page ocr_carea ocr_par ocr_line ocrx_word'/>
</head>
<body>
<div class='ocr_page' id='page_1' title='image "./testing/eurotext.png"; bbox 0 0 1024 800; ppageno 0'>
<div class='ocr_carea' id='block_1_1' title="bbox 98 66 918 661">
<p class='ocr_par' id='par_1_1' lang='eng' title="bbox 98 66 918 661">
<span class='ocr_line' id='line_1_1' title="bbox 105 66 823 113; baseline 0.015 -18; x_size 39; x_descenders 7; x_ascenders 9"><span class='ocrx_word' id='word_1_1' title='bbox 105 66 178 97; x_wconf 90'>The</span> <span class='ocrx_word' id='word_1_2' title='bbox 205 67 347 106; x_wconf 87'><strong>(quick)</strong></span> <span class='ocrx_word' id='word_1_3' title='bbox 376 69 528 109; x_wconf 89'>[brown]</span> <span class='ocrx_word' id='word_1_4' title='bbox 559 71 663 110; x_wconf 89'>{fox}</span> <span class='ocrx_word' id='word_1_5' title='bbox 687 73 823 113; x_wconf 89'>jumps!</span>
</span>
</p>
</div>
</div>
</body>
</html>
Figured out a (gross) way to do it using xpath.
def hocr_to_dataframe(fp):
from lxml import etree
import pandas as pd
import os
doc = etree.parse('fp')
words = []
wordConf = []
for path in doc.xpath('//*'):
if 'ocrx_word' in path.values():
conf = [x for x in path.values() if 'x_wconf' in x][0]
wordConf.append(int(conf.split('x_wconf ')[1]))
words.append(path.text)
dfReturn = pd.DataFrame({'word' : words,
'confidence' : wordConf})
return(dfReturn)