I am using Ruby::Parslet.
I am parsing a document similar to an SV interface, eg:
interface my_intf;
protocol validonly;
transmit [Bool] valid;
transmit [Bool] pipeid;
transmit [5:0] incr;
transmit [Bool] sample;
endinterface
Here is my parser:
class myParse < Parslet::Parser
rule(:lparen) { space? >> str('(') >> space? }
rule(:rparen) { space? >> str(')') >> space? }
rule(:lbox) { space? >> str('[') >> space? }
rule(:rbox) { space? >> str(']') >> space? }
rule(:lcurly) { space? >> str('{') >> space? }
rule(:rcurly) { space? >> str('}') >> space? }
rule(:comma) { space? >> str(',') >> space? }
rule(:semicolon) { space? >> str(';') >> space? }
rule(:eof) { any.absent? }
rule(:space) { match["\t\s"] }
rule(:whitespace) { space.repeat }
rule(:space?) { whitespace.maybe }
rule(:blank_line) { space? >> newline.repeat(1) }
rule(:newline) { str("\n") }
# Things
rule(:integer) { space? >> match('[0-9]').repeat(1).as(:int) >> space? }
rule(:identifier) { match['a-z'].repeat(1) }
rule(:intf_start) { space? >> str('interface') >> space? >> (match['a-zA-Z_'].repeat(1,1) >> match['[:alnum:]_'].repeat(0)).as(:intf_name) >> space? >> str(';') >> space? >> str("\n") }
rule(:protocol) { space? >> str('protocol') >> whitespace >> (str('validonly').maybe).as(:protocol) >> space? >> str(';') >> space? >> str("\n") }
rule(:bool) { lbox >> space? >> str('Bool').as(:bool) >> space? >> rbox }
rule(:transmit_width) { lbox >> space? >> match('[0-9]').repeat.as(:msb) >> space? >> str(':') >> space? >> match('[0-9]').repeat.as(:lsb) >> space? >> rbox }
rule(:transmit) { space? >> str('transmit') >> whitespace >> (bool | transmit_width) >> whitespace >> (match['a-zA-Z_'].repeat(1,1) >> match['[:alnum:]_'].repeat(0)).as(:transmit_name) >> space? >> str(';') >> space? >> str("\n") }
rule(:interface_body) { (protocol | blank_line.maybe) }
rule(:interface) { intf_start >> interface_body }
rule(:expression) { ( interface ).repeat }
root :expression
end
I am having an issue making the rule for interface_body.
It can have 0 or more transmit lines and 0 or 1 protocol line and multiple blanks, comments etc.
Can someone help me out please? The rules I have written in the code snippet works with single transmit and single protocol, i.e. they properly match, but when I parse a whole interface it does not work.
Thanks in advance.
Ok... this parses the file you mentioned. I don't understand the desired format so I can't say it will work for all your files, but hopefully this will get you started.
require 'parslet'
class MyParse < Parslet::Parser
rule(:lparen) { space? >> str('(') }
rule(:rparen) { space? >> str(')') }
rule(:lbox) { space? >> str('[') }
rule(:rbox) { space? >> str(']') }
rule(:lcurly) { space? >> str('{') }
rule(:rcurly) { space? >> str('}') }
rule(:comma) { space? >> str(',') }
rule(:semicolon) { space? >> str(';') }
rule(:eof) { any.absent? }
rule(:space) { match["\t\s"] }
rule(:whitespace) { space.repeat(1) }
rule(:space?) { space.repeat(0) }
rule(:blank_line) { space? >> newline.repeat(1) }
rule(:newline) { str("\n") }
# Things
rule(:integer) { space? >> match('[0-9]').repeat(1).as(:int) >> space? }
rule(:identifier) { match['a-z'].repeat(1) }
def line( expression )
space? >>
expression >>
space? >>
str(';') >>
space? >>
str("\n")
end
rule(:expression?) { ( interface ).repeat(0) }
rule(:interface) { intf_start >> interface_body.repeat(0) >> intf_end }
rule(:interface_body) {
intf_end.absent? >>
interface_bodyline >>
blank_line.repeat(0)
}
rule(:intf_start) {
line (
str('interface') >>
space? >>
( match['a-zA-Z_'].repeat(1,1) >>
match['[:alnum:]_'].repeat(0)).as(:intf_name)
)
}
rule(:interface_bodyline) {
line ( protocol | transmit )
}
rule(:protocol) {
str('protocol') >> whitespace >>
(str('validonly').maybe).as(:protocol)
}
rule(:transmit) {
str('transmit') >> whitespace >>
(bool | transmit_width) >> whitespace >>
name.as(:transmit_name)
}
rule(:name) {
match('[a-zA-Z_]') >>
(match['[:alnum:]'] | str("_")).repeat(0)
}
rule(:bool) { lbox >> str('Bool').as(:bool) >> rbox }
rule(:transmit_width) {
lbox >>
space? >>
match('[0-9]').repeat(1).as(:msb) >>
space? >>
str(':') >>
space? >>
match('[0-9]').repeat(1).as(:lsb) >>
space? >>
rbox
}
rule(:intf_end) { str('endinterface') }
root :expression?
end
require 'rspec'
require 'parslet/rig/rspec'
RSpec.describe MyParse do
let(:parser) { MyParse.new }
context "simple_rule" do
it "should consume protocol line" do
expect(parser.interface_bodyline).to parse(' protocol validonly;
')
end
it 'name' do
expect(parser.name).to parse('valid')
end
it "bool" do
expect(parser.bool).to parse('[Bool]')
end
it "transmit line" do
expect(parser.transmit).to parse('transmit [Bool] valid')
end
it "transmit as bodyline'" do
expect(parser.interface_bodyline).to parse(' transmit [Bool] valid;
')
end
end
end
RSpec::Core::Runner.run(['--format', 'documentation'])
begin
doc = File.read("test.txt")
MyParse.new.parse(doc)
rescue Parslet::ParseFailed => error
puts error.cause.ascii_tree
end
The main changes...
Don't consume whitespace both side of your tokens. You had expressions that parsed "[Bool] valid" as LBOX BOOL RBOX SPACE? then expected another WHITESPACE but couldn't find one (as the previous rule had consumed it).
When an expression can validly parse as a zero length (e.g. something with repeat(0)) and there is a problem with who it's written, then you get an odd error. The rule pass and match nothing, then the next rule will typically fail. I explicitly matched 'body lines' as 'not the end line' so it would fail with an error.
'repeat' defaults to (0) which I would love to change. I see mistakes around this all the time.
x.repeat(1,1) means make one match. That's the same as having x. :)
there were more whitespace problems
so....
Write your parser from the top down. Write tests from the bottom up. When your tests get to the top you are done! :)
Good luck.