rubyparsletflexlm

Ruby & Lmstat : parslet and structured multi-line block : where to put the newline statement?


I have a Flexlm/Flexnet licenses service and I want parse the outputs of this service. All output are structured block of multi-lines. My first step is to parse the output of lmutil lmstat -c <port@server> -a to have the usage of licences and increments.

I try to use Ruby and Parslet. All lines are individually well parsed. I have a rule to parse a repetition of a specific type of line, but I can't parse a structured block of lines.

I'm looking for the law (best word in this context than 'rule') that define where to put the 'newline' statement in a multi-line structured block.

I working with Debian Jessie (stable/x86_64) and Ruby 2.1.5p273 and Parslet 1.6.1-1.

I've contacted the author, he is sorry but he haven't enough time to help me. The webpages seen are :

I passed many hours to try to understand how to construct the rules of a multi-line structured block. Below you have my source code with all test strings and the output.

My approach is to build:

  1. the elementary rule to parse a fragment of line
  2. the rule to parse a complete line without the 'newline' statement;
  3. a rule to parse the repetition of an information of the same type, like the line for the used tokens;
  4. the rule to parse a group of data : header + repetition of lines;
  5. the rule to parse a repetition of groups.

I'm not sure of the point 3, and I'm completely lost with '4' and '5'.

Thanks in advance for any help. [ 07/14/2017 : some parts of code was removed ]

#!/usr/bin/env ruby
# This code try to parse the output of 'lmutil lmstat -c <port@server> -a'.
require 'parslet'
require 'parslet/convenience'
require 'pp'

### Begin of the class Lmstat
class Lmstat < Parslet::Parser

 ###
 # Small parts to parse
 rule(:digit)       { match(/\d/).repeat(1) }
 rule(:space)       { str(' ').repeat }
 rule(:eof)         { any.absent? }
 rule(:blank_line)  { space.maybe >> newline >> space.maybe }
 rule(:newline)     { str("\r").maybe >> str("\n") }
 rule(:txt)         { match(/[\w\d\s,_.'",-:]/).repeat }

 def parenthese( atom, qte='()' )
   if (qte == '()' )
    str('(') >> atom >> str(')')
   else
    str(qte) >> atom >> str(qte)
   end
 end
 ###


 ###
 # The header is not parsed for the moment, while I can't
 rule (:header) do
   # Not define until the other parts are OK.
 end


 rule(:feature_line) do
   feature_usage.as(:feature_line) >> # newline >>
   feature_line_id.as(:feature_line_id).repeat.as(:f_line)
 end

 rule(:feature_line_id) do
   feature_version >> newline >> feature_type >>  newline >>
   feature_user_group >> newline
 end

 rule(:feature_line_id_group) do
   (newline >> feature_line_id).repeat(1).as(:f_line_group) >> newline
 end


 rule(:feature_usage) do
   str("Users of ") >> feature.as(:feature_usage) >> str(':') >> space >>
   parenthese( feature_used ) >> space.maybe
 end

 rule(:feature) { match(/[\w_-]/).repeat }

 # Total of 1 license issued;  Total of 0 licenses in use
 rule(:feature_used) do
   feature_token.as(:feature_token_issued) >>
   feature_token.as(:feature_token_used) >> space.maybe >> newline.maybe
 end

 # (Total of 1 license issued;  Total of 0 licenses in use)
 rule(:feature_token) do
   space.maybe >> str('Total of ') >> digit.repeat.as(:feature_token_value) >>
   space >> license >> issued_used >>
   str(';').maybe >> space.maybe
 end

 rule(:license) { str('license') >> str('s').maybe >> space }

 rule(:issued_used) do
   str('issued') | str('in use')
 end

 # v2015.1231
 rule(:version) { match(/[\w\d.-]/).repeat }

 # "incr-1"
 rule(:vendor) { match(/[\w-]/).repeat }

 #  "incr-1" v2015.1231, vendor: ansoftd
 rule(:feature_version) do
   # newline >>
   space.maybe >> parenthese( feature.as(:feature), '"' ) >>
   space >> version.as(:version) >> str(', vendor: ') >>
   vendor.as(:vendor) >> space.maybe >>
   str(', expiry: ').maybe >> match(/[\w\d-]/).repeat.as(:expiration).maybe
 end

 # floating license
 # nodelocked license, locked to "ID=12345"
 rule(:feature_type) do
   space.maybe >>
     ( (space.maybe >> str("floating license").as(:floating) >> space.maybe) |
       (space.maybe >> str('nodelocked license, locked to "ID=') >>
       digit.as(:license_id) >> str('"') >> space.maybe)).as(:feature_type) >>
     space.maybe
 end

 # \t 28 RESERVATIONs for GROUP Better_Group (server/27000)
 rule(:reserve) do
   space.maybe >> str("\t").maybe >> digit.as(:reserve_value) >>
   str(" RESERVATION") >> str("s").maybe >> str(" for ") >>
   word.as(:reserve_type) >> space >> word.as(:reserve_who) >>
   space >>
   parenthese( host.as(:server) >>  str("/") >> digit.as(:port) )
 end

 rule(:reserve_group) do
   (newline >> reserve).repeat(1).as(:reservation)
 end

 rule(:feature_user) do
   space.maybe >>
   word.as(:login) >> space >> host.as(:host_user) >> space >> host.as(:id) >>
   space >> parenthese( version.as(:version) ) >> space >> port >> date_queue
 end

 rule(:feature_user_group) do
   (newline >> feature_user).repeat(1).as(:feature_user_group)
 end

 # queued for 1 license
 rule(:queue) do
  str('queued for ') >> digit.as(:queued) >> str(' license') >> str('s').maybe
 end

 rule(:date_queue) do
   ( ( str(',') >> space >> date >> cmt.as(:comment)) | (space >> queue) )
 end

 rule(:cmt) do
   space.maybe >> match(/[^\r\n]/).repeat#.as(:cmt)
 end

 rule(:word) { match(/[\w\d-]/).repeat }
 rule(:host) { match(/[\w\d_.-]/).repeat }

 rule(:port) do
   parenthese( host.as(:server) >> str('/') >> digit.as(:server_port) >>
               space >> digit.as(:vendor_port) )
 end

 rule(:date) do
   str('start ') >> word.as(:date_dayname) >> space >>
   digit.as(:date_month) >> str('/') >> digit.as(:date_day) >> space >>
   digit.as(:date_hour) >> str(':') >> digit.as(:date_minute)
 end

end
### End of the class Lmstat

###
# Some multiline tests case.

t_feature_line_id = %q{ "incr-2" v9999.9999, vendor: vendor-daemon
    floating license

    henry abc057 abc057 (v2015.0623) (shoe/28512 3886) queued for 1 license
    simon abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:37, 10 licenses
    simon abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:37 queued for 1 license
}


t_feature_line_id_group = %q{ "incr-2" v9999.9999, vendor: vendor-daemon
    floating license

    henry abc057 abc057 (v2015.0623) (shoe/28512 3886) queued for 1 license
    jason abc057 abc057 (v2015.0623) (shoe/28512 3886), start Fri 11/20 14:41
    simon abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:37, 10 licenses
    simon abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:37 queued for 1 license

   "inc2" v9999.9999, vendor: inc2vendor
   floating license

    simon abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:37, 10 licenses
    simon abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:37 queued for 1 license }


t_feature_line = %q{Users of ansys:  (Total of 9 licenses issued;  Total of 6 licenses in use)

  "incr-2" v9999.9999, vendor: vendor-daemon
  floating license

    jason abc057 abc057 (v2015.0623) (shoe/28512 3886), start Fri 11/20 14:41
    simon abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:37, 10 licenses
    simon abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:3}

t_feature_line_group = %q{
  "incr-2" v9999.9999, vendor: vendor-daemon
  floating license

    jason abc057 abc057 (v2015.0623) (shoe/28512 3886), start Fri 11/20 14:41
    simon abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:37, 10 licenses
    simon abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:37 queued for 1 license

  "incr-2" v9999.9999, vendor: vendor-daemon
  floating license

    jason abc057 abc057 (v2015.0623) (shoe/28512 3886), start Fri 11/20 14:41
    simon abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:37, 10 licenses
    simon abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:37}

t_feature_user= %q{jason abc057 abc057 (v2015.0623) (shoe/28512 3886), start Fri 11/20 14:41}

t_feature_group = %q{ "incr-2" v9999.9999, vendor: vendor-daemon
  floating license

    jason abc057 abc057 (v2015.0623) (shoe/28512 3886), start Fri 11/20 14:41
    simon abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:37, 10 licenses
    jessica abc057 abc057 (v2014.1110) (shoe/28512 4166), start Fri 11/20 15:37

  "MATLAB" v35, vendor: MLM, expiry: 01-jan-0000
  nodelocked license, locked to "ID=12345"

    albert node7563 node7563 (v34) (shoe/27000 201), start Mon 5/23 6:16 (linger: 1235700)
    victoria abc087 /dev/pts/1 (v29) (shoe/27000 3401), start Mon 5/23 6:30}
###

###
# Method to test the parsing.
def parse_method(method,str)
  lmstat = Lmstat.new
  unless lmstat.respond_to?(method)
    raise ArgumentError,
          "\n\n\t*****   ERROR: Unknown method -> '#{method}'  ******\n\n",
          caller[1..-1]
  end
  begin
    m = "lmstat.#{method}.parse('"+ str + '\')'
    puts "=> Test of #{m}"
    eval (m)
  rescue Parslet::ParseFailed => failure
    puts failure.cause.ascii_tree
  end
end
###

###
# Not called if 'irb' is used to load the program.
if __FILE__ == $PROGRAM_NAME
    puts "\n ###### Multilines #####"
      parse_method('feature_user_group',t_feature_user_group)
      parse_method("feature_line_id",t_feature_line_id)
      pp parse_method("feature_line_id_group",t_feature_line_id_group)
end

The output [ 07/13/2017 : Removed to put the functional version ]

[ UPDATE - 04/29/2017 - Problem solved] Thanks Nigel Thorne for your answer, it's solve my problem. I've corrected the rule for 'space' following your advice.

[ 07/13/2014 : Remove some text to put a fully functional version. ]

[ UPDATE - 07/13/2017 - Application to test the parsing]

I've finished an application to test the parsing of the lmstat's outputs with Ruby and Parslet. As the parsing is depending of each editor, some cases could be not covered, but more than 30 license services are used to validate the parsing.

I could give 3 files :

  1. parse_lmstat.rb : Use './parse_lmstat --help' to have the help. The application to test the parsing.
  2. readstdin_lmstat.rb : Read from STDIN the parsed lmstat's output generated with the YAML format.
  3. display_lmstat.rb : Show how to have access at the data and it was used to improve the structure of the parsing. The script is simply better than an irb session. It read from STDIN a parsed lmstat's output with the YAML format.

An example :

    ~/bin/lmutil lmstat -a -c 1234@licserver | ./parse_lmstat.rb --screen | ./display_lmstat.rb

One bug known : when a [CTRL-C] is done with the signal seems not to be well trapped, Ruby send some error messages in some cases.

Now, I dream to have a small WEB application (SINATRA ?) to select the license server and show the data, but I don't speak either HTML or CSS ... Any help will be appreciate ;-)

You'll find below only the classes to parse and transform the lmstat's output, because of the limit to 30000 characters.

#!/usr/bin/env ruby
#
# class_lmstat.rb
#
# This code try to parse the output of 'lmutil lmstat -c <port@server> -a'.
#
# Scapin - 11/07/2017
#
# For the Stackoverflow forums
#
require 'parslet'
require 'parslet/convenience'
require 'open3'

### Begin of the class Lmstat
class Lmstat < Parslet::Parser

 ###
 # Small parts to parse
 rule(:digit)       { match(/\d/).repeat(1) }
 rule(:space)       { str(' ').repeat(1) }
 rule(:eof)         { any.absent? }
 rule(:blank_line)  { space.maybe >> newline >> space.maybe }
 rule(:newline)     { str("\r").maybe >> str("\n") }
 rule(:txt)         { match(/[\w_.\)\('\t ",-:\\]/).repeat }
 rule(:word)        { match(/[\w-]/).repeat }
 rule(:host)        { match(/[\w_\.-]/).repeat }
 rule(:cnx_id)      { match(/[\/\w_.:]/).repeat }
 rule(:cmt)         { space.maybe >> match(/[^\r\n]/).repeat }
 rule(:error_code)  { match(/[,\d-]/).repeat }

 def parenthese( atom, qte='()' )
   if (qte == '()' )
    str('(') >> atom >> str(')')
   else
    str(qte) >> atom >> str(qte)
   end
 end
 ###

 root(:lmstat)

 rule(:lmstat) do
   (header.as(:header) >> body.repeat.as(:service) >> newline).as(:lmstat)
 end

 ###
 # The header is not parsed for the moment, while I can't
 # handle the multiline block correctly.
 #
 # lmutil - Copyright (c) 1989-2013 Flexera Software LLC. All Rights Reserved.
 # Flexible License Manager status on Fri 11/20/2015 16:39
 #
 # License server status: 1141@lic-server
 #     License file(s) on lic-server: /opt/license/soft/vendor1.lic:/opt/license/soft/vendor2.lic:
 #
 #     lic-server: license server UP (MASTER) v11.13
 #
 # Vendor daemon status (on lic-server):
 #
 #   vendor-daemon: UP v11.13
 # Feature usage info:
 #
 ###
 rule (:header) do
   copyright >> status_date >> newline >>
   server >> license_file >> newline >>
   server_status >> newline >>
   vendor_daemon_status >> newline
 end

 rule (:body) do
   (vendor_daemon.as(:vendor_daemon) >> feature_info.maybe >> newline >>
    feature_line.repeat.maybe.as(:features))
 end

 # lmutil - Copyright (c) 1989-2013 Flexera Software LLC. All Rights Reserved.
 rule (:copyright) do
   space.maybe >> (str("lmutil - Copyright ") >> match(/./).repeat).as(:copyright) >> newline
 end

 # Flexible License Manager status on Fri 11/20/2015 16:39
 rule(:status_date) do
   space.maybe >> str("Flexible License Manager status on ") >>
   word.as(:status_dayname) >> space >>
   digit.as(:status_month) >> str("/") >> digit.as(:status_day) >>
   str("/") >> digit.as(:status_year) >>
   str(" ") >> digit.as(:status_hour) >>
   str(":") >> digit.as(:status_min) >> newline
 end

 rule(:server) do
   str("License server status: ") >>
   digit.as(:server_port1) >> str("@") >> host.as(:server1) >>
   ( str(",") >> digit.as(:server_port2) >> str("@") >> host.as(:server2) >>
     str(",") >> digit.as(:server_port3) >> str("@") >> host.as(:server3) ).maybe >>
   newline
 end

 #  License file(s) on lic-server: /opt/license/soft/licfile-1.lic:/opt/soft/licfile-2.lic:
 rule(:license_file) do
   space.maybe >> str("License file(s) on ") >>
   match(/[\w\d._-]/).repeat.as(:license_files_server) >>
   str(": ") >> txt.as(:license_files_names) >> newline
 end

 rule(:server_status) do
   (space.maybe >> host.as(:server_host) >>  str(": ") >>
    ( server_up | server_down) >> newline).repeat(0).as(:server_list)
 end

 rule(:server_up) do
   str("license server ")>>str("UP").as(:server_up)>>
   server_pos.maybe >> str(" ") >> cmt.as(:server_version)
 end

 rule(:server_pos) do
   space >> parenthese( match(/[A-Za-z]/).repeat.as(:server_role))
 end


 # licserver: Cannot connect to license server system. (-15,570:115 "Operation now in progress")
 rule(:server_down) do
   space.maybe >> str("Cannot connect to license server system").as(:server_down) >>
   str(". ") >> cmt.as(:server_error)
 end

 rule(:vendor_daemon_status) do
   str("Vendor daemon status (on ") >> host.as(:server_daemon) >>
   str("):") >> space.maybe >> newline
 end

 rule(:vendor_daemon) do
   ( vendor_daemon_up |  vendor_daemon_down )
 end

 rule(:vendor_daemon_up) do
   space.maybe >> word.as(:daemon) >> str(": ") >> word.as(:daemon_status) >>
   space >> host.as(:daemon_version) >> newline
 end

 rule(:vendor_daemon_down_ini) do
   space.maybe >> word.as(:daemon) >> str(": The desired vendor daemon is down. ") >>
   parenthese( error_code.as(:daemon_status) ) >> space.maybe >> newline
 end

 # \n\n dconcept: No socket connection to license server manager. (-7,96)
 rule(:vendor_daemon_down) do
   space.maybe >> word.as(:daemon) >> str(": The desired vendor daemon is down. ") >>
   parenthese( error_code.as(:daemon_status) ) >> space.maybe >> newline
   space.maybe >> word.as(:vendor_daemon_down_msg_feature).maybe >>
   str(': No socket connection to license server manager.').maybe >> space.maybe >>
   cmt.as(:vendor_daemon_down_msg).maybe >> newline.maybe
 end

 rule(:feature_info) do
   space.maybe >> str("Feature usage info:") >> space.maybe >> newline
 end


 ###
 # Users of soft_a:  (Total of 1 license issued;  Total of 0 licenses in use)
 #
 #  "incr-1" v2015.1231, vendor: soft_ad
 #  floating license
 #
 #  28 RESERVATIONs for GROUP Better_Group (server/27000)
 #  1 RESERVATION for USER toni (server/27000)
 #   scott abc056 abc056 (v2015.0623) (shoe/28512 3644), start Fri 11/20 15:45, 2 licenses
 #   scott abc056 abc056 (v2015.0623) (shoe/28512 4669), start Fri 11/20 15:45, 10 licenses
 rule(:feature_line) do
   feature_usage.as(:feature_line) >> newline >>
   feature_line_id.repeat(0).as(:feature_line_id)
 end

 #  "incr-1" v2015.1231, vendor: soft_ad
 #  floating license
 #
 #   scott abc056 abc056 (v2015.0623) (shoe/28512 3644), start Fri 11/20 15:45, 2 licenses
 #   scott abc056 abc056 (v2015.0623) (shoe/28512 4669), start Fri 11/20 15:45, 10 licenses
 rule(:feature_line_id) do
   feature_version >> newline >> feature_type >>  newline >>
   # ( reserve.as(:reservation) | feature_user.as(:user)).repeat(1).as(:users) >> newline
   ( reserve.as(:reservation) | feature_user.as(:user)).repeat(1).as(:who) >> newline
 end

 # Users of soft_a:  (Total of 1 license issued;  Total of 0 licenses in use)
 # Users of SOFT_B:  (Uncounted, node-locked)
 # Users of soft_c:  (Error: 6 licenses, unsupported by licensed server)
 rule(:feature_usage) do
   str("Users of ") >> feature.as(:feature_name) >> str(':') >> space >>
   parenthese( feature_used ) >> space.maybe >> newline
 end


 # Total of 1 license issued;  Total of 0 licenses in use
 # Uncounted, node-locked
 rule(:feature_used) do
   ( ( feature_token.as(:feature_token_issued) >> feature_token.as(:feature_token_used)) |
     ( word.as(:feature_token_issued) >> str(', ') >> word.as(:feature_token_used) ) |
     ( str('Error: ') >> digit.repeat.as(:feature_token_error) >> space >> str( 'license') >>
       str('s').maybe >> str(', ') >>
       match(/[\w_', :-]/).repeat.as(:feature_token_error_cause) ) ) >>
   space.maybe >> newline.maybe
 end

 # (Total of 1 license issued;  Total of 0 licenses in use)
 rule(:feature_token) do
   space.maybe >> str('Total of ') >> digit.repeat.as(:feature_token_value) >>
   space >> license >> issued_used >>
   str(';').maybe >> space.maybe
 end

 rule(:license) { str('license') >> str('s').maybe >> space }

 rule(:issued_used) do
   str('issued') | str('in use')
 end

 # v2015.1231
 rule(:version) { match(/[\w\d.-]/).repeat }

 # "incr-1"
 rule(:vendor) { match(/[\w-]/).repeat }

 rule(:feature) { match(/[\w\d\/_+-]/).repeat }

 #  "incr-1" v2015.1231, vendor: soft_ad
 rule(:feature_version) do
   # newline >>
   space >> parenthese( feature.as(:feature), '"' ) >>
   space >> version.as(:version) >> str(', vendor: ') >>
   vendor.as(:vendor) >> space.maybe >>
   str(', expiry: ').maybe >> match(/[\w\d-]/).repeat.as(:expiration).maybe
 end

 rule(:feature_type) do
   space >> ( float_type | node_type ).as(:feature_type)
 end

 # floating license
 rule(:float_type) do
   str("floating license").as(:floating) >> cmt.maybe >> newline
 end

 # nodelocked license, locked to "ID=654321"
 # nodelocked license locked to NOTHING (hostid=ANY)
 # uncounted nodelocked license locked to NOTHING (hostid=ANY)
 # uncounted nodelocked license, locked to Vendor-defined "PTC_HOSTID=01-0A-01-0A-01"
 rule(:node_type) do
   str('uncounted ').maybe >> str("nodelocked license") >> str(',').maybe >> str(' locked to ').maybe >>
   ( ( str('"ID=') >> digit.as(:nodelocked_id) >> str('"') ) |
     ( host.as(:nodelocked_to) >> space >> parenthese(str('hostid=') >> host.as(:nodelocked_hostid)) ) |
     ( host.as(:nodelocked_to) >> space >>
                                  parenthese(match(/[\w:_=' -]/).repeat.as(:nodelocked_hostid), '"') ) ) >>
   space.maybe >> newline
 end

 # \t 28 RESERVATIONs for GROUP Better_Group (server/27000)
 rule(:reserve) do
   space.maybe >> str("\t").maybe >> digit.as(:reserve_value) >>
   str(" RESERVATION") >> str("s").maybe >> str(" for ") >>
   word.as(:reserve_type) >> space >> word.as(:reserve_who) >>
   space >>
   parenthese( host.as(:server) >>  str("/") >> digit.as(:port) ) >>
   newline
 end

 rule(:feature_user) do
   (u_std | u_aselta | u_ans | u_c1 | u_c2 )
 end

 # scott abc056 abc056 (v2015.0623) (shoe/28512 3644), start Fri 11/20 15:45, 2 licenses
 # albert node7563 node7563 (v34) (shoe/27000 201), start Mon 5/23 6:16 (linger: 1235700)
 # hector node088 dev/tty (v2015.0312) (licserver/1446 3730), start Thu 11/19 9:08
 # will pim.my.domain.org pim.my.domain.org 6656 (v2016.1129) (licserver/1446 2216), start Fri 5/12 14:51
 rule(:u_std) do
   space >> word.as(:login) >> space >> host.as(:host_user) >> space >>
   cnx_id.as(:host_id) >>
   space >> parenthese( version.as(:version) ) >> space >> port >> date_queue >>
   newline
 end

 # scott cat :0 Token Lic (v7.000) (shoe/5300 15434), start Thu 7/6 17:07
 rule(:u_c1) do
   space >> word.as(:login) >> space >> host.as(:host_user) >> space >>
   cnx_id.as(:host_id) >> space >> match(/[^(]/).repeat.as(:common_name) >>
   parenthese( version.as(:version) ) >> space >> port >> date_queue >>
   newline
 end

 # jessie cat bird:1144.0 APS Multi-core (Max. 16 cores) (v11.100) (licserver/5303 13188), start Thu 7/6 15:22, 4 licenses
 rule(:u_c2) do
   space >> word.as(:login) >> space >> host.as(:host_user) >> space >>
   cnx_id.as(:host_id) >> space >> match(/[^(]/).repeat.as(:common_name) >>
   str('(') >> match(/[^)]/).repeat.as(:common_info) >> str(')')  >> space >>
   parenthese( version.as(:version) ) >> space >> port >> date_queue >>
   newline
 end

 # tiger pam.my.domain.org pam.my.domain.org 6656 (v2016.1129) (licserver/1446 2216), start Fri 5/12 14:51
 rule(:u_ans) do
   space >> word.as(:login) >> space >> host.as(:host_user) >> space >>
   cnx_id.as(:host_id) >>  ( space >> host.as(:further) ).maybe >>
   space >> parenthese( version.as(:version) ) >> space >> port >> date_queue >>
   newline
 end


 # clark node07 SOMETHING Inscale / grid (worker) (v1.0) (licserv01/27016 5506), start Fri 4/28 13:42, 4 licenses
 # bunny orca SOMETHING Inscale / graphical (v1.0) (licserv01/27016 650), start Thu 4/13 10:27
 rule(:u_aselta) do
   space >> word.as(:login) >> space >> host.as(:host_user) >> space >>
   word.as(:daemon)  >> space >> word.as(:soft) >> str(" / ") >>
   word.as(:function) >> (space >> parenthese( word.as(:tools) )).maybe  >>
   space >> parenthese( version.as(:version) ) >> space >> port >> date_queue >>
   newline
 end

 # queued for 1 license
 rule(:queue) do
   str('queued for ') >> digit.as(:queued) >> str(' license') >> str('s').maybe
 end

 rule(:lic) do
   str(',') >> space >> digit.as(:licenses) >> str(' license') >> str('s').maybe
 end

 rule(:date_queue) do
   ( ( str(',') >> space >> date >> ( lic | cmt.as(:comment))) | (space >> queue) )
 end

 rule(:port) do
   parenthese( host.as(:server) >> str('/') >> digit.as(:server_port) >>
               space >> digit.as(:vendor_port) )
 end

 rule(:date) do
   str('start ') >> word.as(:date_dayname) >> space >>
   digit.as(:date_month) >> str('/') >> digit.as(:date_day) >> space >>
   digit.as(:date_hour) >> str(':') >> digit.as(:date_minute)
 end

end
### End of the class Lmstat


### Begin of the class Trans 
class Trans < Parslet::Transform
  rule(:feature_token_value => simple(:v)) { Integer(v) }

  rule(:user => subtree(:t)) do
    if ( t.has_key?(:date_month) )
      cal = { "Sun"=>"Dimanche", "Mon"=>"Lundi", "Tue"=>"Mardi",
              "Wed"=>"Mercredi", "Thu"=>"Jeudi", "Fri"=>"Vendredi"}
      clock = Time.now

      # Addition of keys
      t.merge!( { :date_year => 0, :since => "", :delay_min => 0, :delay_string => ""})

      # Convert to integer
      t[:date_minute] = t[:date_minute].to_s.sub(/^0/,"")  if (t.has_key?(:date_minute))
      t.each do |k,v|
        [ :server_port, :vendor_port, :date_month, :date_day,
          :date_hour, :date_minute, :queued, :licenses ].each do |symbol|
          t[k] = Integer(v)  if k == symbol
        end
      end

      t[:date_dayname] = cal[t[:date_dayname].to_s]
      t[:date_year] = clock.year
      t[:date_year] = t[:date_year] - 1 if (clock.month < t[:date_month])
      t[:since] = sprintf( "%2.2d/%2.2d/%2d-%2.2d:%2.2d", t[:date_day], t[:date_month],
                           t[:date_year], t[:date_hour], t[:date_minute])
      t[:delay_min], t[:delay_string] = Tools.delay( clock, t[:date_year],
                 t[:date_month], t[:date_day], t[:date_hour], t[:date_minute], 0 )
      t[:delay_min] = Integer(t[:delay_min] / 60)
      t[:delay_string].chop!.chop!.chop!

      # Add a key for a borrowed token.
      t.merge!( {:borrow => true} ) if ( /linger/ =~ t[:comment] )
    end

    # Restore the hash.
    { :user => t }
  end

end
###


####
module Tools

  def Tools.check_file( file )
    return false unless file
    if File.exist?(file)
      File.file?(file)
    else
      false
    end
  end


  def Tools.delay( clock = Time.now, year, month, day, hour, minute, second )
    delay = clock - Time.local(year.to_i,month.to_i,day.to_i,hour.to_i,minute.to_i, second.to_i)
    d = delay.divmod(3600.0*24.0)
    h = d[1].divmod(3600.0)
    m = h[1].divmod(3600.0)[1].divmod(60.0)
    s = m[1].divmod(60.0)[1].divmod(60.0)
    [ delay, sprintf("%3.3dj%2.2dh%2.2dmin%2.2ds", d[0], h[0], m[0], s[1].round) ]
  end


  def Tools.grab_list( list_file, separator = ' ' )
    return nil unless Tools.check_file(list_file) && File.stat(list_file).readable?
    lines = Array.new
    list = Array.new
    open(list_file).each_line { |l| lines << l.chomp if l }
    # 'split' ignore the multiple '/\s/'.
    lines.each { |l| list.concat(l.split(separator)) }
    # Suppress the spaces if the separator isn't a "\s".
    list.each_index { |i| list[i]= list[i].delete(" ") } unless
    list.delete_if { |l| l.length < 1 }
    list
  end

  def Tools.create_output( name_of_file, extension = '', mode = "w" )
    begin
      file_name =  name_of_file + extension
      line = __LINE__; File.new( file_name, mode )
    rescue Errno::EACCES => error_create
      STDERR.puts $PROGRAM_NAME + "(#{line})" +
        " ERREUR ! create_output(\"#{file_name}\")"
      STDERR.puts $PROGRAM_NAME + "(#{line})" +
        " ERREUR !   Message = '#{error_create.message}'"
      raise error_create
    end
  end
end
###

Solution

  • It seems you are having trouble with "newline"s.

    A good guideline is... * consume them at the end of a rule (as a terminating character) * if they are not symantically part of the token, then let the parent rule consume them.

    Say I had a document:

    A
    B
    
    C
    

    I would parse this as:

    #tokens
    rule :a do str("A") end
    rule :b do str("B") end
    rule :c do str("C") end
    rule :nl do str("\n") end
    
    # lines
    rule :a_line do a>>nl end
    rule :b_line do b>>nl end
    rule :c_line do c>>nl end
    
    # doc
    rule :doc do a_line>>b_line>>nl>>c_line
    

    Note that "b_line" doesn't consume both "\n"s as it should know nothing about it's context.

    I also noticed you have "space" defined as "str(' ').repeat". This is short for "str(' ').repeat(0)" which can match zero times. This makes "space" optional... therefore "space.maybe" doesn't make sense.