regexdevopscheckpointpython-textfsm

Is there a way to record multiple lines at once with TextFSM?


I want to parse a Checkpoint Firewall cphaprob -a if executed via Netmiko using TextFSM. The final generated list is not well formatted.

I already tried a lot of TextFSM combination of commands but maybe I just fail to understand how it properly works.

Original command output

Below is the cphaprob -a if original output. I want to parse the virtual context (e.g 'vcont 0'), interface names (e.g 'bond0'), virtual interfaces (e.g 'bond0.2121') and their hostnames (e.g '10.105.0.42').

vcont 0:
------
Required interfaces: 2
Required secured interfaces: 1

eth0       UP                    non sync(non secured), multicast
eth1       UP                    sync(secured), broadcast

Virtual cluster interfaces: 1

eth0            10.105.0.42        


vcont 1:
------
Required interfaces: 3
Required secured interfaces: 1

eth1       UP                    sync(secured), broadcast
bond0      UP                    non sync(non secured), multicast, bond Load Sharing  (bond0.2101)
bond1      UP                    non sync(non secured), multicast, bond Load Sharing  (bond1.2126)

Virtual cluster interfaces: 3

bond0.2121      10.65.29.21         
bond1.2122      10.65.29.22        
bond1.2123      10.65.29.23        


vcont 2:
------
Required interfaces: 3
Required secured interfaces: 1

eth1       UP                    sync(secured), broadcast
bond1      UP                    non sync(non secured), multicast, bond Load Sharing  (bond1.2127)
bond0      UP                    non sync(non secured), multicast, bond Load Sharing  (bond0.2102)

Virtual cluster interfaces: 2

bond1.4242      10.65.29.42        
bond0.4243      10.65.29.43         

TextFSM template

# template for ```cphaprob -a if``` command.
Value Context (\S+\s\d+)
Value List Interface (\S+)
Value List VirtualInterface (\S+)
Value List IPv4 (\d{1,3}.\d{1,3}.\d{1,3}.\d{1,3})

Start
  ^${Context}:
  ^${Interface}.*(UP|DOWN|Disconnected)
  ^Virtual cluster interfaces: \d+ -> Cluster

Cluster
  ^${VirtualInterface}\s+${IPv4} -> Record Start

Expected results

$ python tests/test_checkpoint_functions.py 
[['vcont 0', ['eth0', 'eth1'], ['eth0'], ['10.105.0.42']],
 ['vcont 1', ['eth1', 'bond0', 'bond1'], ['bond0.2121', 'bond1.2122', 'bond1.2123'], ['10.65.29.21', '10.65.29.22', '10.65.29.23']],
 ['vcont 2', ['eth1', 'bond1', 'bond0'], ['bond1.4242', 'bond0.4243'], ['10.65.29.42', '10.65.29.43']]]

Actual results

$ python tests/test_checkpoint_functions.py 
[['vcont 0', ['eth0', 'eth1'], ['eth0'], ['10.105.0.42']],
 ['vcont 1', ['eth1', 'bond0', 'bond1'], ['bond0.2121'], ['10.65.29.21']],
 ['vcont 2', ['eth1', 'bond1', 'bond0'], ['bond1.4242'], ['10.65.29.42']]]

As you can see I only get the 1st occurrence of the virtual interfaces and their corresponding IP addresses. The reason may be that in my template in Cluster state I record right after ^${VirtualInterface}\s+${IPv4} -> Record Start. I just can't figure out how to get all virtual interfaces and IP addresses in their corresponding lists.


Solution

  • Value Context (\S+\s\d+)
    Value List Interface (\S+)
    Value List VirtualInterface (\S+)
    # Add escaping for "."
    Value List IPv4 (\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})
    
    Start
      ^${Context}:
      ^${Interface}.*(UP|DOWN|Disconnected)
      ^Virtual cluster interfaces: \d+ -> Cluster
    
    Cluster
      ^${VirtualInterface}\s+${IPv4}
      # The reason for the multiple VirtualInterface and IPv4 entries
      # is because you Record after each match.
      # Instead, you can Record after you have matched all entries.
      # I look for what I know will be the start of the next entry in table,
      # and use Continue.Record.
      # Continue will continue looking for matches for the current line that is being parsed,
      # but will move onto the next regex line, instead of starting back at the top of State.
      # Another thing about Continue, is that you cannot do that and have a State Change,
      # which is why I change to the Start state afterwards
      ^.+: -> Continue.Record
      ^${Context}: -> Start