pythonipythonlanguage-extension

How to parse a domain-specific language on top of Python? What architecture with IPython?


I want to construct a domain-specific language as a superset of Python. Cryptic commands like

f7:10y=x^2

designed to minimize typing shall be parsed into plain Python

for k in range(7,10):
    f[k].set_y( expr='x^2' )

before being executed. Probably, the command-line interface shall be IPython.

What would be an appropriate architecture: Shall I implement the cryptic-to-plain-Python translation in the IPython command-line shell or in its kernel daemon? Are there helpful libraries / tutorials / examples?

Or more generically: Are there examples how to add complex syntactic sugar to Python?


Solution

  • A solution that will work everywhere is not really an option here. What you are trying to achieve is modifying the core of the Python parser.

    The options are:

    1. Use a preparser. Within IPython there are hooks available for this: http://ipython.org/ipython-doc/dev/config/inputtransforms.html

      Here's an example for your use case:

      import re
      
      range_search = re.compile('''
          (?P<variable>.+)
          (?P<start>\d+):(?P<stop>\d+)
          (?P<k>[^=]+)=(?P<v>.+)
      ''', re.VERBOSE)
      
      range_replace = '''for k in range(\g<start>, \g<stop>):
          \g<variable>[k].set_\g<k>(expr='\g<v>')'''
      
      print range_search.sub(range_replace, 'f7:10y=x^2')
      
      @StatelessInputTransformer.wrap
      def inline_loop(line):
          return range_search.sub(range_replace, line)
      
    2. Recompile Python with your modifications

    3. Use a language which is more flexible in this regard like ruby