I am currently working on writing my own language(shameless plug), which is centered around flexibility. I am trying to make almost any part of the language syntax exchangeable through things like extensions/plugins. While writing the whole thing, it has got me thinking. I am wondering how that sort of flexibility could affect the language.
I know that Lisp is often referred to as one of the most extensible languages due to its extensive macro system. I do understand that concept of macros, but I am yet to find a language that allows someone to change the way it is parsed. To my knowledge, almost every language has an extremely concrete syntax as defined by some long specification.
My question is how could having a flexible syntax affect the intuitiveness and usability of the language? I know the basic "people might be confused when the syntax changes" and "semantic analysis will be hard". Those are things that I am already starting to compensate for. I am looking for a more conceptual answer on the pros and cons of having a flexible syntax.
The topic of language design is still quite foreign to me, so I apologize if I am asking an obvious or otherwise stupid question!
Edit: I was just wanting to clarify the question I was asking. Where exactly does flexibility in a language's syntax stand, in terms of language theory? I don't really need examples or projects/languages with flexibility, I want to understand how it can affect the language's readability, functionality, and other things like that.
Thanks to SK-logic's answer for pointing me in the direction of Alan Blackwell. I sent him an email asking his stance on the matter, and he responded with an absolutely wonderful explanation. Here it is:
So the person who responded to your StackOverflow question, saying that flexible syntax could be useful for DSLs, is certainly correct. It actually used to be fairly common to use the C preprocessor to create alternative syntax (that would be turned into regular syntax in an initial compile phase). A lot of the early esolangs were built this way.
In practice, I think we would have to say that a lot of DSLs are implemented as libraries within regular programming languages, and that the library design is far more significant than the syntax. There may be more purpose for having variety in visual languages, but making customisable general purpose compilers for arbitrary graphical syntax is really hard - much worse than changing text syntax features.
There may well be interesting things that your design could enable, so I wouldn’t discourage experimentation. However, I think there is one reason why customisable syntax is not so common. This is related to the famous programmer’s editor EMACS. In EMACS, everything is customisable - all key bindings, and all editor functions. It’s fun to play with, and back in the day, many of us made our own personalised version that only we knew how to operate. But it turned out that it was a real hassle that everyone’s editor worked completely differently. You could never lean over and make suggestions on another person’s session, and teams always had to know who was logged in order to know whether the editor would work. So it turned out that, over the years, we all just started to use the default distribution and key bindings, which made things easier for everyone.
At this point in time, that is just about enough of an explanation that I was looking for. If anyone feels as though they have a better explanation or something to add, feel free to contact me.