Greetings. I have been looking at Literate Programming a bit now, and I do like the idea behind it: you basically write a little paper about your code and write down as much of the design decisions, the code probably surrounding the module, the inner workins of the module, assumptions and conclusions resulting from the design decisions, potential extension, all this can be written down in a nice way using tex. Granted, the first point: it is documentation. It must be kept up-to-date, but that should not be that bad, because your change should have a justification and you can write that down.
However, how does Literate Programming Scale to a larger degree? Overall, Literate Programming is still just text. Very human readable text, of course, but still text, and thus, it is hard to follow large systems. For example, I reworked large parts of my compiler to use >> and some magic to chain compile steps together, because some "x.register_follower(y); y.register_follower(z); y.register_follower(a);..." got really unwieldy, and changing that to x >> y >> z >> a made it a bit better, even though this is at its breaking point, too.
So, how does Literate Programming scale to larger systems? Does anyone try to do that?
My thought would be to use LP to specify components that communicate with each other using event streams and chain all of these together using a subset of graphviz. This would be a fairly natural extension to LP, as you can extract a documentation -- a dataflow diagram -- from the net and also generate code from it really well. What do you think of it?
-- Tetha.
Excellent question. The motivation for literate programming will never go away, but I think it should be treated as fluid. It means "give the reader a break, and educate them to what you're trying to do". I don't think it means "make your code really wordy".
That said, the reader will have to put some effort into it, depending on what they already know. Presumably the code is worth understanding, and nothing comes for free.
I also think it means more than just making readable code. Most likely the reason someone is reading the code is because they need to make a change. You should anticipate the possible changes that might be needed, and tell them how to do it if necessary.