Allow specification of arbitrary syntax trees from grammars

[kneasle]

As said in the README, this is a very difficult challenge. I believe that the closest project to this is tree-sitter, and to be pragmatic I think that (at least initially) Sapling should try to use tree sitter as much as possible for parsing (even though it is not ideal for our use case). This way Sapling can actually function as an editor whilst we potentially build a more niche parsing system.

[conartist6]

@kneasle there's a new parsing system you might consider, BABLR. It's built in a similar way to tree-sitter, but it's designed from the ground up to support your use case. For example, it makes a formal guarantee that print(parse(source)) === source

[kneasle]

@conartist6 - BABLR looks pretty cool, and something that I will keep an eye on for sure. My current feeling is that the best way to create a 'vim-like editor with cool AST stuff' is as a plugin into some other vi-like editor with tree-sitter support (neovim, helix, etc.), but if one wanted to make a fully AST-centric editor then a custom parsing system would come in very handy.

Apologies for the terribly slow reply, by the way - I realise it's been a hot while but I still wanted to get back to this :)

[conartist6]

Yeah I'm a bit out there, as my goal isn't to make one semantic editor but to make every existing editor semantic at the same time.

[conartist6]

The way I describe it as transitioning editors towards being a lot more like web browsers. You can have many of them, they're written in a variety of languages but many share internals. You can even have terminal browsers like Lynx that do presentation very differently, yet still you can write a website once that will run on all (most) browsers.

If you think about it, a webapp is really more or less a set of automated tools for editing a DOM tree. We have all these amazing libraries and frameworks that can exist because at the end of the day everything comes down to editing that DOM tree that is embedded into the JS VM. There's a great explanation of the dynamics here: https://glazkov.com/2024/01/02/the-protocol-force/

I imagine that if a code-DOM existed and was shared by all IDEs there would spring up a rich ecosystem of tools for accomplishing many kinds of tree alterations. For example you'd have codemods that could help any project upgrade past breaking changes in its dependencies!

[Note: I think I did this explanation pretty well, so I'm copying it into the official docs.]

[kneasle]

That's a very wild goal (which is absolutely not a criticism!), and a really good blog article :) It would be really neat to have a thin waist like LSP but for syntax trees. Good luck, it seems like a plenty interesting problem to work on!

[conartist6]

Thank you! If you'd like to follow along, come join the Discord -- that's where I do all my thinking out loud at the moment and where I collect the people whose feedback I would want, which includes yours!