Using MDAST to do "smart" Markdown formatting

[eliyap]

Hi All!

I'm working on an iPad app that heavily relies on MDAST.

The user can use a stylus to draw a line through some text, causing a strikethrough format. To be a bit more clever, I try to reason about the formatting that's already applied.

• e.g. drawing a line through **~~test~~ post**should result in ~~**test post**~~ instead of ~~**~~test~~ post**~~.

MDAST has been absolutely indispensable for this! It's also the only library I found with "source position" support that actually works (cmark tries to provide it but it's horrifically buggy and inaccurate).

Thanks for the great library! If you'd like to try out the app, there's a Testflight here.

[eliyap]

@wooorm related to this use case, I'm hitting a performance wall with mdast-util-from-markdown:

On a few paragraphs of plain (no Markdown formatting) text, the Javascript takes ~0.5s to parse.
Overall, the tree construction consumes 90% of the CPU time in the app(!).
0.5s is really great for Javascript on a Node server, but of course fatal for a user interface.

My approach is to reconstruct the MDAST often (max every 0.5s) when the user is editing the text; clearly this is wasteful because we discard and reconstruct the whole AST needlessly.

Is there a library extension / method you'd suggest for adjusting the MDAST given knowledge of a similar AST based on similar input text?

[wooorm]

One part is debouncing such things. If someone is dragging a stylus, it doesn’t have to update immediately.
Another part is putting this work in another thread. Away from the UI thread.
Third, is that micromark/mdast/etc all mostly focus on being perfect. There are other tools such as tree-sitter that focus on being fast at these things, at the cost of being perfect

[eliyap]

Debouncing: definitely, I'll try to be more aggressive about it.

Threading: I haven't looked at this yet, since concurrency is a scary thing, but will definitely try it.

I hadn't heard of tree sitter! Unfortunately, it looks like other folks are having trouble with tree-sitter-markdown instability, which is probably disqualifying. I'll look into it further.

I'm also exploring "chopping up" the tree on double newlines, so that I can selectively rebuild parts of it, though code fences are really frustrating that effort.

Thanks for the pointers! Will report progress as it's made.