Rewrite it in Rust

I’ve been alternating between obsessed with and simply always entertaining in the back of my mind the idea of a rust-powered fish for many years now (and have discussed that with other team members), so I’m approaching this with an open mind, but also with concerns. I must say that I knew this was coming for many months now from following @ridiculousfish on GitHub (❤️), and have been wondering how it would the topic would be broached.

I passionately believe that rust is the way forward for writing any new code, especially in a cross-platform context. Quite apart from all the memory and concurrency safety it brings to the table, the general approach of the language and the community to always put correctness first and foremost has built a strong culture of correct, well-designed, and fairly maintainable software engineering. The package management is great, the team is fantastic, and support in the broader software world has been extremely positive and welcoming.

To go further, I have written my own fish-compatible rewrite (not port) of fish’s tokenizer and was making progress on the AST and parser when I last had my “burning desire” to see something like fish but in rust. I was maintaining it on my private git server and not GitHub, but I’ve pushed it to GH in case anyone cares to take a look: https://github.com/mqudsi/velvet

Having said all that, I worry that rushing any part of this could be a death knell for the project. A quality rewrite (or port) takes time, on the magnitude of years rather than days. One good example to take inspiration from is tectonic, a xelatex port that started off with machine-rewritten code and then manually made progress on actual, idiomatic rust conversions… but even that was a greenfield project and not a first-party effort and so not saddled with some of the concerns we would be saddled by. The recommended module-by-module approach is definitely a great suggestion for a starting place, and I think it has great potential.

Off the top of my head, some concerns or points:

• Compatibility, obviously. Fish runs on archaic hardware for fun. This would be the end of that. That’s not a reason that should stand in the way of progress, but it’s something we can’t turn a blind eye to and would have to quite clearly signal that we’re ok giving up all platforms that don’t run (modern versions of) rust.
• Idiomatic rust is too high-level for fish. You don’t fork in rust, using the standard library interface doesn’t give you access to poll/select/epoll/kqueue/whatever natively. async io is a horrible fake cludge with tokio/async_std faking it with thread pools or whatnot and performing far worse than truly native async implementations. We’d have to go that mostly alone to keep the same code we have (thinking “port” and not “rewrite” and aiming to preserve what we have).
• Despite whether we call it a “port” or a “rewrite” there’s going to be a heck of a lot of rewriting taking place. Subtle bugs will creep in. Subtle compatibility fixes that weren’t properly documented or annotated will be lost.
• We will be in a state of partial limbo until this completes one way or the other. We won’t want to make any major changes to the C++ codebase while we’re trying to get to feature parity in the rust one. Depending on how things unfold this could greatly hinder further progress in the project.
• Not all core team members are fluent in rust. This seems unfair. (This isn’t just something I’m saying in passing but rather something I’ve given a lot of thought to and broached with others before.)
• C++ isn’t really a “legacy” language (yet), no matter how much we wish the world would start treating it like that given all its warts and shortcomings.
• I understand what @ridiculousfish meant by “Rust is what we need to turn on concurrent function execution” but respectfully disagree. A complete rewrite (or port, if you prefer) to another language is a great time to facilitate that, but it’s hardly a requirement, though I guess it could functionally be considered one if you want to take into account correctness, maintainability, etc.
• (Really minor compared to everything else, but if we’re ultimately going to be a rust project, then I think it makes sense for all types coming from the rust codebase to use idiomatic rust naming conventions, but that would cause churn cpp-side. I suggest using idiomatic rust naming conventions and then using statements to alias them with cpp-style names in the cpp headers.)