Microsoft rewriting typscript in Go

[Cpvr]

A 10x Faster TypeScript​

The core value proposition of TypeScript is an excellent developer experience. As your codebase grows, so does the value of TypeScript itself, but in many cases TypeScript has not been able to scale up to the very largest codebases. Developers working in large projects can experience long load and check times, and have to choose between reasonable editor startup time or getting a complete view of their source code. We know developers love when they can rename variables with confidence, find all references to a particular function, easily navigate their codebase, and do all of those things without delay. New experiences powered by AI benefit from large windows of semantic information that need to be available with tighter latency constraints. We also want fast command-line builds to validate that your entire codebase is in good shape.

To meet those goals, we’ve begun work on a native port of the TypeScript compiler and tools. The native implementation will drastically improve editor startup, reduce most build times by 10x, and substantially reduce memory usage. By porting the current codebase, we expect to be able to preview a native implementation of tsccapable of command-line typechecking by mid-2025, with a feature-complete solution for project builds and a language service by the end of the year.

You can build and run the Go code from our new working repo, which is offered under the same license as the existing TypeScript codebase. Check the README for instructions on how to build and run tsc and the language server, and to see a summary of what’s implemented so far. We’ll be posting regular updates as new functionality becomes available for testing.

How Much Faster?​

Our native implementation is already capable of loading many popular TypeScript projects, including the TypeScript compiler itself. Here are times to run tsc on some popular codebases on GitHub of varying sizes:


While we’re not yet feature-complete, these numbers are representative of the order of magnitude performance improvement you’ll see checking most codebases.

We’re incredibly excited about the opportunities that this massive speed boost creates. Features that once seemed out of reach are now within grasp. This native port will be able to provide instant, comprehensive error listings across an entire project, support more advanced refactorings, and enable deeper insights that were previously too expensive to compute. This new foundation goes beyond today’s developer experience and will enable the next generation of AI tools to enhance development, powering new tools that will learn, adapt, and improve the coding experience.

Editor Speed​

Most developer time is spent in editors, and it’s where performance is most important. We want editors to load large projects quickly, and respond quickly in all situations. Modern editors like Visual Studio and Visual Studio Code have excellent performance as long as the underlying language services are also fast. With our native implementation, we’ll be able to provide incredibly fast editor experiences.

Again using the Visual Studio Code codebase as a benchmark, the current time to load the entire project in the editor on a fast computer is about 9.6 seconds. This drops down to about 1.2 seconds with the native language service, an 8x improvement in project load time in editor scenarios. What this translates to is a faster working experience from the time you open your editor to your first keystroke in any TypeScript codebase. We expect all projects to see this level of improvement in load time.

Overall memory usage also appears to be roughly half of the current implementation, though we haven’t actively investigated optimizing this yet and expect to realize further improvements. Editor responsiveness for all language service operations (including completion lists, quick info, go to definition, and find all references) will also see significant speed gains. We’ll also be moving to the Language Server Protocol (LSP), a longstanding infrastructural work item to better align our implementation with other languages.

Versioning Roadmap​

Our most recent TypeScript release was TypeScript 5.8, with TypeScript 5.9 coming soon. The JS-based codebase will continue development into the 6.x series, and TypeScript 6.0 will introduce some deprecations and breaking changes to align with the upcoming native codebase.

When the native codebase has reached sufficient parity with the current TypeScript, we’ll be releasing it as TypeScript 7.0. This is still in development and we’ll be announcing stability and feature milestones as they occur.

For the sake of clarity, we’ll refer to them simply as TypeScript 6 (JS) and TypeScript 7 (native), since this will be the nomenclature for the foreseeable future. You may also see us refer to “Strada” (the original TypeScript codename) and “Corsa” (the codename for this effort) in internal discussions or code comments.

While some projects may be able to switch to TypeScript 7 upon release, others may depend on certain API features, legacy configurations, or other constraints that necessitate using TypeScript 6. Recognizing TypeScript’s critical role in the JS development ecosystem, we’ll still be maintaining the JS codebase in the 6.x line until TypeScript 7+ reaches sufficient maturity and adoption.

Our long-term goal is to keep these versions as closely aligned as possible so that you can upgrade to TypeScript 7 as soon as it meets your requirements, or fall back to TypeScript 6 if necessary.

Next Steps​

In the coming months we’ll be sharing more about this exciting effort, including deeper looks into performance, a new compiler API, LSP, and more. We’ve written up some FAQs on the GitHub repo to address some questions we expect you might have. We also invite you to join us for an AMA at the TypeScript Community Discord at 10 AM PDT | 5 PM UTC on March 13th.

A 10x performance improvement represents a massive leap in the TypeScript and JavaScript development experience, so we hope you are as enthusiastic as we are for this effort!

Source; https://devblogs.microsoft.com/typescript/typescript-native-port/



video transcript:
When you watch the interview with Anders Hejlsberg (the creator of TypeScript), it actually makes a lot of sense why Go was chosen.

TypeScript is being ported to Go

[08:58] "we realized, we can get to 10x with this!"

Anders: In August, I started porting the scanner and the parser just to get a baseline so we could see how fast this could go and how hard it is to port from JavaScript to Go. It actually progressed pretty well, and within a couple of months we had something that we could run that would parse all the source code we have with no errors, and we could start to extrapolate some numbers from that. That's sort of when we started to realize, "Okay, we can get to 10x with this," because we get about three to three and a half times the speed from being native, and then we get another three to three and a half times from concurrency. Together, that gets us to 10x -- and 10x is pretty darn dramatic.

[12:34] why not Rust?

Anders: When you have a product that has been in use for more than a decade, with millions of programmers and, God knows how many millions of lines of code out there, you are going to be faced with the longest tail of incompatibilities you could imagine. So, from the get-go, we knew that the only way this was going to be meaningful was if we ported the existing code base. The existing code base makes certain assumptions -- specifically, it assumes that there is automatic garbage collection -- and that pretty much limited our choices. That heavily ruled out Rust. I mean, in Rust you have memory management, but it's not automatic; you can get reference counting or whatever you could, but then, in addition to that, there's the borrow checker and the rather stringent constraints it puts on you around ownership of data structures. In particular, it effectively outlaws cyclic data structures, and all of our data structures are heavily cyclic.

[16:29] adding it all up: Go makes a lot of sense

Anders: When we go down the list, we want a language that gives us excellent optimized native code on all major platforms. We want a language that has great expressiveness in data structures, allows cyclic data structures, but also allows inline data structures like structs -- so that, you know, in JavaScript everything you do with an object is an allocation, and we would rather avoid that, especially for small objects if we can store them inline. We need automatic garbage collection, and we also needed concurrency -- and shared memory concurrency. We can talk about the distinction there, because technically JavaScript has concurrency with web workers, but it does not have shared memory concurrency. I can explain why we need that for the compiler, but that was also a must. And when you look at all of that -- and then, of course, we want good tooling (we all live in VS Code, we want excellent support in VS Code, and whatever) -- when you add all of that up, Go actually ended up coming very high on the list.

[19:14] why not C#?

Dimitri: Was C# considered?

Anders: It was, but I will say that I think Go definitely is -- it's, I'd say, the lowest-level language we can get to and still have automatic garbage collection. It's the most native-first language we can get to and still have automatic GC. In C#, it's sort of bytecode first, if you will; there is some ahead-of-time compilation available, but it's not on all platforms and it doesn't have a decade or more of hardening. It was not geared that way to begin with. Additionally, I think Go has a little more expressiveness when it comes to data structure layout, inline structs, and so forth. For us, one additional thing is that our JavaScript codebase is written in a highly functional style -- we use very few classes; in fact, the core compiler doesn't use classes at all -- and that is actually a characteristic of Go as well. Go is based on functions and data structures, whereas C# is heavily OOP-oriented, and we would have had to switch to an OOP paradigm to move to C#. That transition would have involved more friction than switching to Go. Ultimately, that was the path of least resistance for us.

Dimitri: Great -- I mean, I have questions about that. I've struggled in the past a lot with Go in functional programming, but I'm glad to hear you say that those aren't struggles for you. That was one of my questions.

Anders: When I say functional programming here, I mean sort of functional in the plain sense that we're dealing with functions and data structures as opposed to objects. I'm not talking about pattern matching, higher-kinded types, and monads.

[23:44] typescript-go's memory consumption is HALF (!!) (while also being 10 times faster)

Anders: You can have bytes, shorts, ints, 64-bit integers, and what have you -- both signed and unsigned. In JavaScript, everything is a floating-point number. I mean, you want to represent true or false? Yeah, that's eight bits for you. So, that is tough, and that is why we do all sorts of tricks in our existing compiler. At least, we can pack 31 bits into a floating-point number typically in JavaScript. But in Go, we can use all of the bits, and, by the way, we can also lay them out as inline structs and arrays. It shows that -- our memory consumption is roughly half of what the old compiler used. In these days, memory equals speed: the more memory you use, the slower you go, because the more times you hit the write barrier or the read barrier (and you blow your L cache, your L0 cache) and then you have to go get it from real memory. I often joke that in modern CPUs, each instruction takes zero cycles because of prediction -- except when it takes a thousand cycles because you hit the memory wall and have to go fetch. So, if you can condense your data structures, you're going to go faster.

[58:36] the transition from JavaScript to Go is actually pretty gentle (compared to Rust)

Anders: I will say, too, that the transition from JavaScript to Go is actually pretty gentle on the system. It is not a super complicated language with an awful lot of ceremony, which I would say Rust comes a lot closer to. I mean, Rust is more like a modern C++ than a modernized JavaScript, and Go is actually, in some ways, a modernized, native Python/JavaScript-like thing.

Dimitri: When I used to write Go professionally for about two years, during an all-hands engineering meeting, there were some engineers complaining. One of them said it's mediocre and they just weren't satisfied with not being able to do fancy things in Go. The CTO -- I'll never forget -- stopped them and said, "You have to understand that Go is mediocre by design; it's not trying to be fancy."

Anders: It's trying to be a simple language -- and honestly, it is. But the results are not mediocre. I mean, 10x -- there's nothing mediocre about that, right? So, you can do great things with this thing.