Microsoft Delivers Compiler As A Service Preview

Microsoft has released a Community Technology Preview of its new compiler-as-a-service offering, codenamed Roslyn.

Microsoft said the reasoning behind a compiler as a service is to open up the inner workings so developers can take full advantage of the features. The Roslyn project enables Microsoft’s C# and Visual Basic compilers to be used as a service. In essence, Roslyn opens up the Visual Basic and C# compilers as APIs. These APIs allow tools and end users to share in the wealth of information the compilers have about code, Microsoft said on the MSDN page on the project.

No more opaque boxes

In a Roslyn blog post, ‘Soma’ Somasegar, corporate vice president of Microsoft’s Developer Division, explained that, historically, the managed compilers Microsoft shipped in Visual Studio “have been opaque boxes: you provide source files and they churn those files into output assemblies”.

“Developers haven’t been privy to the intermediate knowledge that the compiler itself generates as part of the compilation process, and yet such rich data is incredibly valuable for building the kinds of higher-level services and tools we’ve come to expect in modern-day development environments like Visual Studio,” he said.

The Roslyn overview documentation overview document describes the scenario this way:

Traditionally, compilers are black boxes – source code goes in one end, magic happens in the middle, and object files or assemblies come out the other end. As compilers perform their magic, they build up deep understanding of the code they are processing, but that knowledge is unavailable to anyone but the compiler implementation wizards, and it is promptly forgotten after the translated output is produced.

For decades, this world view has served us well, but it is no longer sufficient. Increasingly, we rely on integrated development environment (IDE) features such as IntelliSense, refactoring, intelligent rename, “Find all references”, and “Go to definition” to increase our productivity. We rely on code-analysis tools to improve our code quality and code generators to aid in application construction.

As these tools get smarter, they need access to more and more of the deep code knowledge that only compilers possess. This is the core mission of the Roslyn project: opening up the black boxes and allowing tools and end users to share in the wealth of information compilers have about our code. Instead of being opaque source-code-in and object-code-out translators, through the Roslyn project, compilers become services – APIs that you can use for code-related tasks in your tools and applications.

Thus, the Roslyn compilers become services exposed for general consumption, with all of that internal compiler-discovered knowledge made available for developers and their tools to harness, Somasegar said. “The stages of the compiler for parsing, for doing semantic analysis, for binding, and for IL emitting are all exposed to developers via rich managed APIs,” he said.

In a separate blog post, Kevin Pilch-Bisson, a development lead at Microsoft responsible for the C# and Visual Basic language services in Roslyn, called it a forward-looking “effort to make the wealth of language understanding that the compiler generates available to developers in other scenarios”.

“The foundation of this work is a new C# compiler, written in C# (and a new VB compiler, written in VB too, see the VB Team blog for details). This compiler is written as a library that exposes a rich public API. Next up is a new language service written purely using that public API and exposing its own extensibility points to allow third parties to do amazing things inside Visual Studio with that language understanding,” he wrote.

Somasegar said Microsoft’s tools team has been busy working on C# 5 and Visual Basic 11. Yet, via Roslyn, they also have been working concurrently on a complete rewrite of the C# and Visual Basic compilers, he said. “Whereas today’s compilers are implemented in native C++, in Roslyn we’ve rewritten the compilers from the ground up, implementing the C# compiler in C# and the Visual Basic compiler in Visual Basic. That in and of itself isn’t entirely noteworthy, as it’s long been a tradition for a language compiler to be implemented in its target language, something that’s been true of both our F# and Visual C++ compilers. What’s quite noteworthy are the scenarios and services this work enables.”

What it enables is the next generation of language object models for code generation, analysis and refactoring, and the upcoming support for scripting and interactive use of VB and C#, Microsoft said.

Microsoft began talking about its compiler-as-a-service technology at least as far back as the Microsoft Professional Developers Conference (PDC) in 2009.