Making sense of Scala development tools

Question

There is a myriad of development tools and terms in the ecosystem, for example, language server, build server, Metals, BSP, LSP, Bloop, Zinc, Coursier, incremental compiler, presentation compiler, etc.

I was wondering if someone could demonstrate how they fit together and briefly explain relations and differences. Specifically I am hoping for a diagram and answer along the lines of Making sense of Scala FP Libraries. For example, here is my attempt

(Concept)                       (Example implementation)
--------------------------------------------------------------------
IDE                             Visual Studio Code
 |                               |
Scala IDE plugin                Metals Visual Studio Extension
 |                               |
Language Server Protocol        Microsoft LSP
 |                               |
Scala language server           Metals
 |                               |
Build Server Protocol           BSP from JetBrains and Scala Center
 |                               |
Scala build server              Bloop
 |                               |
Build tool                      sbt
 |                               |
Dependency resolution           Coursier
 |                               |
Incremental compiler            Zinc
 |                               |
Presentation compiler           parser and typer phases of scalac
 |                               |
Bytecode generation             remaining compiler phases

Answer 1

IDEs like Intellij or (once) Scala IDE are intended for "smart" development, where the editor tells you if your code is correct, suggest fixes, autogenerate some code, provide navigation around code, refactoring - in other words many features that expand way beyond simple text edition with perhaps only syntax highlighting.

Intellij has a Scala extension which makes use of their own reimplementation of a Scala compiler - they needed that for better partial compilation and intellisense working even if part of the code is broken. The import build definition from other build tool (e.g. sbt or bloop) and from then Intellij doesn't reply on anything external (unless you use some option like "build with sbt").

Scala IDE relied on Scala presentation compiler for intellisense. As you can read on scala-ide.org:

The Scala IDE for Eclipse uses the Scala Presentation Compiler, a faster asynchronous version of the Scala Compiler. The presentation compiler only runs the phases up until and including the typer phase, that is, the first 4 of the 27 scala compilation phases. The IDE uses the presentation compiler to provide semantic features such as live error markers, inferred type hovers, and semantic highlighting. This document describes the key classes you need to know in order to understand how the Scala IDE uses the presentation compiler and provides some examples of interactions between the IDE and the presentation compiler.

Every other editor/IDE is intended to use Language Server Protocol - LSP is Microsoft's invention in order to standardize a way of supporting languages within different editors (though they invented it for the sake of VS Code) that would allow them to provide IDE features. Metals (from ScalaMeta Language Server) is LSP implementation for Scala. As you can read here:

Code completions, type at point and parameter hints are implemented using the Scala presentation compiler, which is maintained by the Scala compiler team at Lightbend.

You can add it to VS Code using Scala Metals extension.

sbt, gradle, mill, fury, cbt, etc are build tools which use something like ivy2 or coursier to resolve and download dependencies, and then use Zinc incremental compiler in order to provide ability to (re)build things incrementally to normal compiler. Build tools can run tests, generate artifacts and deploy them to repository.

bloop is a solution to problem that compiletion is fast if JVM is hot, and JVM gets cold every time you kill your build tool/IDE. For that reason you use nailgun to keep some JVM warm, running build tasks in background. On its own bloop cannot generate configuration and in general it is something that should be generated by other build tool to speed up compilation during development. Protocol used to communicate with bloop server running in background is build server protocol (bsp).

Coursier, while is used primarily for dependency resolution, can also be used to install scala programs. Some of the noteworthy programs that you can install are:

• scalafmt - scala formatter
• ammonite - alternative REPL to scala which provides a lot of nice features
• scalafix - code-rewriting tool which is used to provide automatic code migrations

I gave up on describing thing in table, as it would much better be shown on a graph, but since SO doesn't support that visuals I just resorted to a plain text.