How can Visual Studio automatically build and test C# code?

Question

I'm used to Eclipse for Java projects which automatically builds whenever I save a file. (I can turn this off.)

I then install Infinitest which automatically runs all tests affected by the change saved.

Hows do I do this for Visual Studio, writing C# software?

Answer 1

Important note:

If you're only concerned about C#/.NET code and only want to run Unit Tests, then this feature already exists in Visual Studio 2017 (Enterprise edition only) called Live Unit Testing, read more about it here: https://docs.microsoft.com/en-us/visualstudio/test/live-unit-testing-intro?view=vs-2017

Live Unit Testing is a technology available in Visual Studio 2017 version 15.3 that executes your unit tests automatically in real time as you make code changes.

My original answer:

(I used to be an SDE at Microsoft working on Visual Studio (2012, 2013, and 2015). I didn't work on the build pipeline myself, but I hope I can provide some insight:)

How can Visual Studio automatically build and test code?

It doesn't, and in my opinion, it shouldn't, assuming by "build and test code" you mean it should perform a standard project build and then run the tests.

Eclipse only builds what is affected by the change. Works like a charm.

Even incremental builds aren't instant, especially if there's significant post-compile activity (e.g. complicated linking and optimizations (even in debug mode), external build tasks (e.g. embedding resources, executable compression, code signing, etc).

In Eclipse, specifically, this feature is not perfect. Eclipse is primarily a Java IDE and in Java projects it's quite possible to perform an incremental build very quickly because Java's build time is very fast anyway, and an incremental build is as simple as swapping an embedded .class file in a Java .jar. In Visual Studio, for comparison, a .NET assembly build time is also fast - but not as simple, as the output PE (.exe/.dll) file is not as simple to rebuild.

However in other project types, especially C++, build times are much longer so it's inappropriate to have this feature for C/C++ developers, in fact Eclipse's own documentation advises C/C++ users to turn this feature off:

https://help.eclipse.org/mars/index.jsp?topic=%2Forg.eclipse.cdt.doc.user%2Ftasks%2Fcdt_t_autobuild.htm

By default, the Eclipse workbench is configured to build projects automatically. However, for C/C++ development you should disable this option, otherwise your entire project will be rebuilt whenever, for example, you save a change to your makefile or source files. Click Project > Build Automatically and ensure there is no checkmark beside the Build Automatically menu item.

Other project types don't support this feature either, such as Eclipse's plugin for Go:

https://github.com/GoClipse/goclipse/releases/tag/v0.14.0

Changes in 0.14.0:
[...] Project builder is no longer invoked when workspace "Build Automatically" setting is enabled and a file is saved. (this was considered a misfeature anyways)

(That parenthetical remark is in GoClipse's changelist and certainly makes clear that plugin's authors' opinions of Automatic Builds)

I then install Infinitest which automatically runs all tests affected by the change saved.

Visual Studio can run your tests after a build automatically (but you still need to trigger the build yourself), this is a built-in feature, see here:

https://docs.microsoft.com/en-us/visualstudio/test/run-unit-tests-with-test-explorer?view=vs-2017

To run your unit tests after each local build, choose Test on the standard menu, and then choose Run Tests After Build on the Test Explorer toolbar.

As for my reasons why Visual Studio does not support Build-on-Save:

• Only the most trivial C#/VB and TypeScript projects build in under one second, the other project types, like C, C++, SQL Database, etc take between a few seconds for warm rebuilds of simple projects - to literally hours for large-scale C++ projects with lots of imported headers on a single-core CPU, low RAM and with a 5,400rpm IDE hard-drive.
• Many builds are very IO-intensive (especially C/C++ projects with lots of headers *cough*like <windows.h>*cough*) rather than CPU-bound, and disk IO delays are a major cause of lockups and slowdowns in other applications running on the computer because a disk paging operation might be delayed or because they're performing disk IO operations on the GUI thread, and so on - so with this feature enabled in a disk IO-heavy build it just means your computer will jitter a lot every time you press Ctrl+S or whenever autosave runs.
• Not every project type supports incremental builds, or can support a fast incremental build. Java is the exception to this rule because Java was designed so that each input source .java file maps 1-to-1 to an output .class file, this makes incremental builds very fast as only the actually modified file needs be rebuilt, but other projects like C# and C++ don't have this luxury: if you make even an inconsequential 1-character edit to a C preprocessor macro or C++ template you'll need to recompile everything else that used that template - and then the linker and optimizer (if code-inlining) will both have to re-run - not a quick task.
• A build can involve deleting files on disk (such as cleaning your build output folder) or changing your global system state (such as writing to a non-project build log) - in my opinion if a program ever deletes anything under a directory that I personally own (e.g. C:\git\ or C:\Users\me\Documents\Visual Studio Projects) it had damn well better ask for direct permission from me to do so every time - especially if I want to do something with the last build output while I'm working on something. I don't want to have to copy the build output to a safe directory first. This is also why the "Clean Project" command is separate and not implied by "Build Project".
• Users often press Ctrl+S habitually every few seconds (I'm one of those people) - I press Ctrl+S even when I've written incomplete code in my editor: things with syntax errors or perhaps even destructive code - I don't want that code built at all because it isn't ready to be built! Even if I have no errors in my code there's no way for the IDE to infer my intent.
• Building a project is one way to get a list of errors with your codebase, but that hasn't been necessary for decades: IDEs have long had design-time errors and warnings without needing the compiler to run a build (thanks to things like Language Servers) - in fact running a build will just give me double error messages in the IDE's error window because I will already have error messages from the design-time error list.
• Visual Studio, at least (I can't speak for Eclipse) enters a special kind of read-only mode during a build: so you can't save further changes to disk while a build is in progress, you can't change project or IDE settings, and so on - this is because a the build process is a long process that depends on the project source being in a fixed, known state - the compiler can't do its job if the source files are being modified while it's reading them! So if the IDE was always building (even if just for a few seconds) after each save, users won't like how the IDE is blocking them from certain editing tasks until the build is done (remember IDEs do more than just show editors: some specialized tool window might need to write to a project file just to open).
• Finally, Building is not free of side-effects (in fact, that's the whole point!) - there is always a risk something could go wrong in the build process and break something else on your system. I'm not saying building is risky, but if you have a custom build script that does something risky (e.g. it runs a TRUNCATE TABLE CriticalSystemParameters) and the build breaks (because they always do) it might leave your system in a bad state.
• Also, there's the (slightly philosophical) problem of: "What happens if you save incomplete changes to the build script of your project?".

Now I admit that some project types do build very, very quickly like TypeScript, Java and C#, and others have source-files that don't need compiling and linking at all and just run validation tools (like PHP or JavaScript) - and having Build-on-Save might be useful for those people - but arguably for the limited number of people whose experience it improves it demonstrably worsens it for the rest of the users.

And if you really want build-on-save, it's trivial enough to write as an extension for Visual Studio (hook the "File Save" command and then invoke the Project Build command in your handler) - or get into the habit of pressing Ctrl+B after Ctrl+S :)

Answer 2

VS 2017 Enterprise Edition supports a Live Unit Testing feature. For older versions, or lower editions, some third party providers such as Mighty Moose or NCrunch are available (other third party solutions almost certainly exist also)