Difference between Android Instrumentation test and Unit test in Android Studio?

Question

As of Android Studio 1.1rc there's Unit testing support and I'm wondering what's the difference between Android Instrumentation Tests and Unit tests.

As I understand it:
Unit tests are useful for testing code which doesn't call the Android API, and the Android instrumentation tests are rather integration tests to test Android API specific elements or GUI components.

However if you use a framework like Robolectric or Mockito in your unit tests, you can test Android code (without the need of a device) if I'm not mistaken.

---

Is this correct, or is there a bigger difference? If so, what's the use of each?

Answer 1

It seems to me that instrumentation testing is integration testing with the ability to control the life cycle and the events (onStart, onCreate etc) of the app.

Unit testing, as i understand it, is testing a Unit (eg Class) for its data and behavior.

For example, say you have a game: this game runs on an activity (main activity) and you have a character based on a Robot class, that has 2 methods (fire and move). You would test the main activity with an instrumentation test to see if it saves correctly when you leave the app, if it restores correctly when you restore it etc. and you would test Robot with a Unit test, to test its attributes and behavior.

Disclaimer: I'm not a java person, but i took interest in your question and i answered it based on a minor search online. You probably have to dig deeper into this to find a more detailed answer.

Answer 2

Unit tests isolate the component under test, and this is the reason why are often used together with Mocks frameworks as Mockito:because isolate the unit from their dependencies. Please notice what you say regarding the Android API is partially true, because there are also Instrumented Unit tests, namely Instrumentation is part of the Junit package as well, and also the classes that extend TestCase as the class AndroidTestCase is part of the Junit package but allows the use of A)Context, that you can call with getContext(), and B)Resources that are part of the Android API! Also please consider AndroidTestCase is a base class and there are several other classes quite useful that extend this class. They test specifically Loaders, ContentProviders and even Services and also them have access to the Android API. so these classes provide JUnit testing framework as well as Android-specific methods. Now with Junit4 there is the ServiceTestRule that extends directly from Object and allow you easier to test a Service, although you cannot start an Intent directly inside this class.

Instrumentation tests they are also into the Junit package, but the control of the Android API is quite total because Instrumentation Tests are instantiated in the system before any application code is run, and to test you need to open the real application(emulator or a phone connected with USB). They access to android components(ex. click a button) and application life cycle, they are usually slower than Junit tests that extend TestCase( the ones examined above) the typical use is with ActivityInstrumentationTestCase2 that has a functional test approach, more user oriented.

EDIT: Regarding Roboelectric and Mockito, that are togheter with Espresso between the most popular testing frameworks at the moment(13 July 2016), Roboelectric allows you to run multiple tests in seconds instead of minutes, and this comes really handy in teams that have to run continuous tests, and are subject to continuous integration.

From the site of Robolectric:

An alternate approach to Robolectric is to use mock frameworks such as Mockito or to mock out the Android SDK. While this is a valid approach, it often yields tests that are essentially reverse implementations of the application code. Roboelectric allows a test style that is closer to black box testing, making the tests more effective for refactoring and allowing the tests to focus on the behavior of the application instead of the implementation of Android. You can still use a mocking framework along with Robolectric if you like.

Mockito that also can be used with Junit, is really used with the exception of when have to manage final classes, anonymous classes or primitive types.