Why is cucumber considered an integration testing tool instead of a unit testing tool?

Question

This has always bothered me. Why is it people say to unit test in rspec but integration test in cucumber? I am not asking why these tests are necessary - I know what the difference is between integration and unit testing. I just don't see why, given cucumber's completely customizable syntax, it isn't used for unit testing?

It seems to me like the same amount of code is written for cucumber and rspec, the only difference is that for cucumber you separate the test logic from the test writing.

Answer 1

There is quite a lot of overhead in using cucumber for unit testing. Not only you have to write the features but then map them to the implementation using a separate bit of code.

Unit testing is meant to be very fast to write and very fast to execute. Naturally, cucumber focuses on end user experience, mainly due to the language used in writing a feature.

Just to refresh, a feature would contain the following:

As some stakeholder of a system
I would like to perform an activity
So that I can get some benefit out of it

Given a precondition
When I perform an action
Then something should happen

The opening paragraph, that is often ignored, is very important as it sets a context for the operation and explains why something happens. Because of the use of natural language, these things are easy to show to non-programmers in order to get some feedback.

Now, using these for unit tests would seem awkward at best. First of all, the end user focus suggests a more integration approach as the feature says nothing about mocks and UI/logic separation. I.e. a feature like the following just seems weird:

Given a that a database mock is configured with the following data
| ID  | Username |
| 0   | igor     |
When call FindAll on User Repository
Then I get the following user back
| ID  | Username |
| 0   | igor     |

Also, as your SUT gets smaller (i.e. a class), the context of operation is not as important. A User repository does not care about the context e.g. it doesn't care if the consumer of it is a normal user or a VIP user. A simple component (which it should be, following SRP), is completely deterministic based on its inputs.

So, the unit test is there to validate that what you wrote is correct, and the cucmber test is there to validate that what you wrote satisfies some higher purpose by putting a behavior of the system in a context.

Answer 2

Cucumber solves a particular set of problems - engaging business stakeholders who can't easily read code and certainly can't write it, and providing reuse between the steps in automated scenarios. The scenarios also usually cover more than one aspect of behaviour, documenting the functionality of the entire system and often covering whole user journeys across multiple components. The step-based architecture which Cucumber encourages is ideal for handling these scenarios.

It also introduces a whole set of other problems. First, you need to tie the Cucumber scenarios to a set of fixtures, so there's another layer of abstraction which makes them slower to write. Second, English is harder to refactor than code - even with dynamic languages like Ruby (the difference is still more pronounced in the C# and Java variants like JBehave, SpecFlow, Cuke4Nuke and Cuke4Duke). It's harder to tell if steps are still being used, and harder to maintain the scenarios. It's also harder to manage state between the various steps.

With unit tests, the audience is technical. Classes ideally have single responsibilities with little to no duplication, so the reuse of steps isn't important. When we want to change an element of code we tend to look for tests whose naming conventions match the files or classes, so a one-to-one mapping with these is ideal.

Because of Cucumber's overheads, and because we don't get value from the benefits Cucumber provides in return for its overheads, RSpec is a better fit for behaviour at a unit level. (This is also true of JUnit, NUnit, etc.)

If you're missing the "Given, When, Then" of Cucumber, try adding them as comments. This works well for me.