best practice to write real bdd cucumber feature/ scenarios

Question

We are new to bdd/ cucumber and discussing in out team how to write correct feature/ scenarios.

We came up with the two following approaches, which should almost describe/ solve the same requirement:

Feature: Give access to dossiers to other registered users
  As a logged in user
  In order to show my dossier to other users
  I want to give other users (limited) access to my dossiers

  Background:
    Given I am logged in as "Oliver"
    And another user "Pascal" exists
    And another user "Tobias" exists

  Scenario: I can give access to my own dossier
    When I grant access to "Pascal" with permisson "readonly"
    Then I should see "Access granted."
    And user "Pascal" should have permission "readonly" on dossier "Oliver"

  Scenario: I can give access to a created dossier
    Given I created a new dossier "Max Müller"
    When I grant access on dossier "Max Müller" to "Pascal" with permisson "readonly"
    Then I should see "Access granted."
    And user "Pascal" should have permission "readonly" on dossier "Max Müller"

  Scenario: I can give access to a managed dossier
    Given I manage the dossier from "Tobias"
    When I grant access on dossier "Tobias" to "Pascal" with permisson "readonly"
    Then I should see "Access granted."
    And user "Pascal" should have permission "readonly" on dossier "Tobias"

  Scenario: I cannot give access to a writable dossier
    Given I have write access to the dossier from "Tobias"
    When I follow "Grant access"
    Then I should see "You are not allowed to grant access on this dossier."

  Scenario: I cannot give access to a readonly dossier
    Given I have readonly access to the dossier from "Tobias"
    When I follow "Grant access"
    Then I should see "You are not allowed to grant access on this dossier."

  Scenario: I can give access to a dossier with an expiration date
    Given I created a new dossier "Max Müller"
    When I grant access on dossier "Max Müller" to "Pascal" with permisson "readonly" until "2020-01-01"
    Then I should see "Access granted till 2020-01-01."
    And user "Pascal" should have permission "readonly" on dossier "Max Müller" until "2020-01-01"

  Scenario: I cannot transfer a created dossier to a new owner who is already registered
    Given I created a new dossier "Max Müller"
    When I transfer dossier "Max Müller" to "Pascal"
    Then I should see "Pascal already has a dossier, transfer not possible."

The second one:

Feature: Grant access on dossiers to registered users
  As a logged in user
  In order to allow others to view/ manage dossiers I have access to
  I want to give access of those to other users

  Background:
    Given I am logged in as "[email protected]"
    And I am working with my own dossier

  Scenario: Invalid data entered 
    When I visit the grant dossier access page
    And I press "Grant access"
    Then I should see a validation error on "eMail-Address"

  Scenario: Valid data entered 
    Given a user "[email protected]" exists
    When I visit the grant dossier access page
    And I fill in "eMail-Address" with "[email protected]"
    And I select "readonly" from "Permissions"
    And I press "Grant access"
    Then I should see "Access granted."
    And I should be on the dossiers page

  Scenario: Valid data entered with expiry date 
    Given a user "[email protected]" exists
    When I visit the grant dossier access page
    And I fill in "eMail-Address" with "[email protected]"
    And I select "readonly" from "Permissions"
    And I fill in "Valid until" with "2010-03-01"
    And I press "Grant access"
    Then I should see "Access granted till 2010-03-01."
    And I should be on the dossiers page

  Scenario: Display calendar on click on "Valid until"
    When I click on the field "Valid until"
    Then a calendar popup should be displayed
    When I click on "1943-01-02"
    Then the field "Valid until" should be have "1943-01-02"
    And the calendar popup should be hidden

  Scenario: Only allow to grant access to categories I have access to myself
    Given I have limited access to the working dossier 
    When I visit the grant dossier access page
    Then I should not see categories I have no access to

  Scenario: Dossier with permission "manager" can only grant readonly, readwrite
    Given I have the permission "manager" on my working dossier 
    When I visit the grant dossier access page
    Then I should only see the permissions "readonly, readwrite"

  Scenario: Dossier with permission "readwrite" is not allowed to grant any permissions
    Given I have the permission "readwrite" on my working dossier 
    When I visit the grant dossier access page
    Then I should the see the error "You cannot grant access on this dossier!"
    And I should be on the dossiers page

Which one would you prefer and why?

Answer 1

The point of writing Cucumber tests is to create a specification about what the code does that can be read by the people on your team who can't read code. I'd start by asking them which one they prefer.

My guess is they'll prefer the first one, because it's more declarative. Because it's written at a higher level of abstraction (rather than being concerned with clicking widgets on a page) it's easier to read.

Contrary to what Josh said, I think having steps that could work either through a UI or not is a really good idea. Surfacing UI concerns on a feature can make it brittle to legitimate design changes, and also rather boring to read.

I did a talk recently about this subject, I think it's really relevant to where you're at: http://skillsmatter.com/podcast/agile-testing/refuctoring-your-cukes

See also these relevant blog posts:

• http://benmabey.com/2008/05/19/imperative-vs-declarative-scenarios-in-user-stories.html
• http://dannorth.net/2011/01/31/whose-domain-is-it-anyway/
• http://elabs.se/blog/15-you-re-cuking-it-wrong

Good luck!

Answer 2

I would go with the solution that is more easily readable by your customers, or people other than developers, in general. A big advantage of using tools like Cucumber is that it reads like a story. Your goal should be to make each test simple enough so that a customer can read your acceptance tests and realize which features are already implemented. This benefits you because your customers can rest easy knowing that certain features have acceptance tests in place.

Having said that I think the first solution represents the readable style more than the second solution.

"I can give access to a dossier with an expiration date"

is easier to read than

"Valid data entered with expiry date"

in my opinion.

You don't necessarily need to prefix each scenario with "I can" or "I cannot", but ease towards the side of a complete thought.

Update

In your comment you asked about validation errors and how you should handle that with option 1. I think you have a couple of options here:

1. Full validation testing in Cucumber
2. Partial validation testing (errors are presented to user) in Cucumber, and partial testing in RSpec (errors are thrown) - or some other unit testing framework.

Option 1

pros:

You don't have to bother using another framework to test your validations. Most of your tests will be included in Cucumber. Your customers can see all of your tests in one place.

cons:

Your Cucumber tests will have varying levels of granularity included. Your customers probably care more about the high-level features than they do about seeing all of the nitty-gritty details pass. If I were a customer interested in using your Cucumber tests as a basis for what features have been implemented and tested, I would prefer a smaller, more readable list of test cases than an all encompassing one. At most I'd probably like to see that error messages are presented to the user - and you can do this in one Scenario Outline.

Option 2

pros:

You separate your testing concerns into appropriate levels of granularity. As I mentioned in the cons for Option 1, your customers are most likely interested in the higher level features than the lower level details. You can test whether validations pass or not in your RSpec unit tests, and use Cucumber to quickly test that the user sees error messages if you have an invalid record (again using Scenario outlines, or maybe just test to see if only one validation message makes it to the front end). The main point is that the more functional-test-oriented Cucumber should not test every little thing concerned with Models - let RSpec or another unit testing framework handle that.

cons:

You have to use another framework to exercise the more fine-grained expectations. Using another framework means taking more time to learn it, etc. Also it's hard to know how to balance when to use one framework over the other, i.e. "should I use Cucumber or RSpec here?"

From what I have read, Option 2 is the most popular right now. Teams use the frameworks appropriate for each level of granularity, and they try to stay away from an "all in" approach.