Rails: Testing authorization (Pundit) with RSpec

Question

I'd like to thoroughly test a Rails application authorization setup (Pundit) with RSpec.

1. The Pundit docs and other respectful sources suggest writing unit tests for Pundit policy classes. Are those tests testing behavior (good) or implementation details (bad)?

2. If we were actually testing behavior, shouldn't our tests still pass if we eventually switched from Pundit to another authorization gem or rolled our own authorization?

3. Suppose a call to the authorize method is accidentally removed from a controller action, leaving it open for public access. Pundit policy tests will keep passing. If we have no other tests covering this, our test suite may be all green while our app has a serious security vulnerability. How can we avoid this? Maybe write unit tests for Pundit controller classes, separate unit tests for controllers and mock the authorize method in the controller tests to ensure they are called?

Edit: On second thought, I had a lapse of judgment regarding the difference between implementation details and the public API for a class. Calling specific public methods, passing specific arguments and expecting specific return values is a requirement to unit-test (and to use) any class. Please disregard item 3 as my original argument is invalid.

4. Instead of testing Pundit policy classes, we could write controller or request specs that login with different user roles, call the controller actions and assert whether access was granted or denied. In this approach, the tests would keep passing even if we switched to another authorization gem. Also, they would fail if the authorize method is not called. However, these would be integration tests and should be slower that unit testing Pundit policy classes. Could that be a better way to test the authorization setup?

Thanks in advance.

Answer 1

1) The Pundit docs and other respectful sources) suggest writing unit tests for Pundit policy classes. Are those tests testing behavior (good) or implementation details (bad)?

A good policy spec is testing the behavior of a component of the system. Not every spec that does not test the system as a whole is evil. Its rather about testing what is does - not how it does it.

You can compare this to testing a service object or any other component.

I find policy specs to be a very succint and accurate way of testing the authorization behavior without the extra layers of abstraction (HTTP, application state ...). This is much like how model specs are better to test validation edge cases rather than dragging along Capybara.

2) If we were actually testing behavior, shouldn't our tests still pass if we eventually switched from Pundit to another authorization gem or rolled our own authorization?

Yes - but this can pretty much be argued for any component in the system. Your higher level specs (request and feature) should still pass (that also cover that the controller properly integrates Pundit).

3) Suppose a call to the authorize method is accidentally removed from a controller action, leaving it open for public access. Pundit policy tests will keep passing. If we have no other tests covering this, our test suite may be all green while our app has a serious security vulnerability. How can we avoid this? Maybe write unit tests for Pundit controller classes, separate unit tests for controllers and mock the authorize method in the controller tests to ensure they are called?

Controller tests are depreachiated in Rails 5. Use request or feature specs instead of prying in the internals.

You should have request / feature specs that ensure that the user cannot perform actions that they are not authorized to do. In a request spec for an API for example you would check that the response code is Unauthorised (401). You can also check that the was no change performed if you are paranoid.

For feature specs you test that the user is redirected or notified that they cannot perform the action.

Again test what is does - not how it does it.

Since this is pretty repetive you can use shared examples to dry it up.

4) Instead of testing Pundit policy classes, we could write controller or request specs that login with different user roles, call the controller actions and assert whether access was granted or denied. In this approach, the tests would keep passing even if we switched to another authorization gem. Also, they would fail if the authorize method is not called. However, these would be integration tests and should be slower that unit testing Pundit policy classes. Could that be a better way to test the authorization setup?

Again the choice here is not binary. Both types of tests provide value. You could get by solely integration tests but its hard and slow to cover every possible case and edge case.