Cannot specify directories for testing after upgrading to rails 5.2.0

Question

I upgraded from rails 5.1.1 to rails 5.2.0. My process was the following:

• Upgraded the rails gems and resolved dependency issues
• Looked at the railsdiff between 5.1.1 and 5.2.0 and updated files as needed

Everything looks to be working well except I noticed some odd behavior within my test suite

• running the command rails test test/system is not running my system tests at all. It runs all of my other tests (policies, services, models, jobs), but does not run my system tests.
  • Strangely: running a specific system test file, ex: rails test test/system/comment_test.rb works.
• Running specific directories of tests no longer works. For example, previously I could run rails test test/policies, rails test test/models, and rails test test/services and it would only run tests in those directories. This is no longer working. Instead it just runs all of tests (except the system tests).

Any idea what changed between rails 5.1.1 and rails 5.2.0 that is causing this behavior? Or, perhaps I am just missing something?

According to the Rails Guides Testing Rails Applications: 2.7 The Rails Test Runner, it states that one should be able to run tests on a per-directory basis:

You can also run an entire directory of tests by providing the path to the directory.

Expected behavior:

rails test => run entire test suite

rails test test/system => run all tests in the system directory

rails test test/models => run only the tests in the models directory

Reproducing the Behavior

I can reproduce the behavior with the following commands:

rails new test_app         # create demo app
cd test_app                # cd into app
rails g scaffold user name # scaffold user
rake db:migrate            # migrate
rails test test/system/    # Expect to run system tests but does not (scaffold creates 4 system tests)

Answer 1

From rails -T you can see the available tasks. The two relevant ones to this questions are

rails test          # Runs all tests in test folder except system ones
rails test:system   # Run system tests only

from which we can see rails test won't run system tests. For that you need to run rails test:system, iirc correctly you can run all tests with rails test:system test