What exactly is the 'Saff Squeeze' method of finding a bug?

Question

I have read Kent Beck's original blog post on the Saff Squeeze method. I have also read this InfoQ post that elaborates a bit more on the topic but does not provide any examples.

I know that it is essentially a way of homing in on a bug without relying on the debugger. However I find Kent's example to be not that clear.

Can someone more enlightened educate me on how to use this approach with a clear, concrete example? It'll hopefully serve as a learning resource for anyone else researching the method too.

Answer 1

The Saff Squeeze is a systematic technique for deleting both test code and non-test code from a failing test until the test and code are small enough to understand.

I agree that Kent's original description of the Saff Squeeze is a little difficult, partly because the software he's testing, JUnit, is highly abstracted, and partly because he doesn't give enough examples of step 2, "Place a (failing) assertion earlier in the test than the existing assertions."

In his first round he just moves the assertion higher in the test, and his summary of later steps might lead you to think that the only thing you can do in step 2 is move existing assertions, but by his final step he's come up with a new, simpler failing assertion. The assertion in step 2 can just be an existing one moved higher in the test, which is common, but it can also be a new one that you come up with as your understanding of the code and the bug evolves.

Here's an example. It's too simple to need the Saff Squeeze, but it illustrates the technique.

I just wrote this mission-critical class:

class Autopilot    def self.fly_to(city)     runways_in_city = runways_in city     runway = closest_available runways_in_city     flight_plan = flight_plan_to runway     carry_out flight_plan   end    def self.runways_in(city)     Airport.where(city: city).map(&:runways).flatten   end    def self.closest_available(runways)     runways.select { |r| r.available? }       .sort_by { |r| distance_between current_position, r.position }.last   end    def self.flight_plan_to(runway)     FlightPlan.new runway.latitude, runway.longitude   end    # other methods left to the imagination  end 

Here's the first rspec example I wrote to test it:

describe Autopilot   describe ".fly_to" do     it "flies to the only available runway" do       Autopilot.stub(:current_position) { Position.new 0, 0 }       nearby_runway = create :runway, latitude: 1, longitude: 1       create :runway, city: nearby_runway.city, latitude: 2, longitude: 2       flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude       # Think of the following line as being at the end of the example, since that's when it takes effect       Autopilot.should_receive(:carry_out).with flight_plan       Autopilot.fly_to nearby_runway.airport.city     end   end end 

Oh no -- the last line fails with this message: "Expectation failed: Expected Autopilot.carry_out to be called with FlightPlan(latitude: 1, longitude: 1), but it was called with FlightPlan(latitude: 2, longitude: 2)". I have no idea how that happened. We'd better use the Saff Squeeze.

Inline the method (renaming a local to avoid name collision):

it "flies to the only available runway" do   Autopilot.stub(:current_position) { Position.new 0, 0 }   nearby_runway = create :runway, latitude: 1, longitude: 1   create :runway, city: nearby_runway.city, latitude: 2, longitude: 2   flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude   Autopilot.should_receive(:carry_out).with flight_plan   runways_in_city = runways_in city   runway = closest_available runways_in_city   actual_flight_plan = flight_plan_to runway   Autopilot.carry_out actual_flight_plan end 

I don't see how that last line could fail to meet the expectation, as long as it's getting the right FlightPlan. Let's see if we can write a failing assertion higher up in the test:

it "flies to the only available runway" do   Autopilot.stub(:current_position) { Position.new 0, 0 }   nearby_runway = create :runway, latitude: 1, longitude: 1   create :runway, city: nearby_runway.city, latitude: 2, longitude: 2   flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude   Autopilot.should_receive(:carry_out).with flight_plan   runways_in_city = runways_in city   runway = closest_available runways_in_city   actual_flight_plan = flight_plan_to runway   actual_flight_plan.should == flight_plan   Autopilot.carry_out actual_flight_plan end 

Ah, the new assertion fails too, with "expected FlightPlan(latitude: 1, longitude: 1), but got FlightPlan(latitude: 2, longitude: 2)". OK, let's simplify the test:

it "flies to the only available runway" do   Autopilot.stub(:current_position) { Position.new 0, 0 }   nearby_runway = create :runway, latitude: 1, longitude: 1   create :runway, city: nearby_runway.city, latitude: 2, longitude: 2   flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude   runways_in_city = runways_in city   runway = closest_available runways_in_city   actual_flight_plan = flight_plan_to runway   actual_flight_plan.should == flight_plan end 

We're getting somewhere, but I still don't see what's wrong. Better Saff Squeeze again, inlining flight_plan_to:

it "flies to the only available runway" do   Autopilot.stub(:current_position) { Position.new 0, 0 }   nearby_runway = create :runway, latitude: 1, longitude: 1   create :runway, city: nearby_runway.city, latitude: 2, longitude: 2   flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude   runways_in_city = runways_in city   runway = closest_available runways_in_city   actual_flight_plan = FlightPlan.new runway.latitude, runway.longitude   actual_flight_plan.should == flight_plan end 

Well, obviously that's going to pass as long as flight_plan_to gets the right Runway. Let's assert that:

it "flies to the only available runway" do   Autopilot.stub(:current_position) { Position.new 0, 0 }   nearby_runway = create :runway, latitude: 1, longitude: 1   create :runway, city: nearby_runway.city, latitude: 2, longitude: 2   flight_plan = FlightPlan.new nearby_runway.latitude, nearby_runway.longitude   runways_in_city = runways_in city   runway = closest_available runways_in_city   runway.should == nearby_runway   actual_flight_plan = FlightPlan.new runway.latitude, runway.longitude   actual_flight_plan.should == flight_plan end 

Good, the new assertion fails, with "expected Runway(id: 1) but got Runway(id: 2)". Simplify the test again:

it "flies to the only available runway" do   Autopilot.stub(:current_position) { Position.new 0, 0 }   nearby_runway = create :runway, latitude: 1, longitude: 1   create :runway, city: nearby_runway.city, latitude: 2, longitude: 2   runways_in_city = runways_in city   runway = closest_available runways_in_city   runway.should == nearby_runway end 

We've pruned our original test and code to the point where it's obvious that the bug is in closest_available -- it should use first instead of last.

But what if it's still not obvious, you say? Well, let's try to Saff Squeeze again, inlining closest_available:

it "flies to the only available runway" do   Autopilot.stub(:current_position) { Position.new 0, 0 }   nearby_runway = create :runway, latitude: 1, longitude: 1   create :runway, city: nearby_runway.city, latitude: 2, longitude: 2   runways_in_city = runways_in city   runway = runways_in_city.select { |r| r.available? }     .sort_by { |r| Autopilot.distance_between Autopilot.current_position, r.position }.last   runway.should == nearby_runway end 

Now, where am I going to place a failing assertion higher in the test? I can't -- the bug is in the very last line of the test. Eventually I'll be forced to realize that it was in closest_available before I inlined it.

Answer 2

The sample shows that he is copying (inlining) the code under test inline into his unit test. Then testing the parts of the code separately from the begin to the end. This enables him to test each path in isolation and the produce unit test on the smallest possible units. One of the tests will demonstrate the defect and you will be able to fix your defect. The sample he shows is depending on the ability of Eclipse to inline methods, if you do not have this you need to do that by hand (copying the called code to your unit test).