Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

PiTest "changed conditional boundary mutation survived" without reason?

I have a small Java 11 example with a JUnit 5 test that results in a pitest result of:

changed conditional boundary → SURVIVED

Main class:

public final class CheckerUtils
 {
  private CheckerUtils()
   {
    super();
   }


  public static int checkPort(final int port)
   {
    if (port < 0)
     {
      throw new IndexOutOfBoundsException("Port number out of range!");
     }
    return port;
   }

 }

Test class:

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;

import org.junit.jupiter.api.Test;

import de.powerstat.security.CheckerUtils;


public final class CheckerUtilsTests
 {
  @Test
  public void checkPortOk()
   {
    final int port = 1023;
    final int resultPort = CheckerUtils.checkPort(port);
    assertEquals(port, resultPort, "Port not as expected!");
   }


  @Test
  public void checkPortNegative1()
   {
    final int port = -1;
    assertThrows(IndexOutOfBoundsException.class, () ->
     {
      CheckerUtils.checkPort(port);
     }
    );
   }


  @Test
  public void checkPortNegative2()
   {
    final int port = -1;
    int resultPort = 0;
    try
     {
      resultPort = CheckerUtils.checkPort(port);
     }
    catch (final IndexOutOfBoundsException e)
     {
      // ignore
     }
    assertEquals(0, resultPort, "Port is not 0");
   }

 }

From my point of view the mutation should not survive, because:

  1. checkPortOk() is the normal path for a legal value that is not negative
  2. checkPortNegative1() is the path for the negative value when noting is mutated an the exception is thrown.
  3. checkPortNegative2(): When nothing is mutated, the exception is thrown and resultPort is still 0 - so the assertion is ok here
  4. checkPortNegative2(): When the < 0 is mutated to < -1 or something lower, then no exception will be thrown, so resultPort will become -1 and the assert will fail (mutation killed)
  5. checkPortNegative2(): When the < 0 is mutated to < 1 or something higher, the same as under 3.

So my question is did I miss something here or is it a bug in pitest (1.4.9)?

Solution

As statet by @henry,adding the following test solves the issue:

@Test
public void checkPortOk2()
 {
  final int port = 0;
  final int resultPort = CheckerUtils.checkPort(port);
  assertEquals(port, resultPort, "Port not as expected!");
 }
like image 705
PowerStat Avatar asked Jul 13 '19 18:07

PowerStat


People also ask

Why did the mutation survive?

Let's take the first mutation – negated conditional – on line 6 as an example. The mutant survived because even if we change the code snippet: The test will pass, and that's why the mutation survived.

What is the difference between negate conditionals mutator and boundary mutator?

The negate conditionals mutator will mutate all conditionals found according to the replacement table below. This mutator overlaps to a degree with the conditionals boundary mutator, but is less stable i.e these mutations are generally easier for a test suite to detect.

Why are the optional mutators not enabled by default?

The reason these are not enabled by default is that there is a large degree of overlap in the tests required to kill these mutations and those required to kill mutations from other default operators such as the conditional boundaries mutator. Optional mutator that removes local variable increments.

What is pit mutation testing?

Real world mutation testing. PIT is a state of the art mutation testing system, providing gold standard test coverage for Java and the jvm. It's fast, scalable and integrates with modern test and build tooling. Get Started.


1 Answers

The conditional boundary mutation will mutate

if (port < 0)

To

if (port <= 0)

As none of the tests provide an input of 0 they are not able to distinguish the mutant from the unmutated program and the mutant will survive.

Adding a test case that describes the expected behaviour when port is 0 should kill the mutant.

like image 83
henry Avatar answered Nov 10 '22 00:11

henry