[FIXED] How to skip remaining steps if previous steps failed in Junit

Issue

I had some integration testing cases, they ran as Junit test cases with a special category:

@Category(IntegrationTest.class)

Because they are integration testing cases, so the cost of every steps is high. Usually I will re-use some results from previous steps to reduce this cost. To make it works, I added this into them:

@FixMethodOrder(MethodSorters.NAME_ASCENDING)

Some samples like this:

@Category(IntegrationTest.class)
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class TestAllocationPlanApi {

    @Test
    public void testStep01_verifyOrigProgram22275() {...}

    @Test
    public void testStep02_CopyProgram() {...}
}

They work well except the failure process:

If step01 failed, we don't need to run step02, but Junit still go to step02.

It is a waste and makes the test cases more complicated because you need to carefully handle those variables which are passed into step02.

I tried

-Dsurefire.skipAfterFailureCount=1

Which is discussed in another thread, but it doesn't work, the test cases still go to next step if previous steps fails.

Another annoying thing about the test cases is that Junit always resets all instance variables before every step. This forces me to use static variable to pass previous result into next step:

private static Integer contractAId;

And I have no way to run them in multiple threads.

Does anybody have good ideas to handle those things?

Thanks!

Happy new year!

Solution

You have written these as distinct tests but there are some dependencies between these tests. So, it sounds like you have split a single logical test flow across multiple test methods.

To cater for these dependencies you adopted a naming convention for the tests and instructed JUnit to run these tests in the order implied by the naming convention. In addition, you have some shared state within your test case which is being 'passed from' step to step.

This approach sounds brittle and, probably, makes the following quite difficult:

• Diagnosing failures, issues
• Maintaining existing steps
• Adding new steps

Instructing JUnit to - somehow - stop executing subsequent tests within a test case if a prior test failed and the use of a static variable to pass previous results into the next step are both symptoms of the decision to split a single logical test across multiple @Test methods.

JUnit has no formal concept of subsequent or prior tests within a test case. This is deliberate since @Test methods are expected to be independent of each other.

So, rather than trying to implement his behaviour: stop executing subsequent tests within a test case if a prior test failed I would suggest revisiting your tests to reduce their run time, reduce costly setup time and remove this approach of splitting a single logical test flow across multiple test methods. Instead each test should be self contained; its scope should cover (a) set up, (b) execution; (c) assertion; (d) tear down.

I can see from your question that this is an integration test so it's likely that the setup, dependency management, execution etc are not simple so perhaps this approach of splitting a single logical test flow across multiple test methods is an effort to decompose a complex test flow into more digestible units. If so, then I'd recommend breaking each of these 'steps' into private methods and orchestrating them from within a single @Test method. For example:

@Test
public void test_verifyOrigProgram22275() {
    // you'll probably want to return some sort of context object from each step
    // i.e. something which allows you to (a) test whether a step has succeeded 
    // and abort if not and (b) pass state between steps

    step01_verifyOrigProgram22275();

    step02_verifyOrigProgram22275();

    ...
}

private void step01_verifyOrigProgram22275() {...}

private void step02_CopyProgram() {...}

Answered By - glytching