Continuous Delivery

Continuous Delivery

>50% of the code is test code

Test Code has no Business Value

Costs of Automated Tests

Jez Humble and David Farley

Typical Test

@Test
public void shouldInsertPetIntoDatabaseAndGenerateId() {
    Owner owner6 = this.clinicService.findOwnerById(6);
    int found = owner6.getPets().size();

    Pet pet = new Pet();
    pet.setName("bowser");
    Collection<PetType> types = this.clinicService.findPetTypes();
    pet.setType(EntityUtils.getById(types, PetType.class, 2));
    pet.setBirthDate(new DateTime());
    owner6.addPet(pet);
    assertThat(owner6.getPets().size()).isEqualTo(found + 1);

    this.clinicService.savePet(pet);
    this.clinicService.saveOwner(owner6);

    owner6 = this.clinicService.findOwnerById(6);
    assertThat(owner6.getPets().size()).isEqualTo(found + 1);
    // checks that id has been generated
    assertThat(pet.getId()).isNotNull();
}
  

Issues with typical automated tests

• Many technical and irrelevant details
• Point of the test often hard to grasp
• Code duplication
• Can only be read by developers
• Cannot be used as documentation

Behavior-Driven Development

Feature Files (Gherkin)

Feature: Finding Owners

  Scenario: Owners can be found by last name

    Given an owner with last name "John"
     When searching for "John"
     Then exactly the given owner is found
        

Step Implementation (Java)

public class CustomerStepdefs {
    @Given("an owner with last name (.*)")
    public void anOwnerWithLastName(String lastName) { ... }

    @When("searching for (.*)")
    public void searchingFor(String lastName) { ... }

    @Then("exactly the given owner is found")
    public void exactlyTheGivenOwnerIsFound() { ... }
}
        

Cost of Tools like Cucumber

Jez Humble and David Farley

BDD Frameworks for Java

• Cucumber: Plain Text + Java
• JBehave: Plain Text + Java
• Concordion: HTML + Java
• Fitness: Wiki + Java
• Spock: Groovy
• ScalaTest: Scala
• Jnario: Xtend
• Serenity (similar concepts as JGiven)

Stack Overflow?

Goals

• Developer friendly (low maintancence overhead)
• Readable tests in in Given-When-Then-Form (BDD)
• Reusability of test code
• Readable by domain experts

Scenarios in JGiven

import org.junit.Test;
import com.tngtech.jgiven.junit.ScenarioTest;

public class FindOwnerTest
       extends ScenarioTest<GivenOwner, WhenSearching, ThenOwner> {

    @Test
    public void owners_can_be_found_by_last_name() {

        given().an_owner_with_last_name("John");

        when().searching_for("John");

        then().exactly_the_given_owner_is_found();
    }
}
      

Stage Classes

@JGivenStage // only needed when using Spring
public class WhenSearching extends Stage<WhenSearching> {

    @Autowired
    ClinicService clinicService;

    @ScenarioState
    Owner owner;

    public WhenSearching searching_for(String name) {
       owner = this.clinicService.findOwnerByName(name);
       return this;
    }
}
          

Two Layers

Steve Freeman and Nat Pryce

Console Output

  Owners can be found by last name

    Given an owner with last name "John"
     When searching for "John"
     Then exactly the given owner is found

HTML5-App

• jgiven.org/jgiven-report/html5/ (Local)
• janschaefer.github.io/xke-jgiven/ (Local)

Practical Experience

• 3 years of experience in a large Java Enterprise project (up to 70 developers)
• Over 3000 Scenarios
• Readability and reusability of test code has been greatly improved
• Maintanence costs of automated tests have been reduced (no hard numbers)
• Developers and domain experts work together on the scenarios
• Well accepted by developers
• Easy to learn by new developers

Additional Features

Parameterized Steps

      given().a_customer_with_name( "John" );
          

      Given a customer with name John
          

Parameters within the Sentence?

      Given there are 5 coffees left
          

$ to the rescue!

      given().there_are_$_coffees_left( 5 );
          

Parameterized Scenarios

@Test
@DataProvider({
   "1, 0",
   "3, 2",
   "5, 4"})
public void the_stock_is_reduced_when_a_book_is_ordered( int initial,
                                                         int left ) {
   given().a_customer()
       .and().a_book()
       .with().$_items_on_stock( initial );

   when().the_customer_orders_the_book();

   then().there_are_$_items_left_on_stock( left );
}

Parameterized Scenarios

Console Output

 The stock is reduced when a book is ordered

   Given a customer
     And a book
    With <initial> items on stock
    When the customer orders the book
    Then there are <left> items left on stock

  Cases:

    | # | initial | left | Status  |
    +---+---------+------+---------+
    | 1 |       1 |    0 | Success |
    | 2 |       3 |    2 | Success |
    | 3 |       5 |    4 | Success |

Parameterized Scenarios

HTML Report

Parameter Formatting

• Default: toString()
• @Format( MyCustomFormatter.class )
• @Formatf( " -- %s -- " )
• @MyCustomFormatAnnotation

Example

@OnOff

@Format( value = BooleanFormatter.class, args = { "on", "off" } )
@Retention( RetentionPolicy.RUNTIME )
@interface OnOff {}

public SELF the_machine_is_$( @OnOff boolean onOrOff ) { ... }

given().the_machine_is_$( false );

Given the machine is off

Tables as Parameters

SELF the_following_books_are_on_stock( @Table String[][] stockTable ) {
   ...
}

• All iterable types are supported

Tables as Parameters

@Test
public void ordering_a_book_reduces_the_stock() {

    given().the_following_books_on_stock(new String[][]{
        {"id", "name", "author", "stock"},
        {"1", "The Hitchhiker's Guide to the Galaxy", "Douglas Adams", "5"},
        {"2", "Lord of the Rings", "John Tolkien", "3"},
    });

    when().a_customer_orders_book("1");

    then().the_stock_looks_as_follows(new String[][]{
        {"id", "name", "author", "stock"},
        {"1", "The Hitchhiker's Guide to the Galaxy", "Douglas Adams", "4"},
        {"2", "Lord of the Rings", "John Tolkien", "3"},
    });

}

Tables as Parameters

Console Output

Ordering a book reduces the stock

   Given the following books on stock

     | id | name                                 | author        | stock |
     +----+--------------------------------------+---------------+-------+
     |  1 | The Hitchhiker's Guide to the Galaxy | Douglas Adams |     5 |
     |  2 | Lord of the Rings                    | John Tolkien  |     3 |

    When a customer orders book 1
    Then the stock looks as follows

     | id | name                                 | author        | stock |
     +----+--------------------------------------+---------------+-------+
     |  1 | The Hitchhiker's Guide to the Galaxy | Douglas Adams |     4 |
     |  2 | Lord of the Rings                    | John Tolkien  |     3 |

Tables as Parameters

HTML Report

@BeforeScenario und @AfterScenario

public class GivenSteps extends Stage<GivenSteps> {

    @ProvidedScenarioState
    File temporaryFolder;

    @BeforeScenario
    void setupTemporaryFolder() {
	    temporaryFolder = ...
    }

    @AfterScenario
    void deleteTemporaryFolder() {
        temporaryFolder.delete();
    }
}
          

@ScenarioRule

public class TemporaryFolderRule {
    File temporaryFolder;

    public void before() {
        temporaryFolder = ...
    }

    public void after() {
        temporaryFolder.delete();
    }
}

public class GivenSteps extends Stage<GivenSteps> {
    @ScenarioRule
    TemporaryFolderRule rule = new TemporaryFolderRule();
}
          

@AfterStage, @BeforeStage

public class GivenCustomer extends Stage<GivenSteps> {
    CustomerBuilder builder;

    @ProvidedScenarioState
    Customer customer;

    public void a_customer() {
        builder = new CustomerBuilder();
    }

    public void with_age(int age) {
        builder.withAge(age);
    }

    @AfterStage
    void buildCustomer() {
        customer = builder.build();
    }
}
          

Tags

@Test @FeatureEmail
void the_customer_gets_an_email_when_ordering_a_book() {
   ...
}

@Test @Story( "ABC-123" )
void the_customer_gets_an_email_when_ordering_a_book() { ... }

@Pending

• Marks the whole scenario or single steps as not implemented yet
• Steps are skipped, but appear in the report

HTML Report

@Hidden

• Marks methods to not appear in the report
• Useful for technically required methods

@Hidden
public SELF doSomethingTechnical() { ... }

Extended Step Descriptions

@ExtendedDescription("The Hitchhiker's Guide to the Galaxy, "
                   + "by default the book is not on stock" )
public SELF a_book() { ... }

HTML Report

Attachments

public class Html5ReportStage {
    @ExpectedScenarioState
    protected CurrentStep currentStep; // provided by JGiven

    protected void takeScreenshot() {
        String base64 = ( (TakesScreenshot) webDriver )
            .getScreenshotAs( OutputType.BASE64 );
        currentStep.addAttachment(
            Attachment.fromBase64( base64, MediaType.PNG )
                      .withTitle( "Screenshot" ) );
    }
}

HTML Report

Summary

• Automated tests are required for Continuous Delivery
• Test code is a significant cost factor
• Typical test code has a lot of issues
• Existing BDD tools either have an overhead or are not Java

JGiven

• Developer friendly
• Highly modular and reusable test code
• Just Java, no further language is needed
• Easy to integrate into existing test infrastructures
• Open Source (Apache 2 Licence)
• Maven and Jenkins plugins
• Domain experts can read scenario reports
• Domain experts can not write scenarios
  (But they can write them in any other format and developers can easily translate them to JGiven)

Take your tests to the next level!