[et_pb_section bb_built=”1″][et_pb_row][et_pb_column type=”4_4″][et_pb_text _builder_version=”3.12.1″]
[Author’s note: This blog post accompanies the Seattle React.js Meetup we hosted on September 19, 2018, watch the recording]
At Qumulo, we build a scale-out distributed file system, and building such a system, as it turns out, is hard.
We have layers upon layers of subsystems, all working with each other to form the product we call Qumulo File Fabric. As a whole, the system is complex, and our customers depend on it to store their valuable data.
As Qumulo engineers, we work in a continuous integration (CI) environment, and automated tests are essential to ensuring that all the code we develop works correctly. We can be confident our customers’ data is safe even as we actively change and develop the software.
From bottom to top, we care deeply about testing. And at the top of the stack, the UI is a very unique place where user requirements are directly exposed. We have iterated through a lot of different ways to test our React/Redux UI. I’d like to share where our experience took us.
Automated tests are a worthy investment
The act of writing tests is expensive. My past experience have been that, even though there is a business goal to have “good test coverage,” there is also the pressure to deliver more quickly, and we end up writing few automated smoke tests here and there as we happily paid money to QA to test manually. Heck, they even populated and managed the bug database for us!
At Qumulo, I learned that automated testing is a great investment for the long run. We change and refactor code over time, and every time we change the code, the tests protect us from inadvertently changing the behavior of the system. With our comprehensive test coverage, our current migration from ES6 to TypeScript as an example has been coming along smoothly, so we can focus on the migration rather than manually checking that the code migration is done correctly.
In the JavaScript world, the software libraries can change very quickly, and sometimes upgrading a library is very disruptive. Our suite of tests allow us to have peace of mind when we upgrade libraries, because we expect breaking changes to also break our UI tests. Our years of investment in testing allows us to get these upgrade done quickly so we can focus on the development, and we save a lot of time and energy with every code change.
The test pyramid
Today, we write our UI code in React and Redux. At the smallest level, we can imagine we need unit tests for these pieces — React components, Redux reducers, action creators, selectors.
We put these together to form pages, then we put the pages together to build the app. Our integration tests follow the same structure: we write page tests to verify that the user flow through a page is as expected, and we write end-to-end system tests to make sure our entire system works together. This fits very well with the test pyramid, conceptualized by Martin Fowler in 2012.
The test pyramid suggests that we write a lot of unit tests, some page tests, and a few end-to-end tests. When the unit tests fail, they should get us very close to the line of code that failed; the page tests should fail when our React and Redux parts are making incorrect assumptions about each other; and the end-to-end tests should fail if our UI is making incorrect assumptions about our backend.
React unit testing
In the past, we tested our React components in a very naive way. We rendered the component in the browser, and checked the DOM to see if it rendered properly.
As our codebase grew, dependencies became a big problem. For example, if a commonly reused component is updated to access the Redux store, all tests for its users will likely need to be updated. One solution might be to test everything with a Redux store provided, but that increases our test scope (we almost always want to mock out the Redux store as dependency injection). Also, these test failures do not aide us in our development. The Redux store is provided at the application level, so these test failures point to a bug in the test and not in our product, which means we spend our time maintaining tests.
We had to learn to get very clear on what a unit is. In a game I developed to illustrate React best practices and testing, the components are layered as such:
Let’s consider unit tests for ClickGame:</var/www/wordpress> the orange arrows are the inputs and outputs. When we consider this diagram closely, we realize that if we use a , the inputs to a component are props and events, and the outputs are shallow rendering and event props. We can then focus on manipulating the props and events, and verify the shallow rendering and events generated:
import * as React from "react";
import { createRenderer } from "react-test-renderer/shallow";
describe("ClickGame", function() {
beforeEach(function() {
this.shallowRender = (gameState: ClickGameState) => {
this.resetGameSpy = jasmine.createSpy("resetGame");
this.shallowRenderer.render(
<ClickGame.WrappedComponent
gameState={gameState}
resetGame={this.resetGameSpy}
/>
);
return this.shallowRenderer.getRenderOutput();
};
this.shallowRenderer = createRenderer();
});
afterEach(function() {
this.shallowRenderer.unmount();
testutil.verifyNoLogOutput();
});
describe("not started", function() {
beforeEach(function() {
this.gameState = new ClickGameState();
this.gameState.gameState = GameState.NotStarted;
this.renderOutput = this.shallowRender(this.gameState);
});
it("should render start game prompt", function() {
expect(this.renderOutput).toHaveShallowRendered(
<h2>A game of clicks: {"Click to start"}</h2>
);
});
it("should reset game on button click", function() {
const buttons = scryDomNodesInShallowRenderTreeByType(
this.renderOutput,
"button"
);
expect(buttons.length).toEqual(1);
const button = buttons[0] as
React.ReactHTMLElement<HTMLButtonElement>;
button.props.onClick(jasmine.createSpy("buttonEvent") as any);
expect(this.resetGameSpy).toHaveBeenCalledWith();
});
});
});
We used react-redux to help us connect the component to Redux, and connect() provides a static member WrappedComponent which is the original component we implemented. Unit testing WrappedComponent directly allows us to mock out Redux by directly accessing the props managed by react-redux.
Redux unit testing
Testing the basic Redux is pretty straightforward.
[/et_pb_text][et_pb_accordion _builder_version=”3.12.1″][et_pb_accordion_item _builder_version=”3.12.1″ title=”Action Creator” use_background_color_gradient=”off” background_color_gradient_start=”#2b87da” background_color_gradient_end=”#29c4a9″ background_color_gradient_type=”linear” background_color_gradient_direction=”180deg” background_color_gradient_direction_radial=”center” background_color_gradient_start_position=”0%” background_color_gradient_end_position=”100%” background_color_gradient_overlays_image=”off” parallax=”off” parallax_method=”on” background_size=”cover” background_position=”center” background_repeat=”no-repeat” background_blend=”normal” allow_player_pause=”off” background_video_pause_outside_viewport=”on” text_shadow_style=”none” box_shadow_style=”none” text_shadow_horizontal_length=”0em” text_shadow_vertical_length=”0em” text_shadow_blur_strength=”0em”]
Verify the action creator returns the correct action with expected payload:
describe("clickGameClick", function() {
it("should create a click action", function() {
expect(ClickGameActions.clickGameClick(2, 3)).toEqual({
payload: {
col: 3,
row: 2
},
type: ClickGameActionType.Click
});
});
it("should throw if row index is invalid", function() {
expect(() => ClickGameActions.clickGameClick(-1, 0)).toThrow();
});
it("should throw if col index is invalid", function() {
expect(() => ClickGameActions.clickGameClick(0, -1)).toThrow();
});
});
[/et_pb_accordion_item][et_pb_accordion_item _builder_version=”3.12.1″ title=”Reducer” use_background_color_gradient=”off” background_color_gradient_start=”#2b87da” background_color_gradient_end=”#29c4a9″ background_color_gradient_type=”linear” background_color_gradient_direction=”180deg” background_color_gradient_direction_radial=”center” background_color_gradient_start_position=”0%” background_color_gradient_end_position=”100%” background_color_gradient_overlays_image=”off” parallax=”off” parallax_method=”on” background_size=”cover” background_position=”center” background_repeat=”no-repeat” background_blend=”normal” allow_player_pause=”off” background_video_pause_outside_viewport=”on” text_shadow_style=”none” box_shadow_style=”none” text_shadow_horizontal_length=”0em” text_shadow_vertical_length=”0em” text_shadow_blur_strength=”0em”]
Given an initial state and a relevant action, verify reducer returns the expected state:
it("should reduce a Click to DoNotClick", function() {
this.startState.getButton(1, 2).state = ButtonGameState.Click;
const newState = clickGameReducer(
this.startState,
ClickGamePlainActions.clickGameClick(1, 2)
);
expect(newState).not.toBe(
this.startState,
"need to have created a new state object"
);
expect(newState.gameState).toEqual(GameState.Started);
expect(newState.getButton(1, 2).state).toEqual(ButtonGameState.DoNotClick);
expect(newState.score).toEqual(1);
});
[/et_pb_accordion_item][et_pb_accordion_item _builder_version=”3.12.1″ title=”Selector” use_background_color_gradient=”off” background_color_gradient_start=”#2b87da” background_color_gradient_end=”#29c4a9″ background_color_gradient_type=”linear” background_color_gradient_direction=”180deg” background_color_gradient_direction_radial=”center” background_color_gradient_start_position=”0%” background_color_gradient_end_position=”100%” background_color_gradient_overlays_image=”off” parallax=”off” parallax_method=”on” background_size=”cover” background_position=”center” background_repeat=”no-repeat” background_blend=”normal” allow_player_pause=”off” background_video_pause_outside_viewport=”on” text_shadow_style=”none” box_shadow_style=”none” text_shadow_horizontal_length=”0em” text_shadow_vertical_length=”0em” text_shadow_blur_strength=”0em”]
Given a Redux state, verify selector returns the right value:
it("should return falsy if button cannot be found in a ClickGameState", function() {
const state: HandbookState = {
clickGame: null
};
expect(ClickGameSelector.getButtonState(state, 0, 0)).toBeFalsy();
});
it("should return the click game state from the global handbook state", function() {
const clickGameState = jasmine.createSpy("ClickGame state") as any;
const state: HandbookState = {
clickGame: clickGameState
};
expect(ClickGameSelector.getClickGameState(state))
.toBe(clickGameState);
});
[/et_pb_accordion_item][et_pb_accordion_item _builder_version=”3.12.1″ title=”Thunked action Creator” use_background_color_gradient=”off” background_color_gradient_start=”#2b87da” background_color_gradient_end=”#29c4a9″ background_color_gradient_type=”linear” background_color_gradient_direction=”180deg” background_color_gradient_direction_radial=”center” background_color_gradient_start_position=”0%” background_color_gradient_end_position=”100%” background_color_gradient_overlays_image=”off” parallax=”off” parallax_method=”on” background_size=”cover” background_position=”center” background_repeat=”no-repeat” background_blend=”normal” allow_player_pause=”off” background_video_pause_outside_viewport=”on” text_shadow_style=”none” box_shadow_style=”none” text_shadow_horizontal_length=”0em” text_shadow_vertical_length=”0em” text_shadow_blur_strength=”0em”]
Thunked action creator helps us dispatch multiple actions as we wait for an asynchronous function to complete, usually a REST API call.
In the spirit of unit testing, we assume the plain action creators are already tested. For the thunked action creator, we control the result of the asynchronous function and expect the correct set of actions to be dispatched. We can do this using a mock Redux store. In this example, the asynchronous function is a JavaScript setTimeout</var/www/wordpress>:
describe("clickGameStartRound", function() {
beforeEach(function() {
jasmine.clock().install();
this.gameState = new ClickGameState();
spyOn(ClickGameSelector, "getClickGameState")
.and.returnValue(this.gameState);
this.mockStore = new ReduxMockStore({});
});
afterEach(function() {
jasmine.clock().uninstall();
});
it("should not dispatch if the game has not started", function() {
this.gameState.gameState = GameState.NotStarted;
this.mockStore.dispatch(ClickGameActions.clickGameNewRound());
expect(this.mockStore.getActions())
.toEqual(
[],
"Expect round to not dispatch right away"
);
jasmine.clock().tick(3001);
expect(this.mockStore.getActions()).toEqual([], "Expected no new rounds");
});
it("should dispatch new round every 3 seconds when the game has started", function() {
this.gameState.gameState = GameState.Started;
this.mockStore.dispatch(ClickGameActions.clickGameNewRound());
expect(this.mockStore.getActions()).toEqual(
[], "Expect round to not dispatch right away");
jasmine.clock().tick(3001);
expect(this.mockStore.getActions()).toEqual(
[
{
type: ClickGameActionType.NewRound
}
],
"Expect a new round to be dispatched after 3 seconds"
);
this.mockStore.clearActions();
jasmine.clock().tick(3001);
expect(this.mockStore.getActions()).toEqual(
[
{
type: ClickGameActionType.NewRound
}
],
"Expect a new round to be dispatched after 6 seconds"
);
});
});
[/et_pb_accordion_item][/et_pb_accordion][et_pb_text _builder_version=”3.12.1″]
Page-level integration tests
Now that all of our units are tested, we need to make sure they fit together correctly. The goal of the page-level integration tests (or “page tests” for short), are to verify that React components are interacting correctly, and that React and Redux are working together correctly.
Tools we need
There were two problems we need to solve to write page tests.
We need a way to generally mock out our REST API calls. We created the AjaxManager which intercepts all calls to $.ajax and provides methods to make a request wait, succeed, or fail.
We also need a way to programmatically wait for our UI to change before taking the next step in the test. We created TestStepBuilder, which is a tool that that allows us to write tests that wait for conditions to be met before taking more steps.
In the demo game, the asynchronous action is taken on a timer, so there is no example of the AjaxManager here, but it makes use of the TestStepBuilder to step through the tests:
beforeAll(function() {
this.handbookPage = new HandbookPage();
this.handbookPage.render(done);
}
afterAll(function() {
this.handbookPage.cleanUp();
});
it("should start the game of clicking when click on the start button", function(done) {
new TestStepBuilder()
.step("Verify game has not started", () => {
expect(this.handbookPage.getGameStatus()).toEqual("Click to start");
})
.waitFor("Start button", () => {
return this.handbookPage.findStartButton();
})
.step("Click the start button", () => {
this.handbookPage.findStartButton().click();
})
.waitFor("ClickGameTable to render", () => {
return this.handbookPage.findClickGameTable();
})
.step("Verify game is in progress", () => {
expect(this.handbookPage.getGameStatus()).toEqual("In progress...");
})
.run(done);
});
it("should continue the game of clicking when click on a green button", function(done) {
new TestStepBuilder()
.step("Click a green button", () => {
expect(this.handbookPage.getScore()).toEqual(0, "score should be 0");
const $greenButtons = this.handbookPage.$findGreenButtons();
expect($greenButtons.length).toBeGreaterThan(
0,
"should have at least 1 green button at game reset"
);
$greenButtons[0].click();
})
.waitFor("score to go up", () => this.handbookPage.getScore() > 0)
.step("Verify score and game status", () => {
expect(this.handbookPage.getGameStatus()).toEqual("In progress...");
expect(this.handbookPage.getScore()).toEqual(1, "score should be 1");
}).run(done);
});
it("should end the game and show restart button when click on a red button", function(done) {
new TestStepBuilder()
.step("Click a red button", () => {
expect(this.handbookPage.getScore()).toEqual(1, "score should be 1");
const $redButtons: JQuery = this.handbookPage.$findRedButtons();
expect($redButtons.length).toBeGreaterThan(
0,
"should be at least one red button after green was clicked"
);
$redButtons[0].click();
})
.waitFor("Restart button to show", () => {
this.handbookPage.findRestartButton();
})
.step("Verify that the game is over", () => {
expect(this.handbookPage.getScore()).toEqual(1, "score should stay 1");
expect(this.handbookPage.getGameStatus()).toEqual("GAME OVER");
}).run(done);
});
Page object design pattern
In the sample code above, you will notice that the code is agnostic to the page implementation.
We made use of a design pattern documented by Selenium called page object design pattern. In the sample code, HandbookPage is a page object which wraps our implementation of the React handbook page component, and we access the UI only via the page object in our tests.
This decoupling has two advantages.
- It makes our tests easier to read.
- If we ever change the implementation of the page, we only need to update the page object and not the tests.
This way the page tests only describe how the page should be tested, and the implementation details are encapsulated in the page object.
End-to-end system tests
In end-to-end UI system tests, we spin up a Qumulo cluster and exercise our UI. We use the same tools as in our page tests, simulating user actions and inspecting the UI using page objects, and work through test steps using TestStepBuilder.
The goal of the systest is to verify that the API is behaving correctly by exercising the UI. There tends to be a lot of overlap between the page tests and end-to-end tests. Usually, the page tests focus on all the different possible asynchronous events (such as network disconnects), whereas the systest specifically checks that the UI and REST API are making the correct assumptions about one another.
Tests help us move fast
Tests help us move fast because they help us produce less bugs, so our workflow gets interrupted less by bug fixes, which means we focus on more development.
Over the last year, we moved away from a model with a layer of unit-ish integration tests and a layer of system tests. In this old model, the “unit” tests had too many dependencies which makes them fragile, while the system tests took too much effort to run regularly during development.
We learned a lot of lessons from this Google Testing Blog post, which describes all the reasons why we moved to the new model with three layers of tests. Today when a unit test fails, it gives us very specific information on what was broken. The TypeScript compiler ensures that our code is syntactically and semantically fitting together correctly, and the page tests check that our React components and Redux code have the correct assumptions about each other. This leaves the system tests to have a much smaller job of ensuring the REST API contracts rather than trying to verify correctness in our UI systems.
Good testing practices have helped us move faster because we are confident in our code as it grows and evolves over time, and the way we break up the tests today makes them easier to write and maintain while giving us much more targeted information when tests fail. We continue to drive for better ways to develop our code, and we hope to share more as we continue our learning!
[/et_pb_text][/et_pb_column][/et_pb_row][/et_pb_section]
