Automated tests are important. Without them, programmers waste a huge amount of time manually checking and fixing their code.
Unfortunately, many automated tests also waste a huge amount of time. The easy, obvious way to write tests is to make broad tests that are automated versions of manual tests. But they’re flaky and slow.
Folks in the know use mocks and spies (I say “mocks” for short in this article) to write isolated interaction-based tests. Their tests are reliable and fast, but they tend to “lock in” implementation, making refactoring difficult, and they have to be supplemented with broad tests. It’s also easy to make poor-quality tests that are hard to read, or end up only testing themselves.
Bad tests are a sign of bad design, so some people use techniques such as Hexagonal Architecture and functional core, imperative shell to separate logic from infrastructure. (Infrastructure is code that involves external systems or state.) It fixes the problem… for logic. But infrastructure is often left untested, and it requires architectural changes that are out of reach for people with existing code.
This pattern language1 describes a fourth option. It avoids all the above problems: it doesn’t use broad tests, doesn’t use mocks, doesn’t ignore infrastructure, and doesn’t require architectural changes. It has the speed, reliability, and maintainability of unit tests and the power of broad tests. But it’s not without tradeoffs of its own.
The patterns combine sociable, state-based tests with a novel infrastructure technique called “Nullables.” At first glance, Nullables look like test doubles, but they’re actually production code with an “off” switch. And that’s the tradeoff: do you want that in your production code? Your answer determines whether this pattern language is for you.
The rest of the article goes into detail. Don’t be intimidated by its size. It’s broken up into bite-sized pieces with lots of code examples.
“Testing Without Mocks” Training
Contents:
- Examples
- Goals
- Tradeoffs
- Foundational Patterns
- Architectural Patterns
- Logic Patterns
- Infrastructure Patterns
- Nullability Patterns
- Legacy Code Patterns
- Conclusion
Examples
Here’s an example of testing a simple command line application. The application reads a string from the command line, encodes it using ROT-13, and outputs the result.
The production code uses the optional A-Frame Architecture pattern. App
is the application entry point. It depends on Rot13
, a Logic class, and CommandLine
, an Infrastructure class. Additional patterns are mentioned in the source code.
// Example production code (JavaScript + Node.js)
import CommandLine from "./infrastructure/command_line"; // Infrastructure Wrapper
import * as rot13 from "./logic/rot13";
export default class App {
constructor(commandLine = CommandLine.create()) { // Parameterless Instantiation
this._commandLine = commandLine;
}
run() {
const args = this._commandLine.args();
if (args.length === 0) { // Tested by Test #2
this._commandLine.writeOutput("Usage: run text_to_transformn");
return;
}
if (args.length !== 1) { // Tested by Test #3
this._commandLine.writeOutput("too many argumentsn");
return;
}
// Tested by Test #1
const input = args[0]; // Logic Sandwich
const output = rot13.transform(input);
this._commandLine.writeOutput(output + "n");
}
};
The tests of App
look like end-to-end integration tests, but they’re actually unit tests. Technically, they’re Narrow, Sociable tests, which means they’re unit tests that execute code in dependencies.
As narrow tests, the tests only care about testing App.run()
. Each of the dependencies is expected to have tests of their own, which they do.
The tests use a Nullable CommandLine
to throw away stdout
and Configurable Responses to provide pre-configured command-line arguments. They also use Output Tracking to see what would have been written to stdout
.
// Example tests (JavaScript + Node.js)
import assert from "assert";
import CommandLine from "./infrastructure/command_line";
import App from "./app";
describe("App", () => {
// Test #1
it("reads command-line argument, transform it with ROT-13, and writes result", () => {
const { output } = run({ args: [ "my input" ] }); // Signature Shielding, Configurable Responses
assert.deepEqual(output.data, [ "zl vachgn" ]; // Output Tracking
});
// Test #2
it("writes usage when no argument provided", () => {
const { output } = run({ args: [] }); // Signature Shielding, Configurable Responses
assert.deepEqual(output.data, [ "Usage: run text_to_transformn" ]); // Output Tracking
});
// Test #3
it("complains when too many command-line arguments provided", () => {
const { output } = run({ args: [ "a", "b" ] }); // Signature Shielding, Configurable Responses
assert.deepEqual(output.data, [ "too many argumentsn" ]); // Output Tracking
});
function run({ args = [] } = {}) { // Signature Shielding
const commandLine = CommandLine.createNull({ args }); // Nullable, Infrastructure Wrapper, Configurable Responses
const output = commandLine.trackOutput(); // Output Tracking
const app = new App(commandLine);
app.run();
return { output }; // Signature Shielding
}
});
If you’re familiar with mocks, you might assume CommandLine
is a test double. But it’s actually production code with an “off” switch and the ability to monitor its output.
// Example Nullable infrastructure wrapper (JavaScript + Node.js)
import EventEmitter from "node:events";
import OutputTracker from "output_tracker";
const OUTPUT_EVENT = "output";
export default class CommandLine {
static create() {
return new CommandLine(process); // 'process' is a Node.js global
}
static createNull({ args = [] } = {}) { // Parameterless Instantiation, Configurable Responses
return new CommandLine(new StubbedProcess(args)); // Embedded Stub
}
constructor(proc) {
this._process = proc;
this._emitter = new EventEmitter(); // Output Tracking
}
args() {
return this._process.argv.slice(2);
}
writeOutput(text) {
this._process.stdout.write(text);
this._emitter.emit(OUTPUT_EVENT, text); // Output Tracking
}
trackOutput() { // Output Tracking
return OutputTracker.create(this._emitter, OUTPUT_EVENT);
}
};
// Embedded Stub
class StubbedProcess {
constructor(args) {
this._args = args; // Configurable Responses
}
get argv() {
return [ "nulled_process_node", "nulled_process_script.js", ...this._args ];
}
get stdout() {
return {
write() {}
};
}
}
The patterns shine in more complex code that has multiple layers of dependencies. Find more examples here:
-
Simple example. The complete source code for the above example. (JavaScript or
TypeScript with Node.js) -
Complex example. The blinged-out version of the above example. A web application and microservice that performs ROT-13 encoding. Production-grade code with error handling, logging, timeouts, and request cancellation. (JavaScript and Node.js)
-
TDD Lunch & Learn Screencast. A series of one-hour webinars that demonstrate how to use the patterns. (JavaScript and Node.js)
-
Nullables Livestream. A series of three-hour livestreams with James Shore and Ted M. Young. They pair on applying the patterns to an existing web application. (Java and Spring Boot)
Goals
This pattern language was created to satisfy these goals:
-
No broad tests required. The test suite consists entirely of “narrow” tests that are focused on specific concepts. Although broad integration tests can be added as a safety net, their failure indicates a gap in the main test suite.
-
Easy refactoring. Object interactions are considered implementation to be encapsulated, not behavior to be tested. Although the consequences of object interactions are tested, the specific method calls aren’t. This allows structural refactorings to be made without breaking tests.
-
Readable tests. Tests follow a straightforward “arrange, act, assert” structure. They describe the externally-visible behavior of the unit under test, not its implementation. They can act as documentation for the unit under test.
-
No magic. Tools that automatically remove busywork, such as dependency-injection frameworks and auto-mocking frameworks, are not required.
-
Fast and deterministic. The test suite only executes “slow” code, such as network calls or file system requests, when that behavior is explicitly part of the unit under test. Such tests are organized so they produce the same results on every test run.
Experience has revealed these additional benefits:
-
Faster than mocking frameworks. In a head-to-head comparison, tests using these patterns were 2–3 orders of magnitude faster than tests using a mocking framework. (Comparison code here.)
-
Simple test setup. Test setup is straightforward and easy to encapsulate in helper methods.
-
High reusability. The most complicated code needed for these patterns is also the most generic and reusable.
-
In-memory infrastructure testing. High-level infrastructure wrappers, such as a client for a specific web service, can be tested without network calls or complicated setup. (Example test.)
-
Edge case support. It’s easy to test complex edge cases, such as error conditions and timeouts. (Example tests.)
-
Legacy code compatibility. The patterns are completely compatible with mocks and other test doubles, and can even be used together in the same test. Legacy code can be converted incrementally without impacting existing code.
Tradeoffs
Nothing’s perfect. These are the downsides of using this pattern language:
-
Changes to production code. The patterns require you to modify your production code, particularly for infrastructure classes. Although the modifications are usable in production, and have production use cases, many of the changes will only be used by tests.
-
Hand-written stub code. Some third-party infrastructure code has to be mimicked with hand-written stub code. It can’t be auto-generated and takes extra time to write. However, the results are highly reusable.
-
Multiple test failures. Although tests are written to focus on specific concepts, the units under test execute code in their dependencies. (Jay Fields coined the term “sociable tests” for this behavior.) This can result in multiple tests failing when a bug is introduced.
Foundational Patterns
Start here. These patterns establish the ground rules.
- Narrow Tests
- State-Based Tests
- Overlapping Sociable Tests
- Smoke Tests
- Zero-Impact Instantiation
- Parameterless Instantiation
- Signature Shielding
Narrow Tests
Broad tests, such as end-to-end tests, tend to be slow and brittle. They’re complicated to read and write, often fail randomly, and take a long time to run. Therefore:
Instead of using broad tests, use narrow tests. Narrow tests check a specific function or behavior, not the system as a whole. Unit tests are a common type of narrow tests.
When testing infrastructure, use Narrow Integration Tests. When testing pure logic, use the Logic Patterns. When testing code that has infrastructure dependencies, use Nullables.
To ensure your code works as a whole, use State-Based Tests and Overlapping Sociable Tests.
State-Based Tests
Mocks and spies result in “interaction-based” tests that check how the code under test uses its dependencies. However, they can be hard to read, and they tend to “lock in” your dependencies, which makes structural refactorings difficult. Therefore:
Use state-based tests instead of interaction-based tests. A state-based test checks the output or state of the code under test, without any awareness of its implementation. For example, given the following production code:
// Production code to describe phase of moon (JavaScript)
import * as moon from "astronomy";
import { format } from "date_formatter";
export function describeMoonPhase(date) {
const visibility = moon.getPercentOccluded(date);
const phase = moon.describePhase(visibility);
const formattedDate = format(date);
return `The moon is ${phase} on ${formattedDate}.`;
}
A state-based test would pass in a date and check the result, like this:
// State-based test of describeMoonPhase() (JavaScript)
import { describeMoonPhase } from "describe_phase";
it("describes phase of moon", () => {
const dateOfFullMoon = new Date("8 Dec 2022"); // a date when the moon was actually full
const description = describeMoonPhase(dateOfFullMoon);
assert.equal(description, "The moon is full on December 8th, 2022.";
});
In contrast, an interaction-based test would check how each dependency was used, like this:
// Interaction-based test of describeMoonPhase() (JavaScript and fictional mocking framework)
const moon = mocker.mockImport("astronomy");
const { format } = mocker.mockImport("date_formatter");
const { describeMoonPhase } = mocker.importWithMocks("describe_phase");
it("describes phase of moon", () => {
const date = new Date(); // specific date doesn't matter
mocker.expect(moon.getPercentOccluded).toBeCalledWith(date).thenReturn(999);
mocker.expect(moon.describePhase).toBeCalledWith(999).thenReturn("PHASE");
mocker.expect(format).toBeCalledWith(date).thenReturn("DATE");
const description = describeMoonPhase(date);
mocker.verify();
assert.equal(description, "The moon is PHASE on DATE");
};
State-based tests naturally result in Overlapping Sociable Tests. To use state-based tests on code with infrastructure dependencies, use the Nullability Patterns.
Overlapping Sociable Tests
Tests using mocks and other test doubles isolate the code under test by replacing its dependencies. This requires broad tests to confirm that the system works as a whole, but we don’t want to use broad tests. Therefore:
When testing the interactions between an object and its dependencies, use the code under test’s real dependencies. Don’t test the dependencies’ behavior, but do test that the code under test uses its dependencies correctly. This happens naturally when using State-Based Tests.
For example, the following test checks that describeMoonPhase
uses its Moon
and format
dependencies correctly. If they don’t work the way describeMoonPhase
thinks they do, the test will fail.
// Example of sociable tests (JavaScript)
// Test code
it("describes phase of moon", () => {
const dateOfFullMoon = new Date("8 Dec 2022");
const description = describeMoonPhase(dateOfFullMoon);
assert.equal(description, "The moon is full on December 8th, 2022.";
};
// Production code
describeMoonPhase(date) {
const visibility = moon.getPercentOccluded(date);
const phase = moon.describePhase(visibility);
const formattedDate = format(date);
return `The moon is ${phase} on ${formattedDate}.`;
}
Write Narrow Tests that are focused on the behavior of the code under test, not the behavior of its dependencies. Each dependency should have its own thorough set of Narrow Tests. For example, don’t test all phases of the moon in your describeMoonPhase()
tests, but do test them in your Moon
tests. Similarly, don’t check the intricacies of date formatting in your describeMoonPhase
tests, but do test them in your format(date)
tests.
In addition to checking how your code uses its dependencies, sociable tests also protect you against future breaking changes. Each test overlaps with dependencies’ tests and dependents’ tests, creating a strong linked chain of tests. This gives you the coverage of broad tests without their speed and reliability problems.
For example, imagine the dependency chain LoginController
→ Auth0Client
→ HttpClient
:
-
The
LoginController
tests checks thatLoginController
is correct, including how it usesAuth0Client
. (Auth0Client
in turn runsHttpClient
, but that isn’t explicitly checked by theLoginController
tests.) -
The
Auth0Client
tests check thatAuth0Client
is correct, including how it usesHttpClient
. -
The
HttpClient
tests check thatHttpClient
is correct, including using Narrow Integration Tests to check how it communicates with HTTP servers. -
Together, they ensure the whole chain is checked. Even if
HttpClient
and its tests are changed intentionally, if that change breaksAuth0Client
, its tests would fail (and possibly theLoginController
tests, too). ChangingAuth0Client
’s behavior would similarly break theLoginController
tests.
In contrast, if the LoginController
tests stubbed or mocked out Auth0Client
, the chain would be broken. Changing Auth0Client
’s behavior would not break the LoginController
tests, because nothing would check how LoginController
used the real Auth0Client
.
To avoid manually constructing the entire dependency chain, use Parameterless Instantiation with Zero-Impact Instantiation. To isolate tests from changes in dependencies’ behavior, use Collaborator-Based Isolation. To prevent your tests from interacting with external systems and state, use Nullables. To catch breaking changes in external systems, use Paranoic Telemetry. For a safety net, use Smoke Tests.
Smoke Tests
Overlapping Sociable Tests are supposed to cover your entire system. But nobody’s perfect, and mistakes happen. Therefore:
Write one or two end-to-end tests that make sure your code starts up and runs a common workflow. For example, if you’re coding a web site, check that you can get an important page.
Don’t rely on smoke tests to catch errors. Your real test suite should consist of Narrow, Sociable tests. If the smoke tests catch something the rest of your tests don’t, fill the gap with more narrow tests.
Zero-Impact Instantiation
Overlapping Sociable Tests instantiate their dependencies, which in turn instantiate their dependencies, and so forth. If instantiating this web of dependencies takes too long or causes side effects, the tests could be slow, difficult to set up, or fail unpredictably. Therefore:
Don’t do significant work in constructors. Don’t connect to external systems, start services, or perform long calculations. For code that needs to connect to an external system or start a service, provide a connect()
or start()
method. For code that needs to perform a long calculation, consider lazy initialization. (But even complex calculations aren’t likely to be a problem, so profile before optimizing.)
Parameterless Instantiation
Overlapping Sociable Tests require your whole dependency tree to be instantiated, but multi-level dependency chains are difficult to set up in tests. Dependency injection (DI) frameworks work around the problem, but we don’t want to require such magic. Therefore:
Ensure all classes have a constructor or factory that doesn’t take any parameters. This factory (or constructor) should have sensible defaults that set up everything the object needs, including instantiating its dependencies. You can make these defaults overridable if desired. (If your language doesn’t support overridable defaults, use method overloading or an Options object, as shown in the Signature Shielding pattern.)
For some classes, particularly Value Objects, a parameterless factory isn’t a good idea in production, because people could forget to provide a necessary value. For example, an immutable Address
class should be constructed with its street, city, and so forth. Providing a default city could result in addresses that seemed to work, but actually had the wrong city.
In that case, provide a test-specific factory method with overridable defaults. Choose defaults that make the class work in as many situations as possible, and use a name such as createTestInstance()
to indicate that it’s only for tests. In your tests, pass in every parameter your test cares about, rather than relying on default values. That way, changes to the factory won’t break your tests.
// Test-specific factory using named, optional parameters (JavaScript)
class Address {
// Production constructor
constructor(street, city, state, country, postalCode) {
this._street = street;
this._city = city;
//...etc...
}
// Test-specific factory
static createTestInstance({
street = "Address test street",
city = "Address test city",
state = State.createTestInstance(),
country = Country.createTestInstance(),
postalCode = PostalCode.createTestInstance(),
} = {}) {
return new Address(street, city, state, country, postalCode);
}
}
This test-specific factory method is easiest to maintain if it’s located in the production code next to the real constructors. However, if you don’t want test-specific code in production, or if the logic gets complicated, you can use the Object Mother pattern to put it in a test-only helper module instead.
Signature Shielding
As you refactor your application, method signatures will change. If your code is well-designed, this won’t be a problem for production code, because most methods will only be used in a few places. But tests can have many duplicated method and constructor calls. When you change those methods or constructors, you’ll have a lot of busywork to update the tests. Therefore:
Provide helper functions to instantiate classes and call methods. Have these helper functions perform any setup your tests need rather than using your test framework’s before()
or setup()
functions.
Make the helper functions take optional parameters for customizing your setup and execution. If convenient in your programming language, return multiple optional values as well. This will allow you to expand your helper functions without breaking existing tests.
// Optional parameters and multiple return values (JavaScript)
// Example test
it("uses hosted page for authentication", () => {
const { url } = getLoginUrl({ // Use the helper function
host: "my.host",
clientId: "my_client_id",
callbackUrl: "my_callback_url"
});
assert.equal(url, "https://my.host/authorize?response_type=code&client_id=my_client_id&callback_url=my_callback_url");
});
// Example helper function
function getLoginUrl({
host = "irrelevant.host", // Optional parameters
clientId = "irrelevant_client_id",
clientSecret = "irrelevant_secret",
connection = "irrelevant_connection"
username = "irrelevant_username",
callbackUrl = "irrelevant_url",
} = {}) {
const client = new LoginClient(host, clientId, clientSecret, connection);
const url = client.getLoginUrl(username, callbackUrl);
return { client, url }; // Multiple return values
}
If you’re using a language without support for optional parameters, use method overloading or an “Options” object. If you’re using a language without support for multiple return values, you can return a simple data structure.
// Optional parameters and multiple return values (Java)
// Example tests
@Test
public void usesHostedPageForAuthentication() {
GetLoginUrlResult actual = getLoginUrl(new GetLoginUrlOptions() // Use the helper function and Options object
.withHost("my.host")
.withClientId("my_client_id")
.withCallbackUrl("my_callback_url")
);
assert.equal(actual.url, "https://my.host/authorize?response_type=code&client_id=my_client_id&callback_url=my_callback_url");
}
// Helper function using Options object and return data structure
private GetLoginUrlResult getLoginUrl(GetLoginUrlOptions options) {
LoginClient client = new LoginClient(options.host, options.clientId, options.secret, options.connection);
String url = client.getLoginUrl(options.username, options.callbackUrl);
return new GetLoginUrlResult(client, url);
}
// Options object
private static final class GetLoginUrlOptions {
public String host = "irrelevant.host";
public String clientId = "irrelevant_client_id";
public String clientSecret = "irrelevant_secret";
public String connection = "irrelevant_connection";
public String username = "irrelevant_username";
public String callbackUrl = "irrelevant_url";
GetLoginUrlOptions withHost(String host) {
this.host = host;
return this;
}
GetLoginUrlOptions withClientId(String clientId) {
this.clientId = clientId;
return this;
}
GetLoginUrlOptions withCallbackUrl(String url) {
this.callbackUrl = url;
return this;
}
}
// Return data structure
private static final class GetLoginUrlResult {
public LoginClient client;
public String url;
public GetLoginUrlResult(LoginClient client, String url) {
this.client = client;
this.url = url;
}
}
Architectural Patterns
Testing works best when you pay careful attention to the dependencies in your codebase. These architectural patterns help you do so. They aren’t required, but they’re useful.
A-Frame Architecture
Code without infrastructure dependencies is much easier to test than code that has infrastructure dependencies. However, a normal layered architecture puts infrastructure at the bottom of the dependency chain:
Application/UI | V Logic | V Infrastructure
Therefore:
Structure your application so that infrastructure and logic are peers under the application layer, with no dependencies between Infrastructure and Logic. Coordinate between them at the Application layer with a Logic Sandwich or Traffic Cop. Use Value Objects to pass data between the Logic and Infrastructure layers.
Application/UI Values / V V Logic Infrastructure
Build the Logic and Values layers using Logic Patterns. Build the Infrastructure layer using Infrastructure Patterns. Build the Application/UI layer with a Logic Sandwich or Traffic Cop, and use Nullables to test it.
Although A-Frame Architecture is a nice way to simplify application dependencies, it’s entirely optional. This pattern language will work without it.
To build a new application using A-Frame Architecture, Grow Evolutionary Seeds. To convert an existing codebase, Descend the Ladder.
Logic Sandwich
When using an A-Frame Architecture, the infrastructure and logic layers aren’t allowed to communicate with each other. But the logic layer needs to read and write data controlled by the infrastructure layer. Therefore:
Implement the Application layer code as a “logic sandwich,” which reads data using the Infrastructure layer, processes it using the Logic layer, then writes it using the Infrastructure layer. Repeat as needed. Each layer can then be tested independently.
// JavaScript
const input = infrastructure.readData();
const output = logic.processInput(input);
infrastructure.writeData(output);
This simple algorithm can handle sophisticated needs if put into a stateful loop. In some cases, your Application layer might need a bit of logic of its own, or you might need multiple sandwiches.
For applications that respond to events, use a Traffic Cop instead.
Traffic Cop
The Logic Sandwich boils infrastructure down into simple infrastructure.readData()
and infrastructure.writeData()
abstractions. But some applications need to respond to changes instigated by the infrastructure and logic layers. Therefore:
Program the application layer to use the Observer pattern to listen for events from the infrastructure and logic layers. For each event, implement a Logic Sandwich.
// Traffic Cop example (JavaScript)
server.onPost("/login", (formData) => { // event from infrastructure layer
const loginData = processLoginForm(formData); // application logic
const userData = userService.logInUser(loginData); // infrastructure layer
this._user = new User(userData); // logic layer
const userIsValid = this._user.isValid(); // logic layer
if (userIsValid) { // application logic
const sessionData = user.sessionData; // logic layer
sessionServer.createSession(sessionData); // infrastructure layer
return redirect(loginData.postLoginUrl); // application logic
}
else {
return redirect(LOGIN_FAILED_URL); // application logic
}
});
this._user.onChange((userData) => { // event from logic layer
userService.updateUser(userData); // infrastructure layer
});
Be careful not to let your Traffic Cop turn into a God Class. If it gets complicated, better infrastructure abstractions might help. Sometimes taking a less “pure” approach and moving some Logic code into the Infrastructure layer can simplify the overall design. In other cases, splitting the application layer into multiple classes or modules, each with its own Logic Sandwich or simple Traffic Cop, can help.
Grow Evolutionary Seeds
One popular design technique is outside-in design, in which an application is programmed by starting with the externally-visible behavior of the application, then working your way in to the details.
This is typically done by writing a broad integration test to describe the externally-visible behavior, then using interaction tests to build higher-level functions before lower-level functions. But we want to use Narrow Tests, not broad tests, and State-Based Tests, not interaction tests. Therefore:
Use evolutionary design to grow your application from a single file. Choose a simple end-to-end behavior as a starting point, then test-drive a single class to implement a trivial version of that behavior. Hardcode one value that would normally come from the Infrastructure layer, don’t implement any significant logic, and return the result to your tests rather than displaying it in a UI. This class forms the seed of your Application layer.
// Simplest possible Application seed (JavaScript)
// Test code
it("renders user name", () => {
const app = new MyApplication();
assert.equal("Hello, Sarah", app.render());
});
// Production code
class MyApplication {
render() {
return "Hello, Sarah";
}
}
Next, implement a barebones Infrastructure Wrapper for the one infrastructure value you hardcoded. Test-drive it with Narrow Integration Tests and code just enough to provide one real result that your Application class needs. Don’t worry about making it robust or reliable yet. This Infrastructure Wrapper class forms the seed of your Infrastructure layer.
Before integrating your new Infrastructure class into your Application class, use the Nullability Patterns to make the Infrastructure class testable from your application layer. Then modify your Application class to use the Infrastructure class, injecting the Nulled version in your tests.
// Application + read from infrastructure (JavaScript)
// Test code
it("renders user name", async () => {
const usernameService = UsernameService.createNull({ username: "my_username" }); // Nullable with Configurable Responses
const app = new MyApplication(usernameService);
assert.equal("Hello, my_username", await app.renderAsync());
});
// Production code
class MyApplication {
static create() { // Parameterless Instantiation
return new MyApplication(UsernameService.create());
}
constructor(usernameService) {
this._usernameService = usernameService;
}
async renderAsync() {
const username = await this._usernameService.getUsername();
return `Hello, ${username}`;
}
}
Next, do the same for your UI. Choose one simple output mechanism that your application will use (such as rendering to the console, the DOM, or responding to a network request) and implement a barebones Infrastructure Wrapper for it. Make it Nullable and modify your Application layer tests and code to use it.
// Application + read/write to Infrastructure (JavaScript)
// Test code
it("renders user name", () => {
const usernameService = UsernameService.createNull({ username: "my_username" }); // Nullable with Configurable Responses
const uiService = UiService.createNull(); // Nullable
const uiOutput = uiService.trackOutput(); // Output Tracking
const app = new MyApplication(usernameService, uiService);
app.render();
assert.deepEqual(uiOutput.data, [ "Hello, my_username "]);
});
// Production code
class MyApplication {
static create() { // Parameterless Instantiation
return new MyApplication(UsernameService.create(), UiService.create());
}
constructor(usernameService, uiService = UiService.create()) {
this._usernameService = usernameService;
this._uiService = uiService;
}
async render() {
const username = await this._usernameService.getUsername();
await uiService.render(`Hello, ${username}`);
}
}
Now your application tests serve the same purpose as broad end-to-end tests: they document and test the externally-visible behavior of the application. They’re Narrow Tests, because they’re focused on the behavior of the Application class, and because they use Nullable dependencies, they don’t communicate with external systems. That makes them fast and reliable. But because they’re also Overlapping Sociable Tests, they provide the same safety net that broad tests do.
At this point, you have the beginnings of a walking skeleton: an application that works end-to-end, but is far from complete. You can evolve that skeleton to support more features. Choose some aspect of your code that’s obviously incomplete and test-drive a slightly better solution. Repeat forever.
// Application + read/write to Infrastructure + respond to UI events (JavaScript)
// Test code
it("renders user name", async () => {
const usernameService = UsernameService.createNull({ username: "my_username" }); // Nullable with Configurable Responses
const uiService = UiService.createNull(); // Nullable
const uiOutput = uiService.trackOutput(); // Output Tracking
const app = new MyApplication(usernameService, uiService);
await app.startAsync();
uiService.simulateRequest("greeting"); // Behavior Simulation
assert.deepEqual(uiOutput.data, [ "Hello, my_username" ]);
});
// Production code
class MyApplication {
static create() { // Parameterless Instantiation
return new MyApplication(UsernameService.create(), UiService.create());
}
constructor(usernameService, uiService = UiService.create()) {
this._usernameService = usernameService;
this._uiService = uiService;
}
async startAsync() {
this._uiService.on("greeting", () => {
const username = await this._usernameService.getUsername();
await uiService.render(`Hello, ${username}`);
});
}
}
At some point, probably fairly early, your Application layer class will start feeling messy. When it does, look for a concept that can be factored into its own class. This forms the seed of your Logic layer. As your application continues to grow, continue refactoring so that class collaborations are easy to understand and responsibilities are clearly defined.
When working with existing code, use the Legacy Code Patterns instead.
Logic Patterns
Logic code is pure computation. To qualify, code can’t involve external systems or state. That means it can’t talk to a database, communicate across a network, touch the file system, read the date and time, look at environment variables, or use most random number generators. It can’t depend on any code that does these things, either.
Pure computation is easy to test. The following patterns make it even easier.
Easily-Visible Behavior
Logic computation can only be tested by State-Based Tests if the results of the computation are visible to tests. Therefore:
Prefer pure functions where possible. Pure functions’ return values are determined only by their input parameters.
// JavaScript
function add(a, b) {
return a + b;
}
When working with objects, prefer immutable objects, which are the object-oriented equivalent of pure functions. The state of immutable objects is determined when the object is constructed, and never changes afterwards.
// JavaScript
class Value {
constructor(initialValue) {
this._value = initialValue;
}
plus(addend) {
return new Value(this._value + addend);
}
}
For mutable objects, provide a way for changes in state to be observed, either with a getter method or an event.
// JavaScript
class RunningTotal {
constructor(initialValue) {
this._total = initialValue;
}
add(addend) {
this._total += addend;
}
getTotal() {
return this._total;
}
}
In all cases, avoid writing code that explicitly depends on (or changes) the state of dependencies more than one level deep. That makes test setup difficult, and it’s a sign of poor design anyway. Instead, design dependencies so they completely encapsulate their next-level-down dependencies.
Testable Libraries
Third-party code doesn’t always have Easily-Visible Behavior. It also tends to introduce breaking API changes with new releases, or simply stop being maintained. Therefore:
Wrap third-party code in code you control. Ensure your application’s use of the third-party code is mediated through your wrapper. Write your wrapper’s API to match the needs of your application, not the third-party code, and add methods as needed to provide Easily-Visible Behavior. (This will typically involve writing getter methods to expose deeply-buried state.) When the third-party code introduces a breaking change, or needs to be replaced, modify the wrapper so no other code is affected.
Frameworks and libraries with sprawling APIs are more difficult to wrap, so prefer libraries that have a narrowly-defined purpose and a simple API.
Some third-party code is pervasive and stable, such as core language frameworks. Other code, such as UI frameworks, can be very costly to wrap. You may be better off not creating a wrapper for these cases.
If the third-party code interfaces with an external system or state, use an Infrastructure Wrapper instead.
Collaborator-Based Isolation
Overlapping Sociable Tests ensure your tests will fail if your code’s behavior changes, no matter how far down the dependency chain those changes may be. On the one hand, this is nice, because you’ll learn when you accidentally break something. On the other hand, this could make feature changes terribly expensive. We don’t want a change in the formatting of addresses to break hundreds of unrelated reports’ tests. Therefore:
When a dependency’s behavior isn’t relevant to the code under test, use the dependency to help define test expectations. For example, if you’re testing a report that includes an address in its header, don’t hardcode “123 Main St.” as your expectation. Instead, ask the address how it would render itself, and use that as part of your test expectation.
Be careful not to write tests that are a copy of the code under test. Collaborator-Based Isolation is for writing Narrow Tests that ignore irrelevant details. For example, in the following code, the test is checking the special case of a report with a single address, not the behavior of address rendering. Address rendering is expected to have its own Narrow Tests.
// JavaScript
// Example test
it("includes the address in the header when reporting on one address", () => {
// Instantiate the unit under test and its dependency
const address = Address.createTestInstance(); // Parameterless Instantiation
const report = new InventoryReport(Inventory.create(), [ address ]);
// Define the expected result using the dependency
const expected = "Inventory Report for " + address.renderAsOneLine();
// Run the production code and make the assertion
assert.equal(report.renderHeader(), expected);
});
// Example production code
class InventoryReport {
constructor(inventory, addresses) {
this._inventory = inventory;
this._addresses = addresses;
}
renderHeader() {
let result = "Inventory Report";
if (this._addresses.length === 1) {
result += " for " + this._address[0].renderAsOneLine();
}
return result;
}
}
This provides the best of both worlds: Overlapping Sociable Tests ensure that your application is wired together correctly and Collaborator-Based Isolation allows you to change behavior without breaking a lot of tests. However, it also ties the tests more tightly to the production code’s implementation, so it should be used sparingly.
Infrastructure Patterns
Infrastructure code is for communicating with the outside world. Although it may contain some logic, that logic should be focused on making infrastructure easier to work with. Everything else belongs in Application or Logic code.
Infrastructure code is unreliable and difficult to test because of its dependencies on external systems and state. The following patterns work around those problems.
Infrastructure Wrappers
Infrastructure code is complicated to write, hard to test, and often difficult to understand. Therefore:
Isolate your Infrastructure code. For each external system—service, database, file system, or even environment variables—create one wrapper class that’s solely responsible for interfacing with that system. Design your wrappers to provide a crisp, clean view of the messy outside world, in whatever format is most useful to the rest of your code.
Avoid creating complex webs of dependencies. In some cases, high-level Infrastructure classes may depend on generic, low-level classes. For example, LoginClient
might depend HttpClient
. In other cases, high-level infrastructure classes might unify multiple low-level classes, such as a DataStore
class that depends on a RelationalDb
class and a NoSqlDb
class. Other than these sorts of simple one-way dependency chains, design your Infrastructure classes to stand alone.
Test your Infrastructure Wrappers with Narrow Integration Tests and Paranoic Telemetry. Make them testable with the Nullability Patterns.
Infrastructure Wrappers are also called “Gateways” or “Adapters,” although those terms are technically a superset of infrastructure wrappers.
Narrow Integration Tests
Ultimately, Infrastructure code talks over a network, interacts with a file system, or involves some other communication with external systems or state. It’s easy to make a mistake. Therefore:
Test your external communication for real. For file system code, read and write real files. For databases, access a real database. Make sure that your test systems use the same configuration as your production environment. Otherwise your code will fail in production when it encounters subtle incompatibilities.
Run your narrow integration tests against test systems that are reserved exclusively for one machine’s use. It’s best if they run locally on your development machine, and are started and stopped by your tests or build script. Otherwise, you could experience unpredictable test failures when multiple people run the tests at the same time.
If you have multiple external systems that use the same technology, such as multiple web services, create a generic, low-level infrastructure wrapper for the underlying technology. Then create higher-level infrastructure wrappers for each system. The high-level wrappers don’t need Narrow Integration Tests. Instead, you can Fake It Once You Make It by delegating to the low-level wrapper.
For example, you could create a high-level LoginClient
that depended on a low-level HttpClient
. The LoginClient
would Fake It Once You Make It and the HttpClient
would be tested with Narrow Integration Tests.
// Example of narrow integration tests for HttpClient (JavaScript + Node.js)
import * as http from "node:http";
import HttpClient from "./http_client";
const HOST = "localhost";
const PORT = 5001;
// Tests
describe("HTTP Client", () => {
let server;
before(async () => {
server = new TestServer();
await server.startAsync();
});
after(async () => {
await server.stopAsync();
});
beforeEach(function() {
server.reset();
});
it("performs request", async () => {
await requestAsync({
host: HOST,
port: PORT,
method: "POST",
path: "/my/path",
headers: { myRequestHeader: "myRequestValue" },
body: "my request body"
});
assert.deepEqual(server.lastRequest, {
method: "POST",
path: "/my/path",
headers: { myrequestheader: "myRequestValue" },
body: "my request body"
});
});
it("returns response", async () => {
server.setResponse({
status: 999,
headers: { myResponseHeader: "myResponseValue" },
body: "my response",
});
const response = await requestAsync();
assert.deepEqual(response, {
status: 999,
headers: { myresponseheader: "myResponseValue" },
body: "my response",
});
});
async function requestAsync(options = {
host: HOST,
port: PORT,
method: "GET",
path: "/irrelevant/path",
}) {
const client = HttpClient.create();
return client.requestAsync(options);
}
});
// Localhost HTTP server
class TestServer {
constructor() {
this.reset();
}
reset() {
this._lastRequest = null;
this._nextResponse = {
status: 500,
headers: {},
body: "response not specified",
};
}
startAsync() {
return new Promise((resolve, reject) => {
this._server = http.createServer();
this._server.once("listening", resolve);
this._server.once("error", reject);
this._server.on("request", this.#handleRequest.bind(this));
this._server.listen(PORT);
});
}
stopAsync() {
return new Promise((resolve, reject) => {
this._server.once("close", resolve);
this._server.close();
});
}
setResponse(response) {
this._nextResponse = response;
}
get lastRequest() {
return this._lastRequest;
}
// In JavaScript, methods that start with "#" are private.
#handleRequest(request, response) {
let body = "";
request.on("data", (chunk) => {
body += chunk;
});
request.on("end", () => {
this.#storeRequest(request, body);
this.#sendResponse(response);
});
}
#storeRequest(request, body) {
const headers = { ...request.headers };
delete headers.connection;
delete headers["content-length"];
delete headers.host;
this._lastRequest = {
method: request.method,
path: request.url,
headers,
body,
};
}
#sendResponse(response) {
response.statusCode = this._nextResponse.status;
Object.entries(this._nextResponse.headers).forEach(([key, value]) => {
response.setHeader(key, value);
});
response.end(this._nextResponse.body);
}
}
Ensure your code works in production with Paranoic Telemetry.
Paranoic Telemetry
External systems are unreliable. The only thing that’s certain is their eventual failure. File systems lose data and become unwritable. Services return error codes, suddenly change their specifications, and refuse to terminate connections. Therefore:
Assume they really are out to get you, and instrument your code accordingly. Expect that everything will break eventually. Test that every failure case either logs an error and sends an alert, or throws an exception that ultimately logs an error and sends an alert. Remember to test your code’s ability to handle requests that hang, too.
All these failure cases are expensive to support and maintain. Whenever possible, use Testable Libraries rather than external services.
Paranoic Telemetry may be supplemented with Contract Tests. Contract Tests are most effective when run by the supplier (but provided by you), because they can’t catch changes that happen between test runs.
Nullability Patterns
Sociable Tests run real code. That’s good for catching errors, but if the dependency chain includes infrastructure—external systems or state—they become hard to manage. The following patterns allow you to “turn off” external dependencies while retaining the benefits of sociable and state-based testing.
- Nullables
- Embedded Stub
- Thin Wrapper
- Configurable Responses
- Output Tracking
- Behavior Simulation
- Fake It Once You Make It
Nullables
Narrow Integration Tests are slow and difficult to set up. Although they’re useful for ensuring that low-level Infrastructure Wrappers work in practice, they’re overkill for code that depends on those wrappers. Therefore:
Program code that includes infrastructure in its dependency chain to have a createNull()
factory method. The factory should create a “Nulled” instance that disables all external communication, but behaves normally in every other respect.2 Make sure it supports Parameterless Instantiation.
For example, calling LoginClient.createNull().getUserInfo(...)
should return a default response without actually talking to the third-party login service.
Nullables are production code and should be tested accordingly. Although Nulled instances are often used by tests, they’re also useful whenever you want the ability to “turn off” behavior in your application. For example, you could use Nullables implement a “dry run” option in a command-line application.
// Example of using Nullables to implement "dry run" option (JavaScript + Node.js)
async initializeGitWriter(config) {
if (config.dryRun) {
return GitWriter.createNull();
}
else {
return GitWriter.create();
}
}
As another example, you can use Nullables in a web server to cache popular URLs when the server starts up:
// Example of using Nullables to implement cache warming (JavaScript + Node.js)
async warmCacheAsync(popularUrls, log) {
for await (const url of popularUrls) {
await this.routeAsync(HttpRequest.createNull(url, log);
}
}
Make low-level infrastructure wrappers Nullable with Embedded Stubs. For all other code, Fake It Once You Make It. To make existing code Nullable, see the Legacy Code Patterns.
If your Nullable reads data from external systems or state, or any of its dependencies do, implement Configurable Responses. If it or its dependencies write data, implement Output Tracking. If they respond to events, implement Behavior Simulation.
Embedded Stub
Nullables need to disable access to external systems and state while running everything else normally. The obvious approach is to surround any code that accesses the external system with an “if” statement, but that’s a recipe for spaghetti. Therefore:
When making code Nullable, don’t change your code. Instead, stub out the third-party code that accesses external systems.
In your stub, implement the bare minimum needed to make your code run. Ensure you don’t overbuild the stub by test-driving it through your code’s public interface. Put the stub in the same file as the rest of your code so it’s easy to remember and update when your code changes.
Write a stub of the third-party code, not your code, so your Sociable Tests test how your code will really work in production. Be careful to have your stub mimic the behavior of the third-party code exactly. To help you do so, write Narrow Integration Tests that document the behavior of the real code, paying particular attention to edge cases such as error handling and asynchronous code. Then write additional tests of the Nulled instance that will fail if your stub doesn’t have the same behavior.
Here’s a simple example of stubbing out JavaScript’s Math
library:
// An Infrastructure Wrapper for a random die roller. (JavaScript)
// Infrastructure Wrapper
export default class DieRoller {
// Normal factory
static create() {
return new DieRoller(Math); // "Math" is a built-in JavaScript global
}
// Null factory
static createNull() {
return new DieRoller(new StubbedMath());
}
// Shared initialization code
constructor(math) {
this._math = math;
}
// Infrastructure wrapper implementation.
// This is the same code you would write without a stub.
roll(amount) {
const randomNumber = this._math.random();
return Math.trunc((randomNumber * 6) + 1); // There's no need to stub Math.trunc, so we use the real Math library here
}
};
// Embedded Stub. Note that we only stub the function we use.
class StubbedMath {
random() {
return 0;
}
}
Here’s a more complicated example. It stubs out Node.js’s http
library:
// An infrastructure wrapper for a generic HTTP client. (JavaScript + Node.js)
import * as http from "node:http";
import { EventEmitter } from "node:events";
export default class HttpClient {
// Normal factory
static create() {
return new HttpClient(http);
}
// Null factory
static createNull() {
return new HttpClient(new StubbedHttp());
}
// Shared initialization code
constructor(http) {
this._http = http;
}
// Infrastructure wrapper implementation.
// This is the same code you would write without a stub.
async requestAsync({ host, port, method, path, headers = {}, body = "" }) {
if (method === "GET" && body !== "") throw new Error("Don't include body with GET requests; Node won't send it");
const httpOptions = { host, port, method, path, headers };
const request = this.#sendRequest(httpOptions, body);
await new Promise((resolve, reject) => {
this.#handleResponse(request, resolve);
this.#handleError(request, reject);
});
}
// In JavaScript, methods that start with "#" are private.
#sendRequest(httpOptions, body) {
const request = this._http.request(httpOptions);
request.end(body);
return request;
}
#handleResponse(request, resolve) {
request.once("response", (response) => {
let body = "";
response.on("data", (chunk) => {
body += chunk;
});
response.on("end", () => {
resolve({
status: response.statusCode,
headers,
body,
});
});
});
}
#handleError(request, reject) {
request.once("error", reject);
}
};
// Embedded Stub. Note that it’s built exactly for the needs of the infrastructure code, nothing more.
class StubbedHttp {
request() {
return new StubbedRequest();
}
}
class StubbedRequest extends EventEmitter {
end() {
// setImmediate() is used to make the emit() call asynchronous, duplicating the behavior of real code
setImmediate(() => this.emit("response", new StubbedResponse()));
}
}
class StubbedResponse extends EventEmitter {
constructor() {
super();
setImmediate(() => {
this.emit("data", "Nulled HttpClient response");
this.emit("end");
});
}
get statusCode() {
return 200;
}
get headers() {
return {};
}
}
Configure the embedded stub’s return values with Configurable Responses. If your language requires it, as with Java or C#, create a Thin Wrapper.
Nullables are production code, and despite appearances, the Embedded Stub is too. It must be tested accordingly. If you don’t like the idea of having stubs in your production code, you can put the Embedded Stub in a separate test-only file instead. However, this will make dependency management more complicated, and it will prevent you from using Nulled instances in production, which can be useful.
Thin Wrapper
Languages such as Java and C# will require your Embedded Stub to share an interface with the real dependency. Often, there won’t be an interface you can use, or it will be bigger than you need. Therefore:
Create a custom interface for your third-party dependency. Match the signature of the dependency exactly, but only include the methods your production code actually uses. Provide two implementations of the interface: a real version that only forwards calls to the third-party dependency, and an Embedded Stub.
// A simple Infrastructure Wrapper for a random die roller. (Java)
// It has an embedded stub for Java's standard “Random” library.
// Based on an example created with Ted M. Young in his Yacht codebase.
public class DieRoller {
private final RandomWrapper random;
// Normal factory
public static DieRoller create() {
return new DieRoller(new RealRandom());
}
// Null factory
public static DieRoller createNull() {
return new DieRoller(new StubbedRandom());
}
// Private constructor with shared initialization code
private DieRoller(RandomWrapper random) {
this.random = random;
}
// Infrastructure wrapper implementation.
// This is the same code you would write without a stub.
public int roll() {
return random.nextInt(6) + 1;
}
// Interface for Thin Wrapper. Note that we match the real code's interface exactly,
// and we only include the function we use.
private interface RandomWrapper {
int nextInt(int bound);
}
// Real implementation of Thin Wrapper
private static class RealRandom implements RandomWrapper {
private final Random random = new Random();
@Override
public int nextInt(int bound) {
return this.random.nextInt(bound);
}
}
// Embedded Stub implementation of Thin Wrapper
private static class StubbedRandom implements RandomWrapper {
@Override
public int nextInt(int bound) {
return 0;
}
}
}
If the third-party code returns custom types, you’ll need to wrap those return types as well. Remember to match the third-party code’s signatures exactly.
// Infrastructure Wrapper for an HTTP request. (Java + Spring Boot's RestTemplate)
// Based on an example created with Ted M. Young in his Yacht codebase.
public class AverageScoreFetcher {
private static final String YACHT_AVERAGE_API_URI = "http://localhost:8080/api/averages?scoreCategory={scoreCategory}";
private final RestTemplateWrapper restTemplate;
// Normal factory
public static AverageScoreFetcher create() {
return new AverageScoreFetcher(new RealRestTemplate());
}
// Null factory
public static AverageScoreFetcher createNull() {
return new AverageScoreFetcher(new StubbedRestTemplate());
}
// Private constructor with shared initialization code
private AverageScoreFetcher(RestTemplateWrapper restTemplate) {
this.restTemplate = restTemplate;
}
// Infrastructure wrapper implementation
public double averageFor(ScoreCategory scoreCategory) {
ResponseEntityWrapper entity = restTemplate.getForEntity(
YACHT_AVERAGE_API_URI,
CategoryAverage.class,
scoreCategory.toString()
);
return entity.getBody().getAverage();
}
// Interfaces for Thin Wrapper. Note that we only include the functions we use.
interface RestTemplateWrapper {
ResponseEntityWrapper getForEntity(String url, Class responseType, Object... uriVariables);
}
interface ResponseEntityWrapper {
T getBody();
}
// Real implementations of Thin Wrapper
private static class RealRestTemplate implements RestTemplateWrapper {
private final RestTemplate restTemplate = new RestTemplate();
public ResponseEntityWrapper getForEntity(String url, Class responseType, Object... uriVariables) {
return new RealResponseEntity(restTemplate.getForEntity(url, responseType, uriVariables));
}
}
private static class RealResponseEntity implements ResponseEntityWrapper {
private ResponseEntity entity;
RealResponseEntity(ResponseEntity entity) {
this.entity = entity;
}
public T getBody() {
return this.entity.getBody();
}
}
// Stubbed implementations of Thin Wrapper
private static class StubbedRestTemplate implements RestTemplateWrapper {
@Override
public ResponseEntityWrapper getForEntity(String url, Class responseType, Object... uriVariables) {
return new StubbedResponseEntity<>();
}
}
private static class StubbedResponseEntity implements ResponseEntityWrapper {
@Override
public T getBody() {
return (T) new CategoryAverage("Nulled AverageScoreFetcher category", 42.0);
}
}
}
Configurable Responses
State-based tests of code with infrastructure dependencies needs to set up the infrastructure’s state, but setting up external systems is complicated and slow. Therefore:
Make the infrastructure dependencies Nullable and program the createNull()
factory to take your desired response as an optional parameter. Define the responses from the perspective of the dependency’s externally-visible behavior, not its implementation.
If the Nullable dependency has multiple types of responses that can be configured, give each one its own configuration parameter. Use named and optional parameters so tests only need to configure the data they care about. If your language doesn’t support optional parameters, use an Options object, as shown in the Signature Shielding pattern.
For example, the following test is for a LoginController
that depends on a Nullable LoginClient
. Although LoginClient
is used to make HTTP requests, its Configurable Responses aren’t about HTTP. Instead, they’re about the logged-in user’s email address and verification status, which is the behavior LoginController
and its tests care about.
// Example of configuring multiple types of responses. (JavaScript)
it("logs successful login", async () => {
// Configure login client dependency
const loginClient = LoginClient.createNull(
email: "my_authenticated_email", // configure email address
emailVerified: true, // configure whether email is verified
);
// Run production code
const { logOutput } = await performLogin({ loginClient })); // Signature Shielding
// Check results
assert.deepEqual(logOutput.data, [ "Login: my_authenticated_email (verified)" ]); // Output Tracking
});
If it makes sense for your class respond differently each time it’s called, configure the responses with an array or list. It’s often helpful to support two data types: a list of values, that results in a different response each time, and causes an exception when it runs out; and a single value, that returns the same response every time, and never runs out.
For example, the following test configures a Nullable DieRoller
with a set of expected die rolls:
// Example of a single type of response with multiple return values. (JavaScript)
// Inspired by an example created with Ted M. Young in his Yacht codebase.
it("rolls a hand of dice", async () => {
// Configure die rolls
const dieRoller = DieRoller.createNull([ 1, 2, 3, 4, 5 ]);
// Run production code
const game = new Game(dieRoller);
const hand = game.roll();
// Check results
assert.deepEqual(hand, HandOfDice.create(1, 2, 3, 4, 5));
});
If your Nullable uses an Embedded Stub, implement the responses in the stub. Otherwise, Fake It Once You Make It. Either way, decompose the responses down to the next level.
The following example uses an Embedded Stub to make a random die roller. It’s configured at the level its callers care about: dice roll results. In the Embedded Stub, those configured values are decomposed to the level DieRoller operates at: random floating point numbers from between zero and one. For example, a configured roll of 6
is turned into the floating point number 0.83333
.
// Example of implementing Configurable Responses in an Embedded Stub. (JavaScript)
// Infrastructure Wrapper
export default class DieRoller {
static create() {
return new DieRoller(Math); // "Math" is a built-in JavaScript global
}
// Null factory with Configurable Responses
// If a number is provided, it always returns that number.
// If an array is provided, it returns exactly the numbers provided, then throws an error when it runs out.
// If nothing is provided, it defaults to returning ones.
static createNull(rolls = 1) { // set default to 1
return new DieRoller(new StubbedMath(rolls)); // pass configuration to Embedded Stub
}
constructor(math) {
this._math = math;
}
roll(amount) {
const randomNumber = this._math.random();
return Math.trunc((randomNumber * 6) + 1); // There's no need to stub Math.trunc, so we use the real Math global
}
};
// Embedded Stub with Configurable Responses
class StubbedMath {
constructor(rolls) {
// Store configured responses
this._rolls = rolls;
}
random() {
// Use configured responses
const roll = this.#nextRoll(); // Get configuration to use
return (roll - 1) / 6; // Convert to float to match behavior of real Math.random()
}
// Retrieve configured response
#nextRoll() {
if (Array.isArray(this._rolls)) {
// Configuration is an array, so return the next roll in the array
const roll = this._rolls.shift();
if (roll === undefined) throw new Error("No more rolls configured in nulled DieRoller");
return roll;
}
else {
// Configuration is a number, so always return that number
return this._rolls;
}
}
}
The above code can be simplified by factoring #nextRoll()
into a generic helper class. The result looks like this:
// Example of implementing an embedded stub with a ConfigurableResponses helper class (JavaScript)
class StubbedMath {
constructor(rolls) {
this._rolls = ConfigurableResponses.create(rolls);
}
random() {
return (this._rolls.next() - 1) / 6;
}
}
This is a JavaScript implementation of ConfigurableResponses
you can use in your own code:
// Copyright 2023 Titanium I.T. LLC. MIT License.
export default class ConfigurableResponses {
// Create a list of responses (by providing an array),
// or a single repeating response (by providing any other type).
// 'Name' is optional and used in error messages.
static create(responses, name) {
return new ConfigurableResponses(responses, name);
}
// Convert all properties in an object into ConfigurableResponse instances.
// For example, { a: 1 } becomes { a: ConfigurableResponses.create(1) }.
// 'Name' is optional and used in error messages.
static mapObject(responseObject, name) {
const entries = Object.entries(responseObject);
const translatedEntries = entries.map(([ key, value ]) => {
const translatedName = name === undefined ? undefined : `${name}: ${key}`;
return [ key, ConfigurableResponses.create(value, translatedName )];
});
return Object.fromEntries(translatedEntries);
}
constructor(responses, name) {
this._description = name === undefined ? "" : ` in ${name}` ;
this._responses = Array.isArray(responses)
? [ ...responses ]
: responses;
}
// Get next configured response. Throws an error when configured with a list
// of responses and no more responses remain.
next() {
const response = Array.isArray(this._responses)
? this._responses.shift()
: this._responses;
if (response === undefined) throw new Error(`No more responses configured${this._description}`);
return response;
}
};
To test code with dependencies that write to infrastructure, use Output Tracking. To test code with dependencies that respond to events, use Behavior Simulation.
Output Tracking
State-based tests of code with dependencies that write to external systems need to check whether the writes were performed, but setting up external systems is complicated and slow. Therefore:
Program each dependency with a tested, production-grade trackXxx()
method that tracks the otherwise-invisible writes. Have it do so regardless of whether the object is Nulled or not.
Track the writes in terms of the behavior your callers care about, not the underlying implementation of your code. For example, a structured logger might write strings to stdout
, but its callers care about the structured data that’s being written. Its Output Tracking would track the data, not the string.
One way to implement Output Tracking is to have trackXxx()
return an OutputTracker
that listens for events emitted by your production code. The following example shows how this works, including implementations of OutputTracker
in JavaScript and Java that you can use in your own code. It starts with a test of LoginPage
, which writes to a structured Log
when the user logs in.
// Example of using Output Tracking (JavaScript)
// Application layer test
it("writes to log when user logs in", async () => {
// Set up a log and track its output
const log = Log.createNull();
const logOutput = log.trackOutput();
// Instantiate the code under test
const loginPage = new LoginPage(log);
// Run the code
const formData = // code to set up "my_email" login here
await loginPage.postAsync(formData);
// Check the log output
assert.deepEqual(logOutput.data, [{
alert: "info",
message: "User login",
email: "my_email",
}]);
});
// Application layer code
class LoginPage {
constructor(log) {
this._log = log;
}
async postAsync(formData) {
const email = // code to parse formData and verify login goes here
// Code under test
this.log.info({
message: "User login",
email,
});
}
}
// High-level "Log" infrastructure wrapper used by the code under test
import Clock from "clock"; // Low-level infrastructure wrapper
import Stdout from "stdout"; // Low-level infrastructure wrapper
import { EventEmitter } from "node:events"; // Standard Node.js event library
import OutputTracker from "output_tracker"; // Output tracking library
const OUTPUT_EVENT = "output"; // Event to emit when output occurs
class Log {
static create() {
return new Log(Clock.create(), Stdout.create());
}
static createNull({
clock = Clock.createNull(), // Fake It Once You Make It
stdout = Stdout.createNull(),
} = {}) {
return new Log(clock, stdout);
}
constructor(clock, stdout) {
this._clock = clock;
this._stdout = stdout;
this._emitter = new EventEmitter(); // Event emitter used when output occurs
}
// Output tracker
trackOutput() {
return OutputTracker.create(this._emitter, OUTPUT_EVENT);
}
// The method called by the code under test
info(data) {
data.alert = "info";
// Write the log
const now = this._clock.formattedTimestamp();
const dataJson = JSON.stringify(data);
this._stdout.write(`${now} ${dataJson}`);
// Emit the event. This is received by the OutputTracker.
this._emitter.emit(OUTPUT_EVENT, data);
}
}
At first glance, Output Tracking might look like the same thing as a spy (a type of test double), but there’s an important difference. Output Tracking records behavior, and spies record function calls. Output Trackers should write objects that represent the action that was performed, not just the function that was called to perform it. That way, when you refactor, you can change your functions without changing your Output Trackers or the tests that depend on them.
This is a JavaScript version of the OutputTracker
class you can use in your own projects:
// Copyright 2020-2022 Titanium I.T. LLC. MIT License.
export default class OutputTracker {
static create(emitter, event) {
return new OutputTracker(emitter, event);
}
constructor(emitter, event) {
this._emitter = emitter;
this._event = event;
this._data = [];
this._trackerFn = (text) => this._data.push(text);
this._emitter.on(this._event, this._trackerFn);
}
get data() {
return this._data;
}
clear() {
const result = [ ...this._data ];
this._data.length = 0;
return result;
}
stop() {
this._emitter.off(this._event, this._trackerFn);
}
}
Below, I’ve also included a Java version of the OutputTracker
library that I created with Ted M. Young. Because Java doesn’t have a built-in event emitter, it’s used slightly differently. Here’s an example of using it in the Log
infrastructure wrapper from the earlier example:
// Example of Output Tracking in Java
public class Log {
// Instantiate the event emitter
private final OutputListener outputListener = new OutputListener<>();
public static Log create...
public static Log createNull...
private Log...
// Create the output tracker
public OutputTracker trackOutput() {
return outputListener.createTracker();
}
public void info(Map data) {
// ...
// Emit the event
outputListener.track(data);
}
}
This is the Java version of OutputTracker
. Split it into two files.
---- OutputListener.java ----
// Copyright 2022 Titanium I.T. LLC and Ted M. Young. MIT License.
package com.jamesshore.output_tracker;
import java.util.ArrayList;
import java.util.List;
public class OutputListener {
private final List> listeners = new ArrayList<>();
public void track(T data) {
listeners.forEach(tracker -> tracker.add(data));
}
public OutputTracker createTracker() {
OutputTracker tracker = new OutputTracker<>(this);
listeners.add(tracker);
return tracker;
}
void remove(OutputTracker outputTracker) {
listeners.remove(outputTracker);
}
}
---- OutputTracker.java ----
// Copyright 2022 Titanium I.T. LLC and Ted M. Young. MIT License.
package com.jamesshore.output_tracker;
import java.util.ArrayList;
import java.util.List;
public class OutputTracker {
private final List output = new ArrayList<>();
private final OutputListener outputListener;
public OutputTracker(OutputListener outputListener) {
this.outputListener = outputListener;
}
void add(T data) {
output.add(data);
}
public List data() {
return List.copyOf(output);
}
public List<T> clear() {
List data = this.data();
output.clear();
return data;
}
public void stop() {
outputListener.remove(this);
}
}
To test code with dependencies that read from infrastructure, use Configurable Responses. To test code with dependencies that emit events, use Behavior Simulation.
Behavior Simulation
Some external systems will push data to you rather than waiting for you to ask for it. Code that depends on those systems need a way to test what happens when their infrastructure dependencies generate those events, but setting up infrastructure to send events is complicated and slow. Therefore:
Add methods to your dependencies that simulate receiving an event from an external system. Share as much code as possible with the code that handles real external events. Write it as tested, production-grade code.
The following example consists of an Application-layer MessageServer
that performs real-time networking. MessageServer
runs on a server and connects to web browsers using WebSocketServer
, a low-level infrastructure wrapper for Socket.IO. When a connected browser sends a message to MessageServer
, it relays the messages to all the other connected browsers.
The test uses Behavior Simulation to simulate web browsers connecting and sending messages, then validates MessageServer
’s behavior by using Output Tracking to confirm that the correct messages were sent.
// Example of using Behavior Simulation (JavaScript)
// Application layer test
it("broadcasts messages from one client to all others", async () => {
// Set up test data
const clientId = "my_client_id";
const message = new TestClientMessage("my_message");
// Set up the infrastructure wrapper and the code under test
const network = WebSocketServer.createNull(); // Create the infrastructure wrapper
const sentMessages = network.trackMessages(); // Track messages sent by infrastructure wrapper (Output Tracking)
const server = new MessageServer(network); // Instantiate the application code under test
await server.startAsync(); // Start listening for messages (Zero-Impact Instantiation)
// Simulate a client connecting
network.simulateConnection(clientId);
// Simulate the client sending a message
network.simulateMessage(clientId, message);
// Check that the message was broadcast (Output Tracking)
assert.deepEqual(sentMessages.data(), [{
type: "broadcast",
excludedClient: clientId,
message
}]);
});
// Application layer code
class MessageServer {
constructor(webSocketServer) {
this._webSocketServer = webSocketServer;
}
async startAsync() {
// Code under test
this._webSocketServer.onMessage((clientId, message) => {
this._webSocketServer.broadcastToAllClientsExcept(clientId, message);
});
await this._webSocketServer.startAsync();
}
//...
}
The Behavior Simulation logic is implemented as production code in WebSocketServer
, the low-level Socket.IO wrapper. Note how the real Socket.IO logic and the simulation methods share as much implementation as possible by delegating to #handleXxx()
methods.
// Example of implementing Behavior Simulation (JavaScript)
import { Server } from "socket.io"; // Socket.IO
import { EventEmitter } from "node:events"; // Standard Node.js event library
import OutputTracker from "output_tracker"; // Output tracking library
const CLIENT_MESSAGE_EVENT = "client_message"; // Event constants
const SERVER_MESSAGE_EVENT = "server_message";
class WebSocketServer {
static create(port) {
return new WebSocketServer(io, port);
}
static createNull() {
return new WebSocketServer(StubbedServer, 42);
}
constructor(server, port) {
this._server = server;
this._port = port;
this._emitter = new EventEmitter();
this._connectedSockets = {};
}
// Real Socket.IO event handler
async startAsync() {
this._io.on("connection", (socket) => {
this.#handleConnection(socket);
socket.onAny((event, ...args) => {
const message = this.#deserializeMessage(event, args);
this.#handleMessage(socket.id, message);
});
socket.on("disconnect", () => {
this.#handleDisconnection(socket.id));
});
});
}
// Behavior Simulation
simulateConnection(clientId) {
this.#handleConnection(new StubbedSocket(clientId));
}
simulateMessage(clientId, message) {
this.#handleMessage(clientId, message);
}
simulateDisconnection(clientId) {
this.#handleDisconnection(clientId);
}
// Shared by event handler and behavior simulation
#handleConnection(socket) {
this._connectedSockets[socket.id] = socket;
)
#handleMessage(clientId, message) {
this._emitter.emit(CLIENT_MESSAGE_EVENT, { clientId, message });
}
#handleDisconnection(clientId) {
delete this._connectedSockets(clientId);
}
// Methods called by the code under test
onMessage(fn) {
this._emitter.on(CLIENT_MESSAGE_EVENT, ({ clientId, message }) => {
fn(clientId, message));
});
}
broadcastToAllClientsExcept(clientId, message) {
const socket = this._connectedSockets[clientId];
socket.broadcast.emit(message.name, message.payload);
this._emitter.emit(SERVER_MESSAGE_EVENT, { // Output Tracking
type: "broadcast",
excludedClient: clientId,
message,
});
});
// Output Tracking
trackMessages() {
return OutputTracker.create(this._emitter, SERVER_MESSAGE_EVENT);
}
//...
}
To test code with dependencies that read from infrastructure, use Configurable Responses. To test code with dependencies that write to infrastructure, use Output Tracking.
Fake It Once You Make It
Narrow Integration Tests are slow and difficult to set up. Similarly, Embedded Stubs can be difficult to create. Although they’re needed for low-level infrastructure wrappers, they’re overkill for code that doesn’t have direct dependencies on third-party infrastructure code. Therefore:
In application-layer code and high-level infrastructure wrappers, delegate to Nullable dependencies rather than using Narrow Integration Tests and Embedded Stubs. In your tests, inject Nulled instances of the code under test’s dependencies. If your production code has a createNull()
factory, implement it by creating Nulled dependencies and decomposing your Configurable Responses into the format your dependencies expect.
For example, the following code tests LoginClient
, which depends on a low-level HttpClient
. The LoginClient
tests Fake It Once You Make It by injecting a Nulled version of HttpClient
into LoginClient
.
// Example of a using Fake It Once You Make It in a test (JavaScript)
it("performs network request", async () => {
// Set up the low-level HTTP client (Configurable Responses)
const httpClient = HttpClient.createNull({
"/oauth/token": [{ // The Auth0 endpoint our code will call.
status: VALID_STATUS, // Status, headers, and body Auth0 could really return.
headers: VALID_HEADERS,
body: VALID_BODY,
}],
});
// Track requests made with the HTTP client (Output Tracking)
const httpRequests = httpClient.trackRequests();
// Instantiate the code under test, injecting the Nulled httpClient
const client = new LoginClient(httpClient, "my_client_id", "my_client_secret", "my_management_api_token");
// Run the code under test
await client.validateLoginAsync("my_login_code", "my_callback_url");
// Assert that the correct HTTP request was made (Output Tracking)
assert.deepEqual(httpRequests.data, [{
host: HOST,
port: PORT,
method: "post",
path: "/oauth/token",
headers: {
authorization: "Bearer my_management_api_token",
"content-type": "application/json; charset=utf-8",
},
body: JSON.stringify({
client_id: "my_client_id",
client_secret: "my_client_secret",
code: "my_login_code",
redirect_uri: "my_callback_url",
grant_type: "authorization_code"
}),
}]);
});
The production LoginClient
code is a wrapper for Auth0, an authentication service. LoginClient.createNull()
has Configurable Responses related to authentication, such as configuring the login email address. They’re implemented by creating a Nulled HttpClient
and decomposing LoginClient
’s Configurable Responses into the actual HTTP responses Auth0 would return.
// Example of a using Fake It Once You Make It to make a class Nullable (JavaScript)
class LoginClient {
// Normal factory
static create(host, clientId, clientSecret, managementApiToken) {
const httpClient = HttpClient.create();
return new LoginClient(httpClient, host, clientId, clientSecret, managementApiToken);
}
// Null factory with Configurable Responses
static createNull({
// Configurable Responses for user’s login
email = "null_login_email", // The email address associated with the login
emailVerified = true, // True if the email address has been verified
forbidden = undefined, // Set to a string to simulate an Auth0 "forbidden" response
// Other parameters unrelated to Configurable Responses
host = "null.host",
clientId = "null_client_id",
clientSecret = "null_client_secret",
managementApiToken = "null_management_api_token",
} = {}) {
// Convert LoginClient's Configurable Response into the response Auth0 would actually return
const auth0Response = nullValidateLoginResponse({ email, emailVerified, forbidden });
// Create a Nulled HttpClient that's configured to return the Auth0 response
const httpClient = HttpClient.createNull({
[VALIDATE_LOGIN_ENDPOINT]: auth0Response;
});
// Instantiate the LoginClient using the Nulled HttpClient
return new LoginClient(httpClient, clientId, clientSecret, managementApiToken);
}
// Shared initialization
constructor(httpClient, host, clientId, clientSecret, managementApiToken) {
this._httpClient = httpClient;
this._host = host;
this._clientId = clientId;
this._clientSecret = clientSecret;
this._authHeaders = {
authorization: `Bearer ${managementApiToken}`,
};
}
// Shared production code
async validateLoginAsync(code, callbackUrl) {
const response = await this._httpClient.requestAsync(
host: this._host,
method: "POST",
path: VALIDATE_LOGIN_ENDPOINT,
headers: this._authHeaders,
body: {
client_id: this._clientId,
client_secret: this._clientSecret,
code,
redirect_uri: callbackUrl,
grant_type: "authorization_code"
},
);
const decodedToken = /* code to validate and decode response here */
return {
email: decodedToken.email,
emailVerified: decodedToken.email_verified
};
}
// Configurable Responses translation code
// This function decomposes the responses passed to LoginClient.createNull() down
// into responses for HttpClient.createNull(). HttpClient.createNull() is configured
// with a status, optional headers, and a body.
function nullValidateLoginResponse({ email, emailVerified, forbidden }) {
// If the "forbidden" response is set, return a 403 (Forbidden) response.
if (forbidden) return { status: STATUS.FORBIDDEN_403, body: forbidden };
// Otherwise, create a JSON Web Token, because that's what Auth0 returns
const response = { email, email_verified: emailVerified };
const id_token = jwt.sign(response, "irrelevant_secret", { noTimestamp: true });
// Return the JWT in a 200 (OK) response
return {
status: STATUS.OK_200,
body: JSON.stringify({ id_token }),
};
}
}
Implement Output Tracking and Behavior Simulation normally, without regard to whether the dependencies are Nulled or not.
To make your dependencies Nullable, either Descend the Ladder or Climb the Ladder.
Legacy Code Patterns
If you’d like to convert your existing code and tests to use Nullables, the patterns in this section will help you do so.
Work incrementally. You can mix Nullables with your current approach in the same codebase, and even in the same test, so there’s no need to convert everything at once. Similarly, focus your efforts on code where testing with Nullables will have noticeable benefit. Don’t waste time converting code that’s already easy to maintain, regardless of how it’s tested.
Descend the Ladder
Complex codebases have a lot of dependencies, and it isn’t feasible to improve all the tests at once. Instead, you’ll need to make progress incrementally. Therefore:
When converting a module or class to use Nullables, convert the code and its direct dependencies, but nothing more. Work your way down through the rest of the dependency tree gradually, when time allows.
Each module or class you convert will fall into one of three categories:
A. No Infrastructure Dependencies
If the code doesn’t have infrastructure anywhere in its dependency tree, it doesn’t need to use Nullables. It can be tested with the Logic Patterns instead.
B. Infrastructure Wrapper with Third-Party Dependencies
If the code is an Infrastructure Wrapper with direct third-party infrastructure dependencies, test it with Narrow Integration Tests, then make it Nullable with an Embedded Stub.
C. Everything Else
For everything else, you’ll make your code’s direct dependencies Nullable, then Fake It Once You Make It. To make the dependencies Nullable, apply one of the following options to each one:
-
In most cases, the dependency will have a combination of logic and infrastructure in its dependency chain. Check the following bullet points. If none of them apply, make the dependency Nullable by introducing a Throwaway Stub.
-
If the dependency is already Nullable, or if it doesn’t have any infrastructure dependencies, no changes are needed.
-
If the dependency isn’t Nullable, but all of its dependencies are, make it Nullable by Faking It Once You Make It.
-
If the dependency is a low-level Infrastructure Wrapper with third-party dependencies, make it Nullable by introducing an Embedded Stub.
-
If the dependency is third-party infrastructure code, extract it into an Infrastructure Wrapper. Test the new Infrastructure Wrapper with Narrow Integration Tests and make it Nullable by introducing an Embedded Stub.
After you’ve updated the dependencies, Fake It Once You Make It. (If your code has a Throwaway Stub, replace it.) Replace Mocks with Nullables and add tests as needed.
When you’re done, the code you’re converting will be Nullable and tested. Its dependencies will be Nullable, but not tested. You can move on to other work. When you’re ready to convert another class or module, Descend the Ladder again. Over time, you’ll gradually convert the entire codebase.
Example
Imagine you have the dependency chain Router
→ LoginController
→ Auth0Client
→ HttpClient
, where HttpClient
is a low-level Infrastructure Wrapper. To convert Router
, you would follow these steps:
-
Router
’s direct dependency is
LoginController
, which has a mix of logic and infrastructure in its dependency chain. Make
LoginController
Nullable with a Throwaway Stub. -
Make
Router
Nullable with Fake It Once You Make It. -
Convert
Router
’s tests with Replace Mocks with Nullables.
Later, if you wanted to convert Auth0Client
, you would follow these steps:
-
Auth0Client
’s direct dependency isHttpClient
, which is a low-level Infrastructure Wrapper. Make
HttpClient
Nullable by introducing an Embedded Stub. -
Make
Auth0Client
Nullable with Fake It Once You Make It. -
Convert
Auth0Client
’s tests with Replace Mocks with Nullables.
When you wanted to convert LoginController
, you would follow these steps:
-
LoginController
’s direct dependency is
Auth0Client
, which was previously converted, so it’s already Nullable. -
LoginController
has a Throwaway Stub from when
Router
was converted. Now thatAuth0Client
is Nullable, replace the stub with Fake It Once You Make It. -
Convert
LoginController
’s tests with Replace Mocks with Nullables.
Finally, when you were ready to convert HttpClient
, you would follow these steps:
-
HttpClient
is a low-level Infrastructure Wrapper, and it was made Nullable when
Auth0Client
was converted, so it only needs to be tested. -
Test
HttpClient
with Narrow Integration Tests.
Code that’s been converted can be refactored without breaking its tests. Once you’ve converted enough code, you can refactor it to use A-Frame Architecture or any other architecture you like.
Descend the Ladder is for code with large dependency trees. If the code you’re converting has a small dependency tree, Climb the Ladder instead.
Climb the Ladder
Descending the Ladder is a careful, methodical approach to improving existing code. However, it involves creating Throwaway Stubs, which is wasteful, and it takes a long time. Simple dependency trees don’t need so much care. Therefore:
When your dependency tree is simple, convert the entire tree at once. Start by graphing out a dependency tree for the code you want to convert, ignoring third-party dependencies. Then convert each node from the bottom of the tree up. (A post-order depth-first traversal). Apply one of the following options to each node:
-
If the node is pure logic, with no infrastructure dependencies, make sure it has Easily Visible Behavior, then add tests as needed.
-
If the node is already Nullable, convert its tests by Replacing Mocks with Nullables and adding tests as needed.
-
If the node is an Infrastructure Wrapper and it uses third-party infrastructure code, make it Nullable by introducing an Embedded Stub. Test it with Narrow Integration Tests.
-
If the node isn’t an Infrastructure Wrapper, but it does use third-party infrastructure code, extract the third-party code into an Infrastructure Wrapper. Apply the above option to the Infrastructure Wrapper, then apply the below option to the remaining code.
-
If the node’s isn’t an Infrastructure Wrapper and doesn’t use third-party infrastructure code, Fake It Once You Make It. Convert its tests by Replacing Mocks with Nullables and adding tests as needed.
When you’re done, the entire dependency tree will be tested and Nullable. You can then refactor it toward A-Frame Architecture or any other architecture you like.
For example, imagine you have the dependency chain Router
→ LoginController
→ Auth0Client
→ HttpClient
, where HttpClient
is a low-level Infrastructure Wrapper. To convert Router
, you would follow these steps:
HttpClient
is a low-level Infrastructure Wrapper. Make it Nullable by introducing an Embedded Stub.- Test
HttpClient
with Narrow Integration Tests. - Make
Auth0Client
Nullable with Fake It Once You Make It. - Convert
Auth0Client
’s tests with Replace Mocks with Nullables. - Make
LoginController
Nullable with Fake It Once You Make It. - Convert
LoginController
’s tests with Replace Mocks with Nullables. - Make
Router
Nullable with Fake It Once You Make It. - Convert
Router
’s tests with Replace Mocks with Nullables.
Climb the Ladder works best when you have a small dependency tree. If you have a large dependency tree, Descend the Ladder instead.
Replace Mocks with Nullables
Existing code is often tested with mocks, spies, and other test doubles. Some of those tests will get in your way. They might be hard to understand and maintain, or they might make refactoring difficult. Therefore:
When an existing test gets in your way, use Nullables in place of the existing test doubles. Depending on the quality of the existing tests, it might be easiest to inline any setup blocks or helper methods prior to starting. Then apply the following options to each mock, spy, or other test double in each test you want to convert:
-
Start by replacing the test double with a Nulled version of the real dependency.
-
If the test double is configured to return specific values, replace the configuration with Configurable Responses.
-
If the test double is configured to emit events, replace the configuration with Behavior Simulation.
-
If the test checks how a test double is called, replace its assertions with Output Tracking. Convert these test doubles last, after test doubles with only configuration have been replaced.
For example, here’s a controller for a web page. When the user posts to the page, it uses the rot13Client
infrastructure wrapper to call a web service, then renders the result.
// Example web page controller (JavaScript + Node.js)
import * as homePageView from "home_page_view";
import Rot13Client from "rot13_client";
import HttpRequest from "http_request";
import WwwConfig from "www_config";
export default class HomePageController {
constructor(rot13Client) {
this._rot13Client = rot13Client;
}
// 'request' is an HttpRequest instance
// 'config' is a WwwConfig instance
async postAsync(request, config) {
// Parse the 'text' field from the request's JSON body
const body = await request.readBodyAsync();
const formData = new URLSearchParams(body);
const textFields = formData.getAll("text");
const userInput = textFields[0];
// Call the web service
const output = await this._rot13Client.transformAsync(config.rot13ServiceHost, userInput);
// Render the page
return homePageView.homePage(output);
}
};
The following test uses spies to check that the above code calls the web service. It’s an interaction-based test that checks whether the dependency’s methods are called correctly.
// Example of spy-based test (JavaScript + testdouble.js)
it("POST asks ROT-13 service to transform text", async () => {
// Create spies
const rot13Client = td.instance(Rot13Client);
const request = td.instance(HttpRequest);
const config = td.instance(WwwConfig);
// Configure spies
config.rot13ServiceHost = "my.rot13.host"; // rot13ServiceHost is a getter, but testdouble.js can’t configure getters’ responses, so we just set the property directly
td.when(request.readBodyAsync()).thenResolve("text=hello%20world");
// Run the code under test
const controller = new HomePageController(rot13Client);
await controller.postAsync(request, config);
// Check that the web service’s wrapper was called correctly
td.verify(rot13Client.transformAsync("my.rot13.host", "hello world"));
});
This test can be converted one spy at a time. First, we replace the HttpRequest
spy with a Configurable Response.
// Replace HttpRequest spy (JavaScript + testdouble.js)
it("POST asks ROT-13 service to transform text", async () => {
const rot13Client = td.instance(Rot13Client);
// Replace the HttpRequest spy with a real HttpRequest. (Nullable with Configurable Responses)
const request = td.instance(HttpRequest);
const request = HttpRequest.createNull({ body: "text=hello%20world" });
const config = td.instance(WwwConfig);
config.rot13ServiceHost = "my.rot13.host";
td.when(request.readBodyAsync()).thenResolve("text=hello%20world"); // Old configuration no longer needed
const controller = new HomePageController(rot13Client);
await controller.postAsync(request, config);
td.verify(rot13Client.transformAsync("my.rot13.host", "hello world"));
});
Because Nullables can coexist with test doubles, the tests still pass after this change is made. Next, we replace the WwwConfig
spy:
// Replace WwwConfig spy (JavaScript + testdouble.js)
it("POST asks ROT-13 service to transform text", async () => {
const rot13Client = td.instance(Rot13Client);
const request = HttpRequest.createNull({ body: "text=hello%20world" });
// Replace the WwwConfig spy with a real WwwConfig. (Nullable with Configurable Responses)
const config = td.instance(WwwConfig);
const config = WwwConfig.createNull({ rot13ServiceHost: "my.rot13.host" });
config.rot13ServiceHost = "my.rot13.host"; // Old configuration no longer needed (and real WwwConfig doesn't allow property to be set)
const controller = new HomePageController(rot13Client);
await controller.postAsync(request, config);
td.verify(rot13Client.transformAsync("my.rot13.host", "hello world"));
});
The tests continue to pass. Finally, we replace the Rot13Client
spy:
// Replace Rot13Client spy (JavaScript + testdouble.js)
it("POST asks ROT-13 service to transform text", async () => {
// Replace the Rot13Client spy with a real Rot13Client. (Nullable)
const rot13Client = td.instance(Rot13Client);
const rot13Client = Rot13Client.createNull();
// Track the requests made by the Rot13Client. (Output Tracking)
const rot13Requests = rot13Client.trackRequests();
const request = HttpRequest.createNull({ body: "text=hello%20world" });
const config = WwwConfig.createNull({ rot13ServiceHost: "my.rot13.host" });
const controller = new HomePageController(rot13Client);
await controller.postAsync(request, config);
// Replace the method call check with a state-based output check. (Output Tracking)
td.verify(rot13Client.transformAsync("my.rot13.host", "hello world"));
assert.deepEqual(rot13Requests, [{
host: "my.rot13.host",
text: "hello world",
});
});
Here’s a side-by-side comparison of the two tests.
// Side-by-side comparison of spy-based test and Nullables-based test (JavaScript + testdouble.js)
// Interaction-based test using spies
it("POST asks ROT-13 service to transform text", async () => {
// Create dependencies
const rot13Client = td.instance(Rot13Client);
const request = td.instance(HttpRequest);
const config = td.instance(WwwConfig);
// Configure dependencies
config.rot13ServiceHost = "my.rot13.host";
td.when(request.readBodyAsync()).thenResolve("text=hello%20world");
// Run code under test
const controller = new HomePageController(rot13Client);
await controller.postAsync(request, config);
// Check that rot13Client was called
td.verify(rot13Client.transformAsync("my.rot13.host", "hello world"));
});
// State-based test using Nullables
it("POST asks ROT-13 service to transform text", async () => {
// Create and configure dependencies
const rot13Client = Rot13Client.createNull();
const rot13Requests = rot13Client.trackRequests();
const request = HttpRequest.createNull({ body: "text=hello%20world" });
const config = WwwConfig.createNull({ rot13ServiceHost: "my.rot13.host" });
// Run code under test
const controller = new HomePageController(rot13Client);
await controller.postAsync(request, config);
// Check that rot13Client made the correct request
assert.deepEqual(rot13Requests, [{
host: "my.rot13.host",
text: "hello world",
}]);
});
To make a dependency Nullable, either Descend the Ladder or Climb the Ladder.
Throwaway Stub
Making a dependency Nullable requires making all of its infrastructure dependencies Nullable, too. Sometimes, that’s too much work to tackle all at once. Therefore:
In the code you’re making Nullable, create Embedded Stubs for any dependencies you don’t want to make Nullable. This will break the chain of Overlapping Sociable Tests, leaving you vulnerable to behavioral changes in the dependencies, so throw away the stub and replace it with Fake It Once You Make It as soon as the dependency is Nullable.
To avoid writing throwaway stubs, Climb the Ladder.
Conclusion
These patterns are an effective way of writing code that is easy to test, easy to refactor, and doesn’t require broad tests.
This article was originally published in 2018. (Read the original version here.) It was completely overhauled in 2023 to add more detail, many more examples, improved structure, and patterns for dealing with legacy code.
A big thank you to everyone who provided comments and feedback on the 2023 update of this article: Dave Foley, Aaron Jenson, Steven Harmon, Dirk Groot, Marcus Rådell, Seth McCarthy, Jay Bazuzi, Dave de Moel, Emily Bache, Steve Freeman, Martin Fowler, and many more. Thanks also to Matt Wynne and Ted M. Young for pairing with me to explore these ideas further.
Leave A Comment