Test a package

Write ordinary test files for a Beyond package and run them with beyond test, Node's own test runner over the compiled public modules; locate a failure in the TypeScript source, narrow a run, mock a public module, assert asynchronous behavior, read coverage on the sources and attach a debugger.

Availability: Experimental
Evidence: Recorded run
Tutorial

What you build

A workspace of two packages, @qa/shared and @qa/app, with five ordinary test files: the contract of a public module, a test beside the sources of that module, a test that mocks a public dependency, a test of asynchronous behavior and a JavaScript test that copies a mutable fixture. You run them with beyond test, make one fail, narrow the run, read a coverage report that names the TypeScript sources, and attach a debugger.

Nothing here is specific to Beyond except the command that runs the files: the runner is Node's own (node:test), the assertions are node:assert, and a test imports a public module by the same bare specifier an application imports.

Declare the packages

The workspace lists its packages; each package publishes its modules through exports, selects the ts bundler and declares "type": "module", so that Node reads a .ts test file as an ES module without guessing. The beyond.modules entry of the shared package says where its module manifests are, which the last step uses.

JSONbeyond.json

{
	"packages": ["shared", "app"]
}

JSONpackage.json

{
	"name": "@qa/shared",
	"version": "1.0.0",
	"type": "module",
	"exports": {
		"./text": "./text/index.ts"
	},
	"beyond": {
		"bundler": "ts",
		"modules": "."
	},
	"bundlers": {
		"ts": "@beyond-js/packages/bundlers/ts"
	}
}

JSONpackage.json

{
	"name": "@qa/app",
	"version": "1.0.0",
	"type": "module",
	"exports": {
		"./main": "./main/index.ts",
		"./clock": "./clock/index.ts"
	},
	"dependencies": {
		"@qa/shared": "1.0.0"
	},
	"beyond": {
		"bundler": "ts"
	},
	"bundlers": {
		"ts": "@beyond-js/packages/bundlers/ts"
	}
}

Write the module under test

@qa/shared/text is one public module of three internal files, one of them in a subdirectory that has an index.ts of its own. shout() is never called by any test: it is the probe that the coverage report must mark.

TypeScriptindex.ts

import { decorate } from './decorate';
import { deep } from './sub/deep';
import { title } from './sub';

/** Greets by name */
export const greet = (name: string) => decorate(name.trim() && `Hello ${name.trim()}`);

/** Never called by the tests: the coverage probe */
export const shout = (name: string) => {
	const upper = name.toUpperCase();
	return deep(title(upper));
};

export default 'shared default';

TypeScriptdecorate.ts

/** Adds the exclamation; it is internal to the module and not importable by a consumer */
export const decorate = (text: string) => {
	if (!text.trim()) throw new Error('nothing to decorate');
	return `${text}!`;
};

TypeScriptindex.ts

/** A second `index.ts`, in a subdirectory: its map must keep the directory */
export const title = (text: string) => `== ${text} ==`;

TypeScriptdeep.ts

/** A source in a subdirectory of the module */
export const deep = (text: string) => `${text}?`;

Write the tests

A test of the public contract sits beside the module directory and imports the module by its bare specifier. The internal file decorate.ts is not importable by a test any more than by a consumer; its behavior is observed through greet.

TypeScripttext.test.ts

// The contract of the public module @qa/shared/text, imported exactly as a consumer imports it: by its bare
// specifier, resolved to the module the development service compiled from the sources beside this file.
import { describe, test } from 'node:test';
import assert from 'node:assert/strict';
import label, { greet } from '@qa/shared/text';

describe('@qa/shared/text', () => {
	test('greets by name', () => {
		assert.equal(greet('QA'), 'Hello QA!');
	});

	test('exports a default value', () => {
		assert.equal(label, 'shared default');
	});

	test('refuses an empty name, with the error the internal file throws', () => {
		// The internal file is not importable; its behavior is observed through the public function
		assert.throws(() => greet('   '), { message: 'nothing to decorate' });
	});
});

A test may also sit inside the module directory. It is collected and run like any other test file, and it is never compiled into the module: the artifact of @qa/shared/text does not contain it.

TypeScriptindex.test.ts

// A test beside the sources of the module. It is collected and run like any other test file, and it is
// never compiled into the public module: the artifact of @qa/shared/text does not contain it.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { greet } from '@qa/shared/text';

test('a colocated test imports the public module, not the file beside it', () => {
	assert.equal(greet('here'), 'Hello here!');
});

The application imports the shared module. Its test replaces that dependency with mock.module() of node:test, registered before the module under test is imported, which is why the import is dynamic and comes after the mock. The compiled application keeps its bare import of @qa/shared/text, so the mock reaches every internal file of the application.

TypeScriptindex.ts

import { greet } from '@qa/shared/text';

/** Decorates the greeting of the shared module: the bare import survives compilation */
export const main = (name: string) => `[app] ${greet(name)}`;

TypeScriptmain.test.ts

// A dependency mocked at the public boundary: @qa/app/main keeps its bare import of @qa/shared/text, so the
// runner's own module mock replaces it for every internal file of the application. The mock is registered
// before the module under test is imported, which is why the import is dynamic and comes after it.
import { test, mock } from 'node:test';
import assert from 'node:assert/strict';

mock.module('@qa/shared/text', {
	namedExports: { greet: (name: string) => `mocked ${name}` },
	defaultExport: 'mocked default'
});

const { main } = await import('@qa/app/main');

test('the application uses the mocked shared module', () => {
	assert.equal(main('QA'), '[app] mocked QA');
});

Asynchronous behavior is awaited and asserted, never observed through logs or fixed waits: a value is awaited, a rejection is asserted with assert.rejects by its code, and a slow operation is bounded with the test's own timeout, which fails the test instead of hanging it.

TypeScriptindex.ts

/** Resolves with a reading once the clock is ready; it settles on a later turn, never synchronously */
export const reading = (): Promise<string> => new Promise(resolve => setTimeout(() => resolve('tick'), 20));

/** Rejects with a coded error, which a test asserts explicitly */
export const broken = (): Promise<never> => Promise.reject(Object.assign(new Error('the clock is broken'), { code: 'CLOCK_BROKEN' }));

/** Settles after the given delay, which a test bounds with a timeout */
export const slow = (ms: number): Promise<string> => new Promise(resolve => setTimeout(() => resolve('late'), ms));

TypeScriptclock.test.ts

// Asynchronous behavior is awaited and asserted, never observed through logs or fixed waits.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { reading, broken, slow } from '@qa/app/clock';

test('a reading is awaited', async () => {
	assert.equal(await reading(), 'tick');
});

test('a rejection is asserted explicitly, by its code', async () => {
	await assert.rejects(broken(), { code: 'CLOCK_BROKEN' });
});

test('a bounded wait fails the test instead of hanging it', { timeout: 200 }, async () => {
	assert.equal(await slow(10), 'late');
});

A JavaScript test works the same way. This one copies a fixture to a temporary directory before editing it, so tests never interfere through shared files, and removes the copy when it ends.

JavaScriptfixture.test.mjs

// A mutable fixture is copied to a directory of its own before a test changes it, so tests never interfere
// through shared files and the permanent template stays as it is. Whoever creates the copy removes it.
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { cp, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { fileURLToPath } from 'node:url';

const template = fileURLToPath(new URL('./fixtures/', import.meta.url));

test('a copied fixture is edited in isolation and removed', async t => {
	const directory = await mkdtemp(join(tmpdir(), 'qa-fixture-'));
	t.after(() => rm(directory, { recursive: true, force: true }));
	await cp(template, directory, { recursive: true });

	const notes = join(directory, 'notes.txt');
	await writeFile(notes, `${await readFile(notes, 'utf8')}second line\n`);
	assert.equal(await readFile(notes, 'utf8'), 'first line\nsecond line\n');
	assert.equal(await readFile(join(template, 'notes.txt'), 'utf8'), 'first line\n', 'the template is untouched');
});

The template it copies is one line of text:

Textnotes.txt

first line

Run the tests

Shellrun.sh

# From the workspace directory, with the Beyond toolchain installed in <installation>
# (its acceptance builds such an installation; nothing is published to a registry)
BEYOND="<installation>/node_modules/.bin/beyond"

$BEYOND test                              # every <name>.test.ts, .mts, .js or .mjs file of the workspace
$BEYOND test shared app/clock.test.ts     # only the files under a directory, or one file
$BEYOND test --name "rejection"           # only the tests whose name matches
$BEYOND test --coverage shared            # with a coverage report of the workspace sources
$BEYOND test --reporter spec              # a reporter of Node's test runner: spec, tap, dot, junit, lcov
$BEYOND test tests -- --inspect-brk       # arguments after -- are given to Node, before --test

beyond test reuses the development server of the workspace or starts one, asks it to build every public module, collects the test files (<name>.test.ts, .mts, .js or .mjs, in sorted order, never under node_modules or a hidden directory) and runs them with node --test, one process per file. Its standard error says which server it used and which files it collected; the standard output is the runner's report, spec on a terminal and tap elsewhere:

Text

beyond: started development server at http://127.0.0.1:59360
beyond: 5 test files: app/clock.test.ts, app/main.test.ts, shared/text.test.ts, shared/text/index.test.ts, tests/fixture.test.mjs
…
# tests 9
# pass 9
# fail 0

The exit code is the runner's: 0 when every test passed. The server started here ends shortly after the run; a server you keep open with beyond run is used and survives.

Checkpoint. Nine tests pass in five files, TypeScript and JavaScript alike, and the command ends.

Make a test fail

Add a file with a wrong expectation and a call that throws inside the compiled module:

TypeScripttests/failing.test.ts

import { test } from 'node:test';
import assert from 'node:assert/strict';
import { greet } from '@qa/shared/text';

test('wrong expectation', () => {
	assert.equal(greet('QA'), 'Hi QA');
});

test('an error thrown by the compiled module', () => {
	greet('   ');
});

Shell

beyond test tests/failing.test.ts --reporter spec

Text

✖ wrong expectation
    at TestContext.<anonymous> (file:///…/tests/failing.test.ts:6:9)
✖ an error thrown by the compiled module
  Error: nothing to decorate
      at decorate (/…/shared/text/decorate.ts:3:26)
      at greet (/…/shared/text/index.ts:6:48)
      at TestContext.<anonymous> (file:///…/tests/failing.test.ts:10:2)
ℹ pass 0
ℹ fail 2

The assertion is located in the test file, and the error thrown by the compiled module is located in decorate.ts and in the greet that called it: the source maps of the compiled module are applied to the stack trace. The exit code is 1. Remove the file before going on.

Checkpoint. Exit code 1, two failures, each with the line of the source that produced it.

Narrow the run

A directory collects only its files; a file names one; --name selects the tests whose name matches:

Shell

beyond test shared                          # 2 test files: shared/text.test.ts, shared/text/index.test.ts
beyond test app/clock.test.ts --name rejection   # 1 test file, 1 test

A path that does not exist, a file that is not a test, and a directory without a test file are failures with a message, never an empty green run:

Text

beyond: error: no test files found in tests/empty (a test file is named <name>.test.ts, .mts, .js or .mjs)

What a workspace that does not build does

Break shared/text/decorate.ts (delete the closing quote of a string, for example) and run the tests again. Nothing runs: the command reports the diagnostic located in the source, ends with exit code 1, and never serves an older artifact in place of the broken module. Correct the file and the same command runs the tests again.

Text

beyond: error: "@qa/shared/text" does not build (BUILD_FAILED)
beyond: error: shared/text/decorate.ts:1:50 TRANSPILE_ERROR: Module "@qa/shared/text": decorate.ts (1:50): Expression expected.
beyond: error: nothing was run: correct the sources and run the tests again

Read the coverage

Shell

beyond test shared --coverage

The report is Node's, remapped to the TypeScript sources of the workspace: test files and installed packages are excluded, the subdirectory keeps its own row, the two index.ts files are two rows, and the body of shout() (lines 10 and 11 of shared/text/index.ts), which no test calls, is uncovered:

Text

# file           | line % | branch % | funcs % | uncovered lines
# shared         |        |          |         |
#  text          |        |          |         |
#   decorate.ts  | 100.00 |   100.00 |  100.00 |
#   index.ts     |  85.71 |   100.00 |   50.00 | 10-11
#   sub          |        |          |         |
#    deep.ts     | 100.00 |   100.00 |    0.00 |
#    index.ts    | 100.00 |   100.00 |   25.00 |
# all files      |  91.30 |   100.00 |   37.50 |

Lines are as Node reports them. A function that was never called always shows in the function count of its file, as deep() and title() do in the subdirectory; it shows in the uncovered lines only when its body has lines of its own, which is why the probe is written on several lines. --reporter lcov writes the same data as LCOV records that name the same sources.

Attach a debugger

Arguments after -- are given to Node before --test, so the inspector attaches to the test process and breakpoints bind to the TypeScript sources through the source maps:

Shell

beyond test tests/fixture.test.mjs -- --inspect-brk

Text

Debugger listening on ws://127.0.0.1:<port>/…

Open the address in a debugger, or use the one of your editor, and run to your breakpoint.

Compile a test file on purpose

The runner collects a test file inside a module directory, but the compiler leaves it out of the module: <name>.test.*, <name>.spec.* and everything under __tests__ or __fixtures__ are not inputs of a processor. A module that must compile such files says so in its manifest, which is why the shared package declares "beyond": { "modules": "." }:

JSONmodule.json

{
	"tests": "included"
}

Any other value of tests is the diagnostic INVALID_TESTS_CONFIGURATION, and the workspace does not build until it is corrected.

Clean up

Every run loads the current build and there is no watch mode: run the tests again after an edit. The served modules are identified in the test processes by placeholder files under .beyond/modules/ of the workspace; add .beyond/ to the ignore file of the project. The server that beyond test started ended on its own after the run.

Every option, message and rule of the command: The beyond test reference.

What you build#

Declare the packages#

Write the module under test#

Write the tests#

Run the tests#

Make a test fail#

Narrow the run#

What a workspace that does not build does#

Read the coverage#

Attach a debugger#

Compile a test file on purpose#

Clean up#

Next#

What you build

Declare the packages

Write the module under test

Write the tests

Run the tests

Make a test fail

Narrow the run

What a workspace that does not build does

Read the coverage

Attach a debugger

Compile a test file on purpose

Clean up

Next