Create a bundler or a processor

The two ways to teach Beyond Packages a new way of compiling a public module, shown with the bundlers it ships: a bundler written against the core module contract, a bundler composed from processors with the SDK, and a reusable processor with its inputs, outputs, diagnostics, dependencies and cleanup.

  • Availability: Experimental
  • Evidence: Recorded run
  • Tutorial

Where a bundler comes from

A package names the bundlers it uses and where each comes from, and every module selects one, or takes the default of the package:

JSONpackage.json
{
	"beyond": { "modules": ".", "bundler": "ts" },
	"bundlers": { "ts": { "specifier": "@beyond-js/packages/bundlers/ts", "runtime": "@beyond-js/local-2026/bundle" } }
}

The specifier is a public module that the running process imports natively; it must export a class named Module. A processor is also a public module, exporting a class named Processor. Neither is registered anywhere else: whoever runs Packages decides where public modules come from.

A bundler against the core contract

The core contract, from @beyond-js/packages/module, is two classes. BaseModule enumerates the conditionals of a module (a platform, optionally with an environment) and constructs one object per conditional; BaseConditional is that object: it receives the specification of the module, processes it into an output, and reports its diagnostics. Nothing else is required: the bundler owns its tools, its outputs and its watching.

The exports bundler compiles one Node conditional from a package export with esbuild:

TypeScriptindex.ts
import type { IConditions } from '@beyond-js/packages/types';
import { BaseModule } from '@beyond-js/packages/module';
import { Conditional } from './conditional';

export /*bundle*/ class Module extends BaseModule {
	_conditionals(): IConditions[] {
		return [{ platform: 'node' }];
	}

	_conditional({ key }: { key: string }): Conditional {
		if (key !== 'node') throw new Error(`Conditional ${key} not implemented`);

		return new Conditional(this, { platform: 'node' });
	}
}
TypeScriptconditional.ts
import type { IProcessedSpec } from '@beyond-js/packages/module';
import type { ExportsType, ExportsTargetType } from '@beyond-js/packages/types';
import type { IDiagnostic } from '@beyond-js/packages/types';
import type { BuildResult } from 'esbuild';
import { BaseConditional } from '@beyond-js/packages/module';
import { ConditionalOutput } from '@beyond-js/packages/module/output';
import { build } from 'esbuild';
import { Plugin } from './plugin';
import * as lexer from 'cjs-module-lexer';
import { Wrapper } from './wrapper';
import { sep } from 'path';

export class Conditional extends BaseConditional {
	get dp() {
		return 'exports-bundler.outputs';
	}

	#errors: IDiagnostic[] = [];
	get errors(): IDiagnostic[] {
		return this.#errors;
	}

	get valid(): boolean {
		return !this.#errors.length;
	}

	#target: string;
	_spec(spec: ExportsType): IProcessedSpec {
		this.#target = (<any>spec).node.require;
		return { values: this.#target };
	}

	#output: ConditionalOutput = new ConditionalOutput();
	get output(): ConditionalOutput {
		return this.#output;
	}

	async _process(): Promise<void | boolean> {
		if (!this.#target) {
			const code = 'NO_TARGET';
			const message = 'No target defined for the conditional';
			this.#errors = [{ code, message }];
			return;
		}

		const entry = this.#target;
		if (typeof entry !== 'string' || !entry) {
			const code = 'INVALID_TARGET';
			const message = `Invalid target: ${entry}`;
			this.#errors = [{ code, message }];
			return;
		}
		console.log('Building exports with entry:', entry);

		let result: BuildResult;
		const plugin = new Plugin(this);
		try {
			result = await build({
				entryPoints: [entry],
				format: 'cjs',
				sourcemap: 'external',
				logLevel: 'silent',
				platform: 'browser',
				bundle: true,
				write: false,
				outfile: 'out.js',
				plugins: [plugin]
			});
		} catch (exc) {
			console.log('Build exception', exc);
			const code = 'BUNDLER_EXCEPTION';
			const message = `Exception caught: ${exc.message}`;
			this.#errors = [{ code, message }];
			return;
		}

		const { warnings, outputFiles: outputs } = result;
		if (result.errors?.length) {
			const code = 'BUNDLER_ERRORS';
			const message = 'Errors found during the bundling process';
			this.#errors = [{ code, message }];
			return;
		}

		await lexer.init();

		const { code, map }: { code: string; map: string } = (() => {
			const output = { code: '', map: '' };
			output.code = outputs?.find(({ path }) => path.endsWith(`${sep}out.js`))?.text;
			output.map = outputs?.find(({ path }) => path.endsWith(`${sep}out.js.map`))?.text;

			const { exports } = lexer.parse(output.code);

			const wrap = new Wrapper();
			const externals = plugin.externals;

			const esm = wrap.build({ code: output.code, map: output.map, externals, exports });
			output.code = esm.code;
			output.map = esm.map;

			return output;
		})();

		this.#errors = [];
		this.#output.set({ code, map });

		require('fs').writeFileSync(`${process.cwd()}/output.js`, code);
		console.log('Build output code written to output.js');
	}
}

Its _spec keeps from the specification what it needs (the target), so a change of anything else does not reprocess it; its _process runs the tool and fills its output, a ConditionalOutput holding code and map; its errors are what the service reports as the diagnostics of the module.

A bundler composed from processors, with the SDK

The SDK, from @beyond-js/packages/sdk, is for a bundler whose conditionals are assembled from the outputs of several processors. ESMConditional is the assembly: it collects the internal modules, the stylesheets and the declarations the processors produce, in deterministic order, and generates the artifact of the conditional against the runtime the package selects, with the registration of a widget when the manifest declares one.

TypeScriptindex.ts
import type { IModuleManifestInfo } from '@beyond-js/packages/module/spec';
import { BaseModule } from '@beyond-js/packages/sdk';
import { ESM } from './esm';
import { Types } from './types';

/**
 * The TypeScript bundler: the runtime-composition mode of a public module.
 *
 * Every platform the module declares is one ESM conditional, whose artifact registers one creator per
 * source file in the runtime the package selects. A `types` conditional is added for the semantic check
 * of the sources and the public declaration of the module; it is not an executable artifact, and the
 * consumers that deliver code never select it.
 */
export /*bundle*/ class Module extends BaseModule {
	static TYPES = 'types';

	_conditionals() {
		const spec = <IModuleManifestInfo>this.spec.values;
		let platforms: string[] = spec.platforms;
		platforms = typeof platforms === 'string' ? [platforms] : platforms;
		platforms = platforms || ['default'];
		platforms = platforms?.filter(platform => platform && typeof platform === 'string' && platform !== Module.TYPES);
		platforms = platforms instanceof Array && platforms.length ? platforms : ['default'];

		return [...platforms.map(platform => ({ platform })), { platform: Module.TYPES }];
	}

	_conditional({ conditions }: { key: string; conditions: { platform: string } }) {
		const { platform } = conditions;
		return platform === Module.TYPES ? new Types(this, { platform }) : new ESM(this, { platform });
	}
}
TypeScriptesm.ts
import type { IProcessorsSetup } from '@beyond-js/packages/sdk';
import type { IProcessedSpec } from '@beyond-js/packages/module';
import { ESMConditional } from '@beyond-js/packages/sdk';
import { Spec } from './spec';

/**
 * The processors of the TypeScript bundler, by name and public module. `ts` transforms the TypeScript and
 * TSX sources; `styles` compiles the CSS and SCSS sources, with Tailwind where a stylesheet imports it;
 * `vue` and `svelte` compile the single-file components of those frameworks into internal modules and
 * stylesheets. A processor whose inputs are absent from a module produces nothing and loads no compiler.
 */
const PROCESSORS = {
	ts: '@beyond-js/packages/bundlers/ts/processors/ts',
	styles: '@beyond-js/packages/bundlers/ts/processors/styles',
	vue: '@beyond-js/packages/bundlers/ts/processors/vue',
	svelte: '@beyond-js/packages/bundlers/ts/processors/svelte'
};

/**
 * The executable conditional of a module compiled by the TypeScript bundler, for one platform
 */
export /*bundle*/ class ESM extends ESMConditional {
	_spec(values: Record<string, any>): IProcessedSpec {
		return Spec.values(values, this.platform, this.environment);
	}

	/**
	 * Every processor receives the values of the conditional that are not reserved for the module, and
	 * the style processor also receives the tailwind inputs the manifest declares
	 */
	_processors(): IProcessorsSetup {
		const values = <Record<string, any>>this.spec.values;
		const processors = new Map(
			Object.entries(PROCESSORS).map(([name, specifier]) => {
				const extra = name === 'styles' && values?.tailwind !== void 0 ? { tailwind: values.tailwind } : {};
				return [name, Spec.processor(values, specifier, extra)];
			})
		);
		return { processors };
	}
}

A bundler like this decides three things: which conditionals a module has (_conditionals), which processors each conditional runs and with what values (_processors), and how the values of the manifest are projected for one conditional (_spec, which is how conditionals.web and conditionals.node give one module one entry per platform).

A reusable processor

A processor, from @beyond-js/packages/sdk, extends ConditionalProcessor. Its constructor declares its sources: the extensions of the files it takes from the module directory (inputs), and fixed auxiliary files such as a tsconfig.json (files), all of them watched. Its _build reads the inputs and writes into outputs: ims for internal modules, styles for stylesheets, types for declarations, each output with its code, its map and its issues.

TypeScriptindex.ts
import type { Conditional, ProcessorOutputs } from '@beyond-js/packages/sdk';
import type { IRequest } from '@beyond-js/dynamic-processor/main';
import type { IDiagnostic } from '@beyond-js/packages/types';
import { ConditionalProcessor } from '@beyond-js/packages/sdk';
import { join } from 'path';
import { Dependencies } from './dependencies';
import { Sass } from './sass';
import { Tailwind, type ITailwindSettings } from './tailwind';

export type { ITailwindSettings } from './tailwind';

/**
 * Compiles the stylesheets of a public module: its `.css` and `.scss` sources, each into one style output
 * that the conditional concatenates, in file order, into the stylesheet of the module.
 *
 * A source is compiled with Sass, or with Tailwind when it imports it. Partials (`_name.scss`) produce
 * nothing of their own. What a compilation reads besides its source (partials, a theme, the sources
 * Tailwind scans, a plugin) is watched as a dependency of the processor, so editing any of it rebuilds
 * the stylesheet; a dependency a later build no longer reads is released. Nothing is scanned outside the
 * module directory: the Tailwind sources are the ones the module manifest declares.
 */
export /*bundle*/ class Processor extends ConditionalProcessor {
	#dependencies: Dependencies;

	/**
	 * The files the last build read besides its inputs
	 */
	get dependencies(): string[] {
		return this.#dependencies.files;
	}

	#diagnostics: IDiagnostic[] = [];
	get errors(): IDiagnostic[] {
		return this.#diagnostics.concat(super.errors);
	}

	constructor(conditional: Conditional, name: string) {
		super(conditional, name, { sources: { inputs: { extname: ['.css', '.scss', '.sass'] } } });
		this.#dependencies = new Dependencies(this);
	}

	/**
	 * Besides the selection of the inputs, the processor keeps the tailwind inputs the manifest declares,
	 * so changing them reprocesses the module
	 */
	_spec(values: any) {
		const { values: output, errors, warnings } = super._spec(values);
		const errors2: IDiagnostic[] = errors ? [...errors] : [];
		if (values?.tailwind !== void 0) {
			const { tailwind } = values;
			const valid = tailwind && typeof tailwind === 'object' && !(tailwind instanceof Array) &&
				(tailwind.sources === void 0 || (tailwind.sources instanceof Array && tailwind.sources.every((source: unknown) => typeof source === 'string' && source)));
			valid ? (output.tailwind = { sources: tailwind.sources }) : errors2.push({ code: 'TAILWIND_INVALID', message: 'The "tailwind" of the module manifest must be an object whose "sources" are paths relative to the module' });
		}
		return { values: output, errors: errors2, warnings };
	}

	async _build(request: IRequest, outputs: ProcessorOutputs): Promise<void> {
		const { module } = this.conditional;
		const directory = join(module.package.path, module.spec.path ?? '');
		const settings = <ITailwindSettings>(<{ tailwind?: ITailwindSettings }>this.spec.values).tailwind;
		const tailwind = new Tailwind(directory, module.package.path);
		const read = new Set<string>();
		const diagnostics: IDiagnostic[] = [];

		for (const input of [...this.sources.inputs.values()].sort((a, b) => a.relative.file.localeCompare(b.relative.file))) {
			if (Sass.partial(input.file)) continue;

			const output = outputs.styles.obtain(input);
			if (!input.valid) {
				output.issues.push('errors', { code: 'SOURCE_ERROR', message: input.errors.join('; ') });
				continue;
			}

			const relative = input.relative.file.replace(/\\/g, '/');
			const compiled = Tailwind.uses(input.content)
				? await tailwind.compile(input.file, input.content, settings, relative)
				: Sass.compile(input.file, input.content, relative);
			if (request !== this._request) return;

			compiled.read.forEach(file => read.add(file));
			compiled.diagnostics.forEach(diagnostic => output.issues.push('errors', diagnostic));
			typeof compiled.code === 'string' && output.code.set({ code: compiled.code, map: compiled.map });
		}

		// The dependencies of this build are watched from now on; the ones of the previous build no longer read are released
		try {
			await this.#dependencies.update(read);
		} catch (error) {
			diagnostics.push({ code: 'STYLE_DEPENDENCY_ERROR', message: error.message });
		}
		if (request !== this._request) return;
		this.#diagnostics = diagnostics;
	}

	destroy() {
		super.destroy();
		this.#dependencies.destroy();
	}
}

What this processor shows about the contract:

  • Inputs and configuration. _spec keeps the tailwind inputs the manifest declares, so editing them reprocesses the module; the path, files and excludes of the inputs are kept by the base class.
  • Outputs and diagnostics. Each source produces one output with its code and map; a problem of one source is an issue of that output, positioned; a problem of the build is a diagnostic of the processor, and the conditional reports both.
  • Dependencies and invalidation. What a compilation reads besides its inputs (partials, a theme, the scanned sources) is registered as a dependency and watched; a dependency a later build no longer reads is released. Editing any of them invalidates the processor, which builds again; a build whose request is no longer current publishes nothing.
  • Disposal. destroy releases the dependencies and calls the base class, which releases the sources and their listeners.

What the assembly gives every processor of the ts bundler

  • The artifact of a conditional imports the runtime the package selects; a source written against @beyond-js/kernel/{bundle,core,styles} is assembled against that runtime.
  • One stylesheet per module, concatenated in file order from the style outputs, with its map; delivered beside the code.
  • A types conditional per module that checks the sources semantically and emits the public declaration of the module; it is not an executable artifact.
  • A widget declared in the manifest is registered before the module initialises, with its element, its attributes and whether its package publishes a shared stylesheet.

Next

See what the artifacts look like at their addresses: URLs and identities.