Menu

Aussom-Script Testing Best Practices

A practical guide to writing, running, and debugging tests in Aussom-Script projects. The goal is to help you pick the right kind of test for the job, write tests that read well and fail clearly, and debug the kinds of problems that are specific to running Aussom under TeaVM in a browser.


1. Introduction and Purpose

Aussom-Script is Aussom compiled to JavaScript via TeaVM. Code runs inside a browser page as normal JavaScript, but it is driven by Aussom source, not JavaScript source. That single fact shapes everything about how tests work:

  • Aussom test code lives in the same page as the code under test. There is no separate "test harness" process; the page IS the harness.
  • The browser event loop is single-threaded, and TeaVM's coroutine mechanism has hard limits when the call stack passes through reflection. That rules out the "block on an async call inside a test" pattern common in server-side Java, and nudges us toward explicit callback / done-style tests.
  • Failures may surface far from where they happened: an Aussom exception silently aborts the enclosing statement block, a @JSBody exception turns into a RuntimeException that a caller has to catch, and a @JSFunctor lambda that mishandles virtual dispatch fails with an opaque JavaScript TypeError.

This guide covers the testing options that ship with aussom-script today, recommends when to use each, and calls out the browser/TeaVM-specific pitfalls that tend to bite new authors.

The tools you will see here are:

  • aunit -- synchronous assertion library with @Test / @Before / @After annotation support. Good for pure logic and simple DOM checks.
  • TestHarness -- isolated mount container for component tests.
  • aspec -- Jasmine-shaped async test runner with describe, it, itAsync, done, beforeEach, afterEach. Good for any test that involves a callback, a Timer, an HTTP call, or a Promise bridge.
  • Playwright (CLI-driven) -- an Aussom script that launches a real Chromium, loads a page, and watches the console for pass/fail output. Good for end-to-end verification in CI.

Pick one tier; mix them when the project gets bigger.


2. The Four Tiers of Testing

Think of an Aussom-Script project as stacking four tiers, cheapest at the bottom:

Tier What it verifies Typical runtime Tool
1. Unit tests Pure logic, no DOM, no network Milliseconds aunit @Test + test.expect*
2. Component tests One UI component mounted alone, asserting on its DOM Tens of ms aunit @Test + TestHarness
3. Async specs Callbacks, timers, HTTP, Promises, event listeners Hundreds of ms aspec + done token
4. End-to-end tests Full page load, real browser, navigation, interaction Seconds Playwright driver (aussom playwright/*.aus)

Rules of thumb:

  • Every new piece of logic should get a tier-1 test first. If a logic bug was caught by a Playwright test, that is a sign you are owed a tier-1 test covering the same bug.
  • Component tests should not talk to the network; mock at the boundary.
  • Async specs should not duplicate what a unit test already covers. Use itAsync only when there is real async work.
  • Reach for Playwright when the bug you are chasing actually requires a real browser (router, clipboard, focus ring, layout, routing, CSS).

3. Unit Tests with aunit

3.1 The page shape

An aunit unit-test page has three parts: it includes aunit, it defines one or more test classes with @Test methods, and its <body onload> fires the test runner:

<!DOCTYPE html>
<html>
<head>
  <script src="teavm/aussom-script.js"></script>
  <script type="text/aussom-script">
    include aunit;

    class Tests {
        @Before
        public setUp() { c.log("before each test"); }

        @After
        public tearDown() { c.log("after each test"); }

        @Test(name = "addition works")
        public addsCorrectly() {
            test.expect(1 + 1, 2);
        }
    }
  </script>
</head>
<body onload="main(['runTests', 'logLevel=INFO'])">
</body>
</html>

The runTests argument flips the runtime into test mode. The runner walks every @Test method, logs PASSED / FAILED per method, then logs a summary line:

PASSED: N SKIPPED: 0 FAILED: K TOTAL: T

That summary line is what Playwright runners grep for; keep it intact if you wrap the page in anything custom.

3.2 Assertion vocabulary

Every test.expect* throws on failure and returns true on pass. Use them directly; do not wrap them in if statements. If the assertion throws, Aussom's automatic exception propagation bubbles out of the test method and the runner records it as FAILED.

Core assertions:

test.expect(actual, expected);              /* equality */
test.expectNotNull(value);
test.expectNull(value);
test.expectTrue(cond);                      /* readability */
test.expectFalse(cond);
test.expectClose(a, b, 0.0001);             /* floating-point */
test.expectString(x); test.expectInt(x); test.expectBool(x);
test.expectDouble(x); test.expectNumber(x);
test.expectList(x); test.expectMap(x);
test.expectObject(x, "ClassName");
test.expectCallback(x);
test.expectContains(list, item);
test.expectKey(map, key);
test.expectSize(collection, n);             /* list, map, or string */
test.expectMatches(s, regexStr);            /* regex match */
test.expectThrows(::cb);                    /* callback must throw */
test.expectThrowsMessage(::cb, "substr");   /* matches the message */
test.fail("explicit failure");

DOM helpers (accept CSS selector or HNode):

test.expectElementExists(selectorOrHNode);
test.expectElementMissing(selector);
test.expectElementText(selectorOrHNode, "exact innerHTML");
test.expectElementTextContains(selectorOrHNode, "substr");
test.expectElementHasClass(selectorOrHNode, "my-class");
test.expectElementLacksClass(selectorOrHNode, "my-class");
test.expectElementAttr(selectorOrHNode, "data-id", "abc");
test.expectElementCount("#root .item", 3);

3.3 Always pass a Msg on non-obvious assertions

Every expect* takes an optional trailing Msg argument that is prepended to the failure text. Use it liberally. A generic Items are not equal (got 42, expected 43) tells you nothing about intent; invoice total after tax :: Items are not equal (got 42, expected 43) tells you exactly which check failed.

/* Good */
test.expectClose(total, 42.0, 0.001, "invoice total after tax");

/* Less helpful */
test.expectClose(total, 42.0, 0.001);

This costs nothing at author time and pays off when a test starts flaking in CI six months later.

3.4 Testing exception paths

Aussom propagates AussomException automatically. You cannot check result.isException() because the result is never bound -- the exception aborts the statement before the assignment. Use expectThrows:

class Service {
    public fetch(string Id) { throw "not found"; }
}

@Test(name = "fetch of missing id throws")
public fetchMissingThrows() {
    s = new Service();
    test.expectThrows(::callFetch, "missing id should throw");
}

public callFetch() {
    s = new Service();
    s.fetch("nope");
}

Or check the message:

@Test(name = "fetch of missing id throws with 'not found'")
public fetchMissingMessage() {
    test.expectThrowsMessage(::callFetch, "not found");
}

Note the callback trick: expectThrows takes a callback, not a block. Define a helper method, then pass ::helper to the assertion. This is the only idiomatic way to assert throws without reaching for try-catch boilerplate.

3.5 @Before and @After limits

@Before fires once per test method (before), @After fires once after. They give you a place to reset per-test state:

class Tests {
    public db = null;

    @Before
    public setUp() { this.db = new InMemoryDb(); }

    @After
    public tearDown() { this.db = null; }

    @Test(name = "insert")
    public insert() { this.db.insert("k", 1); test.expect(this.db.get("k"), 1); }
}

What @Before / @After do not give you:

  • A per-test-failure hook. If you need "run captureState() after any spec fails", use aspec.onFail (section 5.5). The annotation runner does not expose per-test failure callbacks.
  • A tag-based filter. Tests cannot be filtered at the runner level. Put slow or optional tests in their own file, or move to aspec which supports itSkip / itOnly.
  • A per-test timeout. Annotation-runner tests run to completion or throw. aspec.itAsync supports a per-spec timeout.

When you hit those limits, step up to aspec.

3.6 Running unit tests

  • Browser: open the HTML page, watch DevTools console.
  • CLI: aussom -t path/to/page-or-aus-file runs pure-logic tests headless (no DOM helpers -- they need a real Document).
  • Playwright: aussom playwright/your-test.aus wraps the browser run, capturing the summary line for CI gating. See section 6.

4. Component Tests with TestHarness

A component test mounts one UI component into an isolated container, drives it, and asserts on the resulting DOM. It is the "missing middle" between pure-logic unit tests and full-page end-to-end tests.

4.1 When to reach for a component test

  • CSS / layout regressions (classlist presence, attribute on rendered element, z-index ordering by parent).
  • Render-state that depends only on input props (row variants, pill colors, disabled-button gating).
  • DOM-touching helpers whose effect is a side-mutation (busy-marker setters, focus managers).

Do not reach for a component test when:

  • The logic under test takes no DOM and returns a value. That is a unit test. Write that first; it is faster.
  • The scenario requires routing, session state, or a real network request. That is a Playwright test.

4.2 The TestHarness API

include tharness;

@Before
public setUp() { this.h = new TestHarness(); }

@After
public tearDown() { this.h.destroy(); }

TestHarness provides:

  • mount(node) -- append a component into the isolated container. Replaces anything previously mounted.
  • destroy() -- remove the container from document.body. Idempotent (safe to call twice).
  • find(selector) / findAll(selector) -- CSS queries scoped to #test-harness-mount, so a stale component left by another test cannot pollute the query.
  • click(selOrEl) -- fire a real DOM click event.
  • fill(selOrEl, value) -- set an input value and dispatch both input and change events.
  • selectOption(selOrEl, value) -- set a <select>'s value and dispatch change.
  • toggle(selOrEl) -- click a checkbox / radio.
  • textOf(selOrEl) -- return the element's innerHTML.
  • classesOf(selOrEl) -- return the class list as a list.
  • attrOf(selOrEl, attr) -- return an attribute value or null.
  • html() -- dump the mount container's outerHTML for failure diagnostics.

4.3 Worked example

include hnodes;
include aunit;
include tharness;

class HelloBadge : HNode {
    public HelloBadge(string Name) {
        this.obj = new Element("span");
        this.addClass("hello-badge");
        this.setHtml("Hello, " + Name);
    }
}

class HelloBadgeTests {
    public h = null;

    @Before
    public setUp() { this.h = new TestHarness(); }

    @After
    public tearDown() { this.h.destroy(); }

    @Test(name = "renders the name")
    public rendersName() {
        this.h.mount(new HelloBadge("Sarah"));
        test.expectElementText(".hello-badge", "Hello, Sarah");
    }

    @Test(name = "applies the component class")
    public hasClass() {
        this.h.mount(new HelloBadge("Sarah"));
        test.expectElementHasClass(".hello-badge", "hello-badge");
    }
}

A few conventions worth following:

  • Always pair @Before new TestHarness() with @After this.h.destroy();. Without the destroy you leak a #test-harness-mount div for every test.
  • Query using the aunit selector helpers rather than this.h.find(...). The aunit selectors produce better error messages when they miss.
  • Use this.h.find for actions (click, fill) and aunit test.expectElement* for assertions. That split keeps tests reading like "do this, then check that".

4.4 Common surprise: event.target does not exist

The TestHarness's click helper fires a real DOM event, and the listener receives a genuine AEvent. But AEvent.getTarget() is not wired through at this time. Write listeners that remember their binding at registration time rather than reaching back through the event:

/* Good: listener closes over the bound value. */
public buildRow(string Id) {
    row = new Div();
    row.addListener("click", ::onRowClick, Id);
    return row;
}
public onRowClick(eventName, event, id) {
    this.selected = id;
}

/* Avoid: assumes event.target exists. */
public onRowClick(eventName, event, passed) {
    id = event.getTarget().getAttr("data-id");  /* does not work */
}

5. Async Tests with aspec

aspec is the Jasmine-shaped runner that lives outside the @Test annotation path. Use it any time an assertion needs to happen after an async callback fires.

5.1 Why aspec exists

TeaVM's @Async mechanism cannot suspend across the reflection boundary that the Aussom interpreter uses to dispatch method calls. That makes it impossible to write a test.waitFor(...) helper that blocks a test method. Every async test in Aussom has to return control to the event loop and be resumed when the callback fires.

aspec handles that bookkeeping for you: it queues specs, walks them one at a time, yields to the event loop with Timer(0, ...) between specs so pending HTTP / Timer / Promise callbacks can fire, and uses a done token that a spec calls when the async work completes.

5.2 The shape of an aspec page

<script type="text/aussom-script">
    include aspec;
    include thread;
    include html;

    class Specs {
        public lastDone = null;

        public main(args) {
            aspec.describe("HTTP client", ::httpGroup);
            aspec.run();
        }

        public httpGroup() {
            aspec.beforeEach(::setUp);
            aspec.afterEach(::tearDown);

            aspec.it("sync: parses a number",        ::specParse);
            aspec.itAsync("async: loads user data",  ::specLoadUser, 3000);
        }

        public setUp()    { /* ... */ }
        public tearDown() { /* ... */ }

        public specParse() {
            test.expect(Int.parse("42"), 42);
        }

        public specLoadUser(done) {
            this.lastDone = done;
            Http.get("/api/user/1", ::onUserOk, ::onUserErr);
        }

        public onUserOk(resp) {
            try {
                test.expect(resp.statusCode, 200);
                this.lastDone.ok();
            } catch (e) {
                this.lastDone.fail(e.getText());
            }
        }

        public onUserErr(msg) { this.lastDone.fail("HTTP error: " + msg); }
    }
</script>
<body onload="main([])">

Note the page uses main([]), not main(['runTests']). aspec is not the @Test runner; it is its own runner driven from your main(). Playwright code that watches for the PASSED: N FAILED: K TOTAL: T summary line works unchanged because aspec emits that same format.

5.3 Sync vs async specs

  • aspec.it(name, ::fn) -- sync. fn takes no arguments. Spec is considered passed when fn returns normally; failed if fn throws.
  • aspec.itAsync(name, ::fn, timeoutMs = 2000) -- async. fn takes one argument: a DoneToken. Spec is passed when done.ok() is called, failed when done.fail(msg) is called or the timeout elapses.

Only reach for itAsync when the test genuinely awaits something. Synchronous tests are simpler and faster.

5.4 The done token

done is a single-fire token. Subsequent calls are ignored (a forgotten-cleanup callback firing twice will not double-advance the runner).

public specTimerFires(done) {
    this.t = new Timer(50, ::onTimerFired);
    this.t.start();
    this.lastDone = done;
}

public onTimerFired() {
    try {
        test.expectElementText(".status", "done");
        this.lastDone.ok();
    } catch (e) {
        this.lastDone.fail(e.getText());
    }
}

Always wrap the assertion chain inside the callback in try-catch and forward caught exceptions through done.fail. If the assertion throws and you do not catch it, the exception surfaces on the listener's call stack, not the test runner's, and the spec hangs until timeout.

5.5 Lifecycle hooks

aspec.beforeAll(::init);         /* fires once before the first spec of the group */
aspec.beforeEach(::setUp);       /* fires before each spec */
aspec.afterEach(::tearDown);     /* fires after each spec */
aspec.afterAll(::cleanup);       /* fires once after the last spec of the group */
aspec.onFail(::captureState);    /* fires after each FAILED spec in the group */

Hooks must be registered inside a describe callback, not at top level. aspec.onFail replaces the "manually call captureState() before every assertion" pattern -- register it once per describe and forget about it.

5.6 Skip and only

  • aspec.itSkip(name, ::fn) -- always skipped. Fastest way to park a failing spec without deleting the intent.
  • aspec.itOnly(name, ::fn) -- when any spec is marked itOnly, every spec not so marked is skipped. Useful while iterating on one failing case.

Never commit itOnly to main; it suppresses every other spec in the file.

5.7 Per-spec timeout

aspec.itAsync("slow HTTP", ::specSlow, 5000);   /* 5 second timeout */

On timeout, the runner calls done.fail("timeout after 5000ms (done.ok() / done.fail() never called)") and advances. The spec is reported as FAILED with that message.


6. End-to-End Tests with Playwright

Playwright tests live in playwright/*.aus and are run with the CLI Aussom binary. They launch a real Chromium, load your HTML page, and assert on the DOM or the console.

6.1 The standard driver shape

include playwright.playwright;
include file;
include util;

class Main {
    public passed = 0;
    public failed = 0;

    public main(args) {
        pomContent = file.read("/home/you/project/pom.xml");
        versionTag = regex.matchFirst("<version>\\d+\\.\\d+\\.\\d+</version>", pomContent);
        version = versionTag.replace("<version>", "").replace("</version>", "");
        url = "file:///home/you/project/target/my-project-" + version + "/my-page.html";

        pw = new playwright();
        browser = pw.chromium().launchHeadless(true);
        page = browser.newPage();

        page.addInitScript(
            "window.__logs = [];" +
            "['log','info','warn'].forEach(function(m){" +
            "  var orig = console[m];" +
            "  console[m] = function(){" +
            "    window.__logs.push(Array.prototype.join.call(arguments,' '));" +
            "    orig.apply(console, arguments);" +
            "  };" +
            "});"
        );

        page.navigate(url);
        page.waitForLoadState("load");
        page.waitForTimeout(8000.0);   /* give aspec / runTests time to finish */

        summary = page.evaluate(
            "() => { var s = window.__logs.find(function(l){ " +
            "  return l.indexOf('PASSED:') !== -1 && l.indexOf('FAILED:') !== -1 && l.indexOf('TOTAL:') !== -1;" +
            "}); return s || ''; }"
        );
        c.log("Summary: " + summary);

        this.check("Summary contains 'FAILED: 0'", summary.contains("FAILED: 0"));

        browser.close();
        pw.close();

        c.log("Results: " + this.passed + " passed, " + this.failed + " failed.");
    }

    public check(string label, bool result) {
        if (result) { this.passed += 1; c.log("  PASS: " + label); }
        else        { this.failed += 1; c.log("  FAIL: " + label); }
    }
}

6.2 Capturing ALL relevant console streams

This is a very common foot-gun. Patching only console.log captures @Test-runner output but not aspec output, because aspec routes messages through console.info. Always patch log, info, and warn together:

['log','info','warn'].forEach(function(m){
  var orig = console[m];
  console[m] = function(){
    window.__logs.push(Array.prototype.join.call(arguments,' '));
    orig.apply(console, arguments);
  };
});

If you patch only log, your Playwright driver will see zero logs when testing an aspec page, your driver will wait the full timeout, and the failure message will be the entirely unhelpful "no summary line found". This was discovered the hard way during the aspec bring-up; the fix takes three extra lines.

6.3 Mocking browser APIs

Several TeaVM / browser interactions behave differently than you expect from the Playwright docs. Keep these rules in mind when writing a mock:

  • navigator.geolocation is read-only on file://. Direct assignment (navigator.geolocation = {...}) is silently ignored. Use Object.defineProperty(navigator, 'geolocation', { value: ..., configurable: true, writable: true }) instead.
  • navigator.mediaDevices is non-configurable in headless Chromium. Replacing the whole property throws a TypeError. Instead, patch navigator.mediaDevices.getUserMedia on the existing object. Only fall back to Object.defineProperty when mediaDevices is absent.
  • One large addInitScript with multiple independent mocks fails closed if any one mock throws. An uncaught exception at any point terminates the script and silently skips every mock defined after the throw. Split mocks into separate addInitScript calls, each wrapped in its own try-catch.
  • Never put // line comments inside JS strings built by Aussom string concatenation. Aussom concatenation produces one long JavaScript string with no embedded newlines, so a // comment silently consumes everything after it on the same logical line and you get a SyntaxError. Use /* ... */ block comments or omit inline comments entirely.

6.4 When to reach for Playwright

  • You need to verify end-to-end behavior that crosses page loads, routes, or localStorage state.
  • The bug reproduces only in a real browser (layout, clipboard, permissions, geolocation, media).
  • You need a CI gate that asserts the page loads without JS errors and runs to completion.

Do not reach for Playwright when a tier-1 or tier-2 test would catch the same regression faster. A failing aunit test prints its line in ~50 ms; a Playwright test takes 5-15 seconds.


7. Debugging Techniques

7.1 Log levels

main([...]) accepts a logLevel=LEVEL argument:

  • logLevel=TRC -- trace (noisy; shows every include load, parse, and runner step).
  • logLevel=DBG -- debug.
  • logLevel=INFO -- default for runTests; summary and per-spec lines.
  • logLevel=WARN / logLevel=ERR -- quiet.

Start a flaky test at DBG; drop to INFO once it stabilizes. You can also raise the runtime log level inline from Aussom:

c.log("routine message");     /* INFO-level */
c.dbg("diagnostic detail");   /* only when logLevel=DBG or lower */
c.err("something went wrong");/* always logged unless logLevel=ERR only */

7.2 Inspecting the DOM at failure time

The fastest way to debug a failing DOM assertion is to dump the mount container's HTML right before the assertion:

@Test(name = "row renders the phone number")
public rowRendersPhone() {
    this.h.mount(new CustomerRow({ phone: "555-1111" }));

    c.dbg("Mount HTML: " + this.h.html());    /* dump on demand */
    test.expectElementTextContains(".row", "555-1111");
}

At the aspec level, register an onFail hook that does the dump for every failing spec in the group:

aspec.onFail(::onSpecFailed);

public onSpecFailed(specName, errMsg) {
    c.err("Spec failed: " + specName + " :: " + errMsg);
    c.err("DOM at failure:\n" + this.h.html());
}

7.3 Aussom exceptions do not "return"

Aussom propagates AussomException automatically. Code like this is a bug even though it looks reasonable:

/* BROKEN -- the assignment never happens when fetch throws. */
result = Service.fetch(id);
if (result.isException()) {
    c.err("fetch failed: " + result);
    return;
}

When Service.fetch(id) returns an AussomException, the assignment to result is never performed; the enclosing block aborts. Use try-catch:

try {
    result = Service.fetch(id);
} catch (e) {
    c.err("fetch failed: " + e.getText());
    return;
}

This trips people up most often when testing. expectThrows papers over the issue (see section 3.4), but any test-side code that deliberately calls a throwing API needs explicit try-catch.

7.4 JSBody methods can throw RuntimeExceptions

Any @JSBody method that calls browser APIs (canvas, media, network) can throw a JS exception that TeaVM wraps as a Java RuntimeException. That exception climbs the call stack, past the Aussom interpreter, and becomes an unhandled browser exception -- usually one window.onerror can log but cannot recover from.

Inside any @JSBody script that touches a browser API, wrap in try-catch and return a safe default:

@JSBody(params = { "canvas" },
    script =
        "try { " +
        "  var ctx = canvas.getContext('2d');" +
        "  ctx.drawImage(...);" +
        "  return 0;" +
        "} catch (e) { return -1; }")
public static native int drawSafely(JSObject canvas);

If you are writing pure Aussom and only using extern classes that someone else wrote, you do not need to worry about this -- but if one of your Playwright tests mysteriously logs Script error. with no further detail, this is probably why.

7.5 Promise callbacks need setTimeout

A @JSFunctor callback invoked directly from a Promise .then() handler can conflict with TeaVM's CPS state and silently do nothing. Wrap the callback call in setTimeout(..., 0) inside the @JSBody script:

@JSBody(... script =
    "p.then(function(v){ setTimeout(function(){ onResolve(v); }, 0); })" +
    " .catch(function(e){ setTimeout(function(){ onReject(String(e)); }, 0); });")

Again, this is an extern-author concern. But when you are writing an aspec test that passes through js.promise(...) and the test times out with no console output, the root cause is almost always a Promise handler that forgot the setTimeout deferral.

7.6 Common failure modes and where they come from

Symptom Usual cause
"Suspension point reached from non-threading context" @Async method reached through reflection
Silent abort, no console message @Async at reflection boundary, or Aussom exception propagating past an unsafe assignment
"$obj.$method is not a function" Plain Java helper called from @JSFunctor lambda was not declared final
node.add(): Unexpected extern object type Passed an HNode wrapper where an Element extern was expected -- use hnode.obj
Playwright test sees zero logs Driver patched only console.log, not .info / .warn
Script error. with no message Uncaught JS exception from a @JSBody method
Mock callback never fires Synchronous callback from inside @JSFunctor without setTimeout(0)
Parse error at line N column N inside a comment Multi-asterisk "banner" comment block confused the lexer

8. Test Organization and Strategy

8.1 File layout

A typical project ends up with three test surfaces:

webapp/
  index.html
  my-app.aus
tests-unit/
  MyModule-test.html        -- aunit @Test, pure logic
  AnotherModule-test.html
tests-component/
  CustomerCard-test.html    -- aunit @Test + TestHarness
  ProductRow-test.html
tests-async/
  SearchFlow-spec.html      -- aspec, async behavior
playwright/
  smoke-test.aus            -- CLI driver, loads index.html
  tests-unit-runner.aus     -- CI gate, loads every tests-unit/ HTML
  tests-component-runner.aus
  tests-async-runner.aus

The tests-* folders should not share state. A Playwright runner for a folder should load each HTML file in isolation and assert its own FAILED: 0 summary.

8.2 Writing a failing test first

For tier-1 and tier-3 tiers, it is worth writing the test before the fix. A quick way to write a regression test:

  1. Reproduce the bug manually. Note the input and the observed output.
  2. Write a single @Test or aspec.it that runs the same input and asserts the expected output. Run it; confirm it FAILs for the right reason (expected X but got Y, not a NullPointer).
  3. Write the fix. Run the test; confirm it PASSes.
  4. Leave the test checked in.

8.3 Deciding which tier

Default to the cheapest tier that can catch the bug:

  • "This function returns the wrong total." -> tier 1.
  • "This component renders the wrong class." -> tier 2.
  • "When the user clicks save, the status is wrong 50ms later." -> tier 3.
  • "When the user navigates through three pages, step 4 is wrong." -> tier 4.

Over-investing in Playwright tests slows CI and makes failures harder to diagnose. Under-investing leaves tier-4 to catch bugs that a one-line tier-1 test would have flagged.

8.4 Flaky tests

A flaky test is almost always one of:

  • Time-based -- hard-coded waitForTimeout that is too short for CI. Fix: wait for a specific console log line instead of a fixed duration. aspec specs should use itAsync with a generous timeout, not busy-waits.
  • Event-ordering -- assumes a Timer(0) has already fired by the time the next assertion runs. Fix: drive the assertion from the done callback, not from the spec body.
  • State leak -- a prior test left DOM or instance state in the page. Fix: every test class resets via @Before or aspec.beforeEach; every TestHarness pairs with destroy().

Never "fix" a flaky test with a longer sleep. Find the event the test is actually waiting for and wait for that.


9. Running the Full Suite

For a project using all four tiers:

# Tier 1-3: Playwright drivers run the in-browser suites.
cd playwright
aussom tests-unit-runner.aus
aussom tests-component-runner.aus
aussom tests-async-runner.aus

# Tier 4: any standalone Playwright smoke tests.
aussom smoke-test.aus

Each driver prints Results: N passed, M failed. and exits with non-zero on any failure (when you follow the standard pattern shown in section 6.1). Wire those into CI the same way you would any other shell step.


10. Summary Checklist

Before you call a test suite "done", check:

  • [ ] Every new logic function has a tier-1 unit test.
  • [ ] Every new UI component has at least one tier-2 component test covering its render-from-props path.
  • [ ] Every callback / Timer / HTTP path has a tier-3 aspec test using itAsync + done.
  • [ ] The project has at least one Playwright driver that loads the whole suite and gates on FAILED: 0.
  • [ ] Playwright drivers capture console.log, console.info, and console.warn (not just .log).
  • [ ] No test uses a fixed waitForTimeout longer than strictly necessary; async waits hang off done or a specific log line.
  • [ ] Exception-path tests use expectThrows or explicit try-catch, never if (result.isException()).
  • [ ] Every TestHarness is paired with destroy() in @After.
  • [ ] Every @JSBody method that touches a browser API (canvas, media, network) wraps its script in try-catch.