Writing macros

A macro is a data-processing script: it turns a measurement or earlier workbook values into a JSON object of named results. openJII stores Python, JavaScript, and R macros. On the web, the backend runs each language in its own isolated Lambda sandbox. The mobile field flow runs JavaScript locally and Python in Pyodide; it does not currently dispatch R macros.

Open Macros and select New macro to choose a language, write the code, and optionally associate compatible protocols. You can also create or select a macro while adding a Macro cell to a workbook. Put the macro after the values it needs, then run the workbook with representative data.

Know the macro inputs

Question values use an { answer: ... } object. Named protocol, command, and macro entries contain their output object. Only upstream cells are included; the macro cannot read itself or a later cell. The host passes ctx itself to macro code; there is no ctx.byId API.

Treat json as one measurement

Write macros against the direct measurement value. In particular, do not write json.sample[0]: the legacy transport envelope is not part of the macro authoring contract.

The backend single and Databricks batch hosts, and the mobile JavaScript and Python host, call the shared normalizer for every input. The call is unconditional; the normalizer reshapes only a plain top-level object with its own sample property whose value is a JSON object or array:

Intentional root-array behavior change. On generic backend calls, web workbooks, and mobile JavaScript or Python runs, a root array is no longer treated as a compatibility envelope. A macro now receives [] as an empty array and [first, second] as the complete two-item array. A macro that wants one element must select that element itself.

Databricks Gold processing is the deliberate compatibility exception. Its legacy producer wraps a source root array as { sample: data } before calling the backend batch host. The normalizer therefore sees a sample envelope and keeps the existing first-measurement behavior; an empty source array becomes { sample: [] } and fails as an empty envelope. This adapter behavior preserves existing Gold results while the generic backend, web, and mobile contract moves to whole root arrays.

For a multi-entry sample envelope, the host discards later entries and emits an operational additional-items-discarded warning. The warning contains only the operational metadata needed to identify the run, envelope source, and entry counts, never measurement content. Batch execution fails an empty item independently and still runs its valid siblings.

Projection is shallow: the normalizer inspects only that own top-level sample property and never traverses nested values. A selected value that is itself sample-shaped passes unchanged rather than being projected again. Root arrays stay whole when exposed as json, as an entry in ctx, or to a workbook branch condition. Raw protocol and output values remain unchanged in storage and transport.

Start with the triggering measurement in Python

The variable is named json, but in Python it is the measurement dictionary, not the standard-library module:

readings = json["readings"]
mean_reading = sum(readings) / len(readings)

return {
    "mean_reading": mean_reading,
    "reading_count": len(readings),
}

Use keys that are stable in the selected protocol's output. Return a dictionary even for one value so downstream cells and exported tables have named fields.

Read earlier workbook values with ctx

Name the producing cells Baseline and Stress. Their canonical keys are baseline and stress:

baseline = ctx["baseline"]["value"]
stress = ctx["stress"]["value"]
device = ctx.get("$device", {})

return {
    "change_from_baseline": stress - baseline,
    "device_id": device.get("id"),
}

For a question named Treatment group, read ctx["treatment_group"]["answer"]. Names beginning with a number gain a question_ prefix; choose simple, unique cell names rather than depending on that fallback.

In multi-device runs, each macro invocation receives the upstream results for that same device. ctx["$device"] can contain family, id, name, firmwareVersion, batteryPercent, and zero-based index when the host knows them. Keep the device ID in the output when it is useful for auditing, but do not use the connection index as a permanent device identifier.

Run from context only

The web workbook runner can execute a macro with no preceding measurement when at least one upstream ctx value exists. In that case, json is an empty dictionary. An answer-driven Python macro can therefore be as small as:

treatment = ctx["treatment_group"]["answer"]

return {
    "treatment": treatment,
    "is_control": treatment == "control",
}

This ctx-only execution is a web workbook capability. Test the full mobile flow separately before relying on it during field work.

JavaScript equivalent

JavaScript receives the same json and ctx contract:

const baseline = ctx.baseline.value;
const current = json.value;

return {
  change_from_baseline: current - baseline,
  device_id: ctx.$device?.id ?? null,
};

R macros receive json, ctx, and an output environment in the web sandbox; return an R list. The direct-value form is the same:

baseline <- ctx$baseline$value
current <- json$value

return(list(change_from_baseline = current - baseline))

Use the web workbook runner to test R macros because the mobile runner currently selects only JavaScript or Python.

Runtimes and available libraries

Where a macro runs decides which libraries you can rely on. The web sandboxes ship scientific libraries and a shared helper library; the mobile Python runtime is deliberately minimal.

Write mobile-run Python macros against the standard library only. If a macro needs numpy, pandas, scipy, or the helper functions below, keep it on the web workbook runner.

Helper utilities

In every web runtime (Python, JavaScript, R) openJII injects a shared helper library, callable directly by name. It mirrors the PhotosynQ macro helpers so existing analysis code ports over. The main families:

• Arrays: ArrayNth, ArrayRange, ArrayZip, ArrayUnZip.
• Statistics: MathMEAN, MathMEDIAN, MathMIN, MathMAX, MathSUM, MathSTDEV, MathSTDEVS, MathSTDERR, MathVARIANCE, MathROUND, MathLN, MathLOG.
• Regressions: MathLINREG, MathPOLYREG, MathMULTREG, MathEXPINVREG.
• Protocol lookup: GetIndexByLabel, GetLabelLookup, GetProtocolByLabel to pull values out of a measurement by its protocol label.
• Domain: TransformTrace, calcSunAngle.
• Messages: info, warning, danger to attach notes to a result.

readings = json["readings"]

return {
    "mean": MathMEAN(readings),
    "stdev": MathSTDEV(readings),
    "fit": MathLINREG(ArrayRange(1, len(readings) + 1), readings),
}

These helpers are not available in the mobile Pyodide runtime; use plain Python there.

Treat ctx as read-only

The host constructs ctx from shared upstream state, so never assign into it. Enforcement differs by runtime:

• Web Python recursively converts dictionaries to MappingProxyType and lists to tuples.
• Web JavaScript deep-clones and recursively freezes ctx.
• Web R locks the ctx binding; R's copy-on-modify behavior isolates nested changes for each item.
• Mobile JavaScript structured-clones and recursively freezes ctx.
• Mobile Pyodide currently receives a JSON-deserialized Python dictionary. It is not frozen, so the read-only contract depends on macro discipline there.

The triggering json measurement is separate from ctx. Do not mutate either input; create and return a new result object.

Test before publishing

1. Use a saved measurement with the same keys and nesting the protocol will produce.
2. Test missing, empty, and boundary values explicitly.
3. Add the macro after its producers in a workbook and check every canonical ctx name.
4. For multi-device work, confirm each result uses the matching device's measurement and identity.
5. Confirm the returned object contains only JSON-serializable values.
6. Run on both web and mobile if the workbook will be used in both places.

For the measurement recipe that feeds a macro, continue with Writing protocols. For the security and infrastructure rationale, see ADR: Execute macros in isolated language sandboxes.