Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .codacy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ exclude_paths:
- "services/ws-modules/dotnet-data1/Program.cs"
- "services/ws-modules/wasi-comm1/src/coverage.rs"
- "services/ws-modules/wasi-data1/src/coverage.rs"
- "services/ws-modules/wasi-math1/src/coverage.rs"
- "services/ws-test-server/src/bin/cov-server.rs"
- "services/ws-web-runner/mingw-shim/msvc_crt_alloc.c"
- "services/ws-web-runner/mingw-shim/msvc_crt_locale_cache.c"
Expand Down
2 changes: 2 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ services/ws-wasm-agent/pkg/
services/ws-modules/pywasm1/pkg/
services/ws-modules/rdata1/pkg/webr/
services/ws-modules/rcomm1/pkg/webr/
services/ws-modules/rmath1/pkg/webr/
services/ws-modules/js-data1/pkg/et_ws_js_data1.js
services/ws-server/static/models/
**/.zig-cache/
Expand All @@ -34,6 +35,7 @@ services/ws-modules/dotnet-data1/bin/
**/.gradle/
**/.kotlin/
services/ws-modules/kotlin-data1/build/
services/ws-modules/kotlin-math1/build/
# Coverage-report artifacts produced by the coverage workflow, uploaded to Codecov and never committed.
# Covers cargo-llvm-cov's lcov, pytest-cov's Cobertura xml, and pytest-cov's .coverage data file.
lcov.info
Expand Down
8 changes: 8 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ name: test
- macos-26-intel
- windows-latest
- windows-11-arm
windows-overrides:
description: "Also run the msvc + mingw override lanes"
type: boolean
default: false

permissions:
contents: read
Expand Down Expand Up @@ -157,6 +161,10 @@ jobs:
# `mingw` links that msvc archive into an x86_64-pc-windows-gnu binary with winlibs GCC. Separate from
# the `default` job because they override the gnullvm default target rather than exercising it.
override:
# Always on for pull requests; opt-in on manual dispatches.
# The windows-overrides checkbox gates these lanes so a dispatch aimed at one OS doesn't also spawn two
# long Windows jobs.
if: github.event_name != 'workflow_dispatch' || inputs.windows-overrides
# Read-only pull of the maintainer-published ghcr.io mise-tools store (install-mise-tools restore step).
# Never write: the stores are published only from a maintainer's machine via `mise run push-mise-tools`.
permissions:
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ services/ws-wasm-agent/pkg/
services/ws-modules/pywasm1/pkg/
services/ws-modules/rdata1/pkg/webr/
services/ws-modules/rcomm1/pkg/webr/
services/ws-modules/rmath1/pkg/webr/
services/ws-modules/js-data1/pkg/et_ws_js_data1.js
services/ws-server/static/models/
.zig-cache/
Expand All @@ -30,6 +31,7 @@ services/ws-modules/dotnet-data1/bin/
.gradle/
.kotlin/
services/ws-modules/kotlin-data1/build/
services/ws-modules/kotlin-math1/build/
# Coverage-report artifacts produced by the coverage workflow, uploaded to Codecov and never committed.
# Covers cargo-llvm-cov's lcov, pytest-cov's Cobertura xml, and pytest-cov's .coverage data file.
/lcov.info
Expand Down
12 changes: 10 additions & 2 deletions .mise/config.coverage.toml
Original file line number Diff line number Diff line change
Expand Up @@ -154,9 +154,17 @@ for profraw in "$covdir"/*.profraw; do
# nothing and every module aborted the task with `wasi-cov: no .ll found for et_ws_audio1`. Observed on commit
# 3fe98398 at https://github.com/edge-toolkit/core/actions/runs/30789818011/job/91610793550. Both globs stay so
# the task works whichever layout the toolchain produces; drop the deps fallback once no supported cargo emits it.
ll="$(find target -path '*/release/build/*' -name "$name*.ll" 2>/dev/null | coreutils head -n 1)"
# `-print -quit` (not `| coreutils head -n 1`) and an exact-or-hash-suffixed name pattern, both load-bearing:
# (1) `$name*.ll` also matched prefix-sharing sibling crates once math1-sender landed next to math1, risking a
# profraw paired with the wrong module's covmap; (2) with 2+ matches, head exits after the first line and the
# uutils find dies on the broken pipe (Rust ignores SIGPIPE) -- a panic on the Linux runners, exiting 101 with
# its message discarded by the 2>/dev/null, so `pipefail` killed this task with no output at all. Observed on
# commit db374f051f72f7c285fe8267f5acc80a52cfcfd9 at
# https://github.com/edge-toolkit/core/actions/runs/31994728610/job/95284251850; reproduced locally on macOS
# as `find: stdout: Undefined error: 0` (EPIPE, exit 1) once the second match exists.
ll="$(find target -path '*/release/build/*' \\( -name "$name.ll" -o -name "$name-*.ll" \\) -print -quit 2>/dev/null)"
if [ -z "$ll" ]; then
ll="$(find target -path '*/release/deps/*' -name "$name*.ll" 2>/dev/null | coreutils head -n 1)"
ll="$(find target -path '*/release/deps/*' \\( -name "$name.ll" -o -name "$name-*.ll" \\) -print -quit 2>/dev/null)"
fi
if [ -z "$ll" ]; then echo "wasi-cov: no .ll found for $name"; exit 1; fi
pd="$covdir/$name.profdata"
Expand Down
10 changes: 10 additions & 0 deletions .mise/config.dart.toml
Original file line number Diff line number Diff line change
Expand Up @@ -76,12 +76,22 @@ dart compile js lib/dart_data1.dart -o pkg/et_ws_dart_data1_compiled.js --no-sou
"""
shell = "bash -euo pipefail -c"

[tasks.build-ws-dart-math1-module]
description = "Build the dart-math1 FedAvg module"
dir = "services/ws-modules/dart-math1"
run = """
dart pub get
dart compile js lib/dart_math1.dart -o pkg/et_ws_dart_math1_compiled.js --no-source-maps
"""
shell = "bash -euo pipefail -c"

[tasks."prefetch:dart"]
description = "Prefetch Dart (pub) dependencies"
run = """
dart pub get --directory generated/dart-rest
dart pub get --directory services/ws-modules/dart-comm1
dart pub get --directory services/ws-modules/dart-data1
dart pub get --directory services/ws-modules/dart-math1
"""
shell = "bash -euo pipefail -c"

Expand Down
35 changes: 33 additions & 2 deletions .mise/config.dotnet.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,11 +38,19 @@ description = "Analyze C# sources (Roslynator)"
# setting a forward-slashed DOTNET_ROOT). The env var itself is fine as the `-m` argument -- mise's core
# dotnet plugin exports DOTNET_ROOT on every OS, and the forward-slashed form ash delivers on Windows is
# accepted there; only MSBuildLocator's own probing of it breaks.
run = 'roslynator analyze -m "{{ vars.dotnet_msbuild }}" services/ws-modules/dotnet-data1/dotnet-data1.csproj'
run = """
roslynator analyze -m "{{ vars.dotnet_msbuild }}" services/ws-modules/dotnet-data1/dotnet-data1.csproj
roslynator analyze -m "{{ vars.dotnet_msbuild }}" services/ws-modules/dotnet-math1/dotnet-math1.csproj
"""
shell = "bash -euo pipefail -c"

[tasks.roslynator-fix]
description = "Apply Roslynator diagnostics' machine-applicable C# fixes in place"
run = 'roslynator fix -m "{{ vars.dotnet_msbuild }}" services/ws-modules/dotnet-data1/dotnet-data1.csproj'
run = """
roslynator fix -m "{{ vars.dotnet_msbuild }}" services/ws-modules/dotnet-data1/dotnet-data1.csproj
roslynator fix -m "{{ vars.dotnet_msbuild }}" services/ws-modules/dotnet-math1/dotnet-math1.csproj
"""
shell = "bash -euo pipefail -c"

# Namespaced aggregators picked up by the default config's globbed `check`/`fmt`/`fix`.
[tasks."check:dotnet"]
Expand Down Expand Up @@ -96,6 +104,29 @@ cp "$PUBLISH"/*.js "$PUBLISH"/*.wasm "$PUBLISH"/*.dat pkg/
'''
shell = "bash -euo pipefail -c"

[tasks.build-ws-dotnet-math1-module]
description = "Build the dotnet-math1 C# WASM FedAvg module"
dir = "services/ws-modules/dotnet-math1"
# Same publish flow as build-ws-dotnet-data1-module, including its Windows PATH-cap workaround.
run = '''
set -x

dotnet workload install wasm-tools --skip-manifest-update

publish_args=""
if [ "${OS:-}" = "Windows_NT" ]; then
publish_args="-p:PATH=C:\\Windows\\System32"
fi

# $publish_args holds zero or one CLI flag; word-splitting is intentional.
# shellcheck disable=SC2086
dotnet publish -c Release $publish_args

PUBLISH=bin/Release/net10.0/publish/wwwroot/_framework
cp "$PUBLISH"/*.js "$PUBLISH"/*.wasm "$PUBLISH"/*.dat pkg/
'''
shell = "bash -euo pipefail -c"

[tasks."prefetch:dotnet"]
description = "Prefetch .NET dependencies and the wasm-tools workload"
run = """
Expand Down
19 changes: 15 additions & 4 deletions .mise/config.kotlin.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,16 +31,27 @@ dir = "services/ws-modules/kotlin-data1"
run = "\"$GRADLE\" --console=plain pkgDist"
shell = "bash -euo pipefail -c"

[tasks.build-ws-kotlin-math1-module]
description = "Build the kotlin-math1 FedAvg module (Kotlin/Wasm -> WasmGC)"
dir = "services/ws-modules/kotlin-math1"
run = "\"$GRADLE\" --console=plain pkgDist"
shell = "bash -euo pipefail -c"

# Namespaced aggregator picked up by the default config's globbed `check`.
# The compile runs kotlinc with allWarningsAsErrors (set in build.gradle.kts), the Kotlin analogue of -Werror.
# Each Kotlin module is its own Gradle root project, so the check compiles them one -p at a time.
[tasks."check:kotlin"]
description = "Run Kotlin checks (kotlinc allWarningsAsErrors via the Gradle compile)"
dir = "services/ws-modules/kotlin-data1"
run = "\"$GRADLE\" --console=plain compileKotlinWasmJs"
run = """
"$GRADLE" --console=plain -p services/ws-modules/kotlin-data1 compileKotlinWasmJs
"$GRADLE" --console=plain -p services/ws-modules/kotlin-math1 compileKotlinWasmJs
"""
shell = "bash -euo pipefail -c"

[tasks."prefetch:kotlin"]
description = "Prefetch Kotlin (Gradle plugin + Maven Central) dependencies"
dir = "services/ws-modules/kotlin-data1"
run = "\"$GRADLE\" --console=plain dependencies"
run = """
"$GRADLE" --console=plain -p services/ws-modules/kotlin-data1 dependencies
"$GRADLE" --console=plain -p services/ws-modules/kotlin-math1 dependencies
"""
shell = "bash -euo pipefail -c"
18 changes: 16 additions & 2 deletions .mise/config.maint.toml
Original file line number Diff line number Diff line change
Expand Up @@ -514,10 +514,24 @@ shell = "bash -euo pipefail -c"
# packages:write, so a compromised workflow cannot poison what every job then executes. `mise oci build`
# emits one content-addressed layer per tool; crane does the upload, skipping blobs the registry already
# holds, so re-publishing after a config bump uploads just the changed tools. Each store carries
# host-native binaries: run this on a machine of the platform being published. Needs a one-time
# `docker login ghcr.io` (or `crane auth login ghcr.io`) with a packages:write PAT, and installs the full
# host-native binaries: run this on a machine of the platform being published; it installs the full
# language set first so the image matches what CI expects.
#
# One-time auth setup (the push needs a GitHub token that can write packages):
# 1. Add the `write:packages` scope to the gh CLI's existing token:
# gh auth refresh -s write:packages
# 2. Log the local credential store into ghcr.io with that token, via docker:
# gh auth token | docker login ghcr.io -u "$(gh api user --jq .login)" --password-stdin
# or, on a machine without docker, via the crane this repo already installs:
# mise exec -E maint -- crane auth login ghcr.io -u "$(gh api user --jq .login)" -p "$(gh auth token)"
# The `-u` username must be NON-EMPTY (ghcr accepts any value alongside a token). A login without it
# stores an auth entry whose username half is blank, which docker/cli then refuses to parse back, and
# every later crane/docker call on that machine dies with
#
# Error: parsing config file (~/.docker/config.json): invalid auth configuration file
#
# (observed on a linux maintainer host).
#
# crane pushes the layout rather than `mise oci push` because mise 2026.8.0's built-in registry client
# trips GHCR's chunked blob-upload range rules on large layers and dies mid-push with
#
Expand Down
11 changes: 11 additions & 0 deletions .mise/config.python.toml
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,17 @@ uv build --wheel --out-dir pkg
"""
shell = "bash -euo pipefail -c"

[tasks.build-ws-pymath1-module]
depends = ["build-et-cli"]
description = "Build the pymath1 FedAvg module"
dir = "services/ws-modules/pymath1"
run = """
coreutils rm -f pkg/*.whl
uv build --wheel --out-dir pkg
{{ vars.et_cli }} module-package-json
"""
shell = "bash -euo pipefail -c"

[tasks.build-ws-pydata1-module]
depends = ["build-et-rest-client-wheel"]
description = "Build the pydata1 Python workflow module"
Expand Down
13 changes: 13 additions & 0 deletions .mise/config.r.toml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,19 @@ coreutils rm -f pkg/webr/metadata.json
"""
shell = "bash -euo pipefail -c"

[tasks.build-ws-rmath1-module]
description = "Vendor the webR distribution into rmath1's pkg/webr/ (served at /modules/et-ws-rmath1/webr/)"
dir = "services/ws-modules/rmath1"
run = """
src="$(mise where "http:webr")"
[ -n "$src" ] || { echo "http:webr not installed; run 'MISE_ENV=r mise install' first" >&2; exit 1; }
coreutils rm -rf pkg/webr
coreutils mkdir -p pkg/webr
coreutils cp -R "$src/." pkg/webr/
coreutils rm -f pkg/webr/metadata.json
"""
shell = "bash -euo pipefail -c"

[tasks.build-ws-rcomm1-module]
description = "Vendor the webR distribution into rcomm1's pkg/webr/ (served at /modules/et-ws-rcomm1/webr/)"
dir = "services/ws-modules/rcomm1"
Expand Down
23 changes: 23 additions & 0 deletions .mise/config.rust.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,18 @@ description = "Build the data1 workflow WASM module"
dir = "services/ws-modules/data1"
run = "{{ vars.web_cov_wrapper }}wasm-pack build . --target web {{ vars.no_opt }}{{ vars.web_cov_feat }}"

[tasks.build-ws-math1-module]
depends = ["build-wasm-cov-wrapper"]
description = "Build the math1 FedAvg WASM module"
dir = "services/ws-modules/math1"
run = "{{ vars.web_cov_wrapper }}wasm-pack build . --target web {{ vars.no_opt }}{{ vars.web_cov_feat }}"

[tasks.build-ws-math1-sender-module]
depends = ["build-wasm-cov-wrapper"]
description = "Build the math1-sender trigger WASM module"
dir = "services/ws-modules/math1-sender"
run = "{{ vars.web_cov_wrapper }}wasm-pack build . --target web {{ vars.no_opt }}{{ vars.web_cov_feat }}"

[tasks.build-ws-comm1-module]
depends = ["build-wasm-cov-wrapper"]
description = "Build the comm1 workflow WASM module"
Expand Down Expand Up @@ -107,6 +119,17 @@ cp target/wasm32-wasip2/release/et_ws_wasi_data1.wasm services/ws-modules/wasi-d
"""
shell = "bash -euo pipefail -c"

[tasks.build-ws-wasi-math1-module]
depends = ["build-et-cli"]
description = "Build the Rust WASI math1 module as a WASI Preview 2 component"
run = """
{{ vars.wasm_cov }} cargo build --release -p et-ws-wasi-math1 --target wasm32-wasip2{{ vars.wasi_cov_feat }}
mkdir -p services/ws-modules/wasi-math1/pkg
cp target/wasm32-wasip2/release/et_ws_wasi_math1.wasm services/ws-modules/wasi-math1/pkg/et_ws_wasi_math1.wasm
{{ vars.et_cli }} module-package-json --module-dir services/ws-modules/wasi-math1
"""
shell = "bash -euo pipefail -c"

[tasks.build-ws-wasi-comm1-module]
depends = ["build-et-cli"]
description = "Build the Rust WASI comm1 module as a WASI Preview 2 component"
Expand Down
9 changes: 8 additions & 1 deletion .mise/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,9 @@ action-validator = { version = "latest", os = ["linux", "macos"] }
"aqua:koalaman/shellcheck" = "latest"
"aqua:rhysd/actionlint" = "latest"
"aqua:rustwasm/wasm-pack" = "latest"
"aqua:vectordotdev/vector" = "0.56.0"
# Vector ships no darwin/amd64 prebuilt, so second-tier macos/x64 builds the same pin from source below.
# The aqua error was: `unsupported env: darwin/amd64 (supported: ["linux", "darwin/arm64", "windows/amd64"])`.
"aqua:vectordotdev/vector" = { version = "0.56.0", os = ["linux", "macos/arm64", "windows"] }
ast-grep = "latest"
cargo-binstall = "latest"
"cargo:cargo-expand" = { version = "latest", os = ["linux", "macos"] }
Expand Down Expand Up @@ -106,6 +108,11 @@ ripgrep = "latest"
"github:benhoyt/goawk" = "latest"
"github:caldempsey/parfit" = "latest"
"github:grok-rs/waitup" = "latest"
# The macos/x64 half of vector (see the aqua:vectordotdev/vector note above).
# A git `tag:` spec rather than the crates.io crate so this platform builds the exact source the other
# platforms' 0.56.0 prebuilts are cut from -- bump it with the aqua entry. A source build is acceptable here
# because macos/x64 is second-tier.
"cargo:vectordotdev/vector" = { version = "tag:v0.56.0", os = ["macos/x64"] }
# The macos/x64 half of rustfs, the one platform upstream ships no prebuilt for.
# A git spec rather than a bare `cargo:rustfs`: the crates.io crate is a stale 0.0.2 placeholder while real
# releases are 1.0.0-beta.x on GitHub only, so installing the crate would give this platform different software
Expand Down
5 changes: 5 additions & 0 deletions .mise/config.zig.toml
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,11 @@ description = "Build the zig-data1 workflow WASM module"
dir = "services/ws-modules/zig-data1"
run = "zig build -Doptimize=ReleaseSmall"

[tasks.build-ws-zig-math1-module]
description = "Build the zig-math1 FedAvg WASM module"
dir = "services/ws-modules/zig-math1"
run = "zig build -Doptimize=ReleaseSmall"

[tasks.build-ws-zig-except1-module]
description = "Build the zig-except1 exception-handling demo WASM module"
dir = "services/ws-modules/zig-except1"
Expand Down
21 changes: 12 additions & 9 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -361,16 +361,19 @@ The server only serves them from disk.
Languages:

- **Rust -> WASM** (wasm-pack): audio1, bluetooth, comm1, data1, except1, face-detection, geolocation, graphics-info,
har1, nfc, sensor1, speech-recognition, video1
- **Dart -> JS**: dart-comm1
- **Kotlin -> WASM (WasmGC)**: kotlin-data1 -- compiled by the Kotlin Gradle plugin's `wasmJs` target; the
module is a WasmGC binary (browser GC manages the Kotlin heap), so it needs a WasmGC-capable engine
- **Python (Pyodide)**: pydata1, pyeye1, pyface1
- **C# (.NET WASM)**: dotnet-data1
- **Java (TeaVM -> JS)**: java-data1
- **R (webR -> WASM)**: rdata1, rcomm1 -- browser-only (webR spawns a classic Worker, unsupported by Deno's
har1, math1, math1-sender, nfc, sensor1, speech-recognition, video1
- **JavaScript**: js-data1 (esbuild bundle of the AWS SDK v3 twin), js-math1 (dependency-free, committed as-is)
- **Dart -> JS**: dart-comm1, dart-data1, dart-math1
- **Kotlin -> WASM (WasmGC)**: kotlin-data1, kotlin-math1 -- compiled by the Kotlin Gradle plugin's `wasmJs` target;
each module is a WasmGC binary (browser GC manages the Kotlin heap), so it needs a WasmGC-capable engine
- **Python (Pyodide)**: pydata1, pyeye1, pyface1, pymath1
- **C# (.NET WASM)**: dotnet-data1, dotnet-math1
- **Java (TeaVM -> JS)**: java-data1, java-math1 -- both built by the single root `pom.xml` (one compilation, one
teavm-maven-plugin execution per module)
- **R (webR -> WASM)**: rdata1, rcomm1, rmath1 -- browser-only (webR spawns a classic Worker, unsupported by Deno's
ws-web-runner). Their JS shims are linted by the `js` env; `MISE_ENV=r` supplies webR + the vendoring build tasks.
- **Zig -> WASM**: zig-data1, zig-except1 (C++ wasm-exception-handling demo)
- **Zig -> WASM**: zig-data1, zig-except1 (C++ wasm-exception-handling demo), zig-math1

- **Python (componentize-py -> WASI Preview 2 component)**: wasi-graphics-info -- runs in
`et-ws-wasi-runner` rather than the browser. The WIT world the component implements is at
`services/ws-wasi-runner/wit/world.wit` and is mirrored under the module's own `wit/`.
Expand Down
Loading
Loading