🌐 Web: WHATWG & TC39 Extensions for the BEAM

A protocol-agnostic, zero-buffer suite of Web Standard APIs for Elixir.

---

🚀 Beyond Fetch: A Standardized Runtime

Web provides a predictable, spec-pure interface for high-concurrency systems. [cite_start]While most Elixir libraries buffer data into memory by default, Web is built for Zero-Buffer Streaming[cite: 20]. By implementing WHATWG and TC39 standards as Native Process-backed entities (:gen_statem), Web ensures your applications remain responsive even when handling gigabytes of data.

Why Standards on the BEAM?

• Predictability: 100% WPT compliance for core primitives like URL and MIME.
• Flow Control: TC39-aligned concurrency management via Web.Governor.
• Context Propagation: Ambient AsyncContext for metadata that survives process boundaries.
• Structured Data: WHATWG-style structured_clone/2 with transferable ArrayBuffers.
• Capability Messaging: MessageChannel and MessagePort for unforgeable, transferable process-backed handles.
• Cluster Coordination: BroadcastChannel fan-out across BEAM nodes with sender-origin metadata.
• Capability Sockets: WINTERCG-style Socket handles with readable/writable streams and STARTTLS upgrades.
• Resource Disposal: TC39-inspired Symbol.dispose support with a using macro for deterministic cleanup.
• Zero-Buffer Performance: Native streaming with backpressure-aware engines.

---

🛠 The "Web-First" DSL

If you’ve used the modern Web API in a browser, you already know how to use this library. We've mapped those standards to idiomatic Elixir.

defmodule GitHub do
  use Web

  @spec repositories(String.t()) :: Promise.t()
  def repositories(query \\ "elixir") do
    url = URL.new("https://api.github.com/search/repositories")
    
    params = 
      URL.search_params(url)
      |> URLSearchParams.set("q", query)
      |> URLSearchParams.append("sort", "stars")

    url = URL.search(url, URLSearchParams.to_string(params))

    headers = Headers.new(%{
      "Accept" => "application/vnd.github.v3+json"
    })

    request = Request.new(url, 
      method: "GET",
      headers: headers,
      redirect: "follow",
      signal: AbortSignal.timeout(30_000)
    )

    # 3. Fetch and return the Promise of the Response
    fetch(request) |> Promise.then(&Response.json/1)
  end
end

Web.await(GitHub.repositories())

Disposable resources also fit the same runtime-first DSL. use Web now aliases Web.Symbol and imports a using macro that guarantees Symbol.dispose runs in an after block.

use Web

using channel <- BroadcastChannel.new("audit-log") do
  BroadcastChannel.post_message(channel, %{status: "started"})
end

📖 API Usage & Examples

⚡ Concurrency & Async Logic

Web.fetch remains spec-pure. To limit concurrency, apply the TC39 proposal-aligned Governor API to throttle your work explicitly.

use Web

# Limit to 2 concurrent operations globally
governor = CountingGovernor.new(2)

requests =
  for url <- ["https://a.com", "https://b.com", "https://c.com"] do
    Governor.with(governor, fn ->
      fetch(url)
    end)
  end

responses = await(Promise.all(requests))

Async APIs return %Web.Promise{} values. Promise executors capture the current Web.AsyncContext, so logger metadata and signals flow into spawned tasks automatically.

use Web

response = await fetch("https://api.github.com/zen")
text = await Response.text(response)

# Composite multiple async operations
pair = await Promise.all([
  Promise.resolve(:ok),
  Promise.resolve(text)
])

For externally controlled promise settlement, use Promise.with_resolvers/0 to get a promise plus explicit resolve and reject functions.

use Web

{promise, resolve, _reject} = Promise.with_resolvers()

Task.start(fn ->
  resolve.(%{status: :ready})
end)

await(promise)
# => %{status: :ready}

Web.AsyncContext carries scoped values across promise and stream task boundaries.

use Web

request_id = AsyncContext.Variable.new("request_id")

AsyncContext.Variable.run(request_id, "req-42", fn ->
  # Spawning a task or promise here still has access to the request_id
  await(Promise.resolve(AsyncContext.Variable.get(request_id)))
end)
# => "req-42"

🌊 Zero-Buffer Streaming

Managed processes that provide data with spec-compliant backpressure.

# Create a stream from any enumerable
source = ReadableStream.from(["chunk1", "chunk2"])

# Split one stream into two independent branches (Zero-copy)
{branch_a, branch_b} = ReadableStream.tee(source)

# Composable pipelines with pipe_through
upper =
  source
  |> ReadableStream.pipe_through(TransformStream.new(%{
      transform: fn chunk, controller ->
        ReadableStreamDefaultController.enqueue(controller, String.upcase(chunk))
      end
     }))

Standard-compliant gzip/deflate and UTF-8 encoding that works across streamed chunk boundaries.

source = ReadableStream.from(["Hello, ", "🌍"])

encoded =
  source
  |> ReadableStream.pipe_through(TextEncoderStream.new())
  |> ReadableStream.pipe_through(CompressionStream.new("gzip"))
  |> ReadableStream.pipe_through(DecompressionStream.new("gzip"))
  |> ReadableStream.pipe_through(TextDecoderStream.new())

await(Response.text(Response.new(body: encoded)))
# => "Hello, 🌍"

Byte-oriented streams also support WHATWG-style BYOB reads. Consumers can reuse their own Uint8Array buffers while producers fill them through a capability-scoped ReadableStreamBYOBRequest, with ReadableByteStreamController exposing the active BYOB request and byte-oriented desired-size flow control.

use Web

stream =
  ReadableStream.new(%{
    type: "bytes",
    pull: fn controller ->
      case ReadableByteStreamController.byob_request(controller) do
        %ReadableStreamBYOBRequest{} = request ->
          ReadableStreamBYOBRequest.respond(request, "hello")

        nil ->
          :ok
      end
    end
  })

reader = ReadableStream.get_reader(stream, mode: "byob")
view = Uint8Array.new(ArrayBuffer.new(5))

await(ReadableStreamBYOBReader.read(reader, view))
# => %{value: %Web.Uint8Array<...>, done: false}

For a full buffer-reuse example, run:

mix run examples/secure_byob_stream.exs

🔌 Capability Sockets

WINTERCG-style sockets expose a capability-safe handle plus WHATWG ReadableStream and WritableStream endpoints, so raw TCP/TLS sessions use the same backpressure-aware primitives as the rest of the runtime.

use Web

socket = Web.connect("example.com:443", secureTransport: "on")
await(socket.opened)

writer = WritableStream.get_writer(socket.writable)
reader = ReadableStream.get_reader(socket.readable)

await(WritableStreamDefaultWriter.write(writer, "GET / HTTP/1.1\r\nhost: example.com\r\n\r\n"))
chunk = await(ReadableStreamDefaultReader.read(reader))

chunk.value

Explicit STARTTLS upgrades are supported through Socket.start_tls/2. For a full SMTP handshake example, run:

mix run examples/smtp_client.exs

🌍 Data & Metadata

Structured cloning is available directly from Web and preserves supported Web container types, shared references, and transferable ArrayBuffer semantics.

use Web

buffer = ArrayBuffer.new("hello")

clone =
  structured_clone(%{"payload" => buffer}, transfer: [buffer])

ArrayBuffer.data(clone["payload"])
# => "hello"

ArrayBuffer.byte_length(buffer)
# => 0

Unsupported values and non-transferable entries raise Web.DOMException with the standard DataCloneError name.

Web.MessageChannel and Web.MessagePort expose the WHATWG capability messaging model directly on the BEAM. Ports are transferable handles, delivery restores AsyncContext, and the receiving process only gains the capabilities you explicitly transfer.

use Web

{controller, worker} = MessageChannel.new()

spawn(fn ->
  worker =
    MessagePort.onmessage(worker, fn event ->
      received = event.data["port"]
      MessagePort.post_message(received, %{"status" => "ok"})
    end)

  receive do
    :stop -> MessagePort.close(worker)
  end
end)

{client, service} = MessageChannel.new()

MessagePort.post_message(controller, %{"port" => service}, [service])

Web.BroadcastChannel turns those structured-clone guarantees into an idiomatic, distributed coordination primitive. Every broadcast clones its payload before fan-out, preserves sender origin metadata across nodes, restores AsyncContext values while listeners run, and exposes WHATWG-style EventTarget listener APIs on top of a process-backed runtime.

use Web

request_id = AsyncContext.Variable.new("request_id")

sender = BroadcastChannel.new("auth-sync")

receiver =
  BroadcastChannel.new("auth-sync")
  |> BroadcastChannel.onmessage(fn event ->
    {
      event.origin,
      event.data["token"],
      AsyncContext.Variable.get(request_id)
    }
  end)

AsyncContext.Variable.run(request_id, "req-42", fn ->
  BroadcastChannel.post_message(sender, %{"token" => "token-A"})
end)

using receiver do
  BroadcastChannel.close(sender)
end

For a full cross-node example, run:

mix run examples/cluster_auth_sync.exs

For a capability-handover example that keeps secrets isolated behind a transferred port, run:

mix run examples/secure_worker.exs

Strict WHATWG URL parsing with ordered search params, IDNA host handling, and rclone-style URL support.

# WHATWG-style URL parsing
url = URL.new("https://user:pass@example.com:8080/p/a/t/h?query=string#hash")

# URLPattern for matching and ambient route param injection
pattern = URLPattern.new(%{pathname: "/users/:id"})

URLPattern.match_context(pattern, "https://example.com/users/42", fn ->
  # Automatically retrieves captured "id" => "42" from context
  AsyncContext.Variable.get(URLPattern.params())
end)

Standard containers that handle MIME-aware sniffing and live multipart iteration.

# MIME-aware blobs sniff generic binaries
html = Response.new(
  body: "<!doctype html><html>...",
  headers: [{"content-type", "application/octet-stream"}]
)

await(Response.blob(html)).type # => "text/html"

# Live FormData iteration with O(1) memory usage
form = await(Response.form_data(response))
Enum.to_list(form)

---

🧪 Testing & Compliance

Web combines cached JSON fixtures and harvested JS batteries from Web Platform Tests (WPT) with property tests and strict coverage gates.

# Run the full lint + test + coverage gate
mix precommit

# Run the compliance suite directly
mix test --cover

---

Built with ❤️ for the Elixir community.