Hoid replaces compiler heuristics with an optimizing agent — proven so far on one B200 and one H200, not yet on "any hardware"
Asked:
“Can you tell me everything about runhoid.com which is building an AI compiler?”
Hoid (runhoid.com) is a three-person San Francisco startup founded in 2026, backed by a16z speedrun cohort 007 for an undisclosed amount. This deep dive, compiled as of 2026-10-20, rests on 32 public URLs across 12 independent hosts — Hoid's own site and GitHub for product and benchmarks, a16z speedrun for team facts, and startup databases as corroboration. All benchmark figures are single-sourced, vendor-published; no independent reproduction was found.
The agentic compiler loop
Hoid's described pipeline: a profiling-and-rewrite loop run by an agent, iterating unattended for hours; survivors export as plain PyTorch custom ops. Hover a stage for detail.
Published B200 speedups span 1.03× to 1.59×
Speedup of Hoid-generated kernels over the per-metric strongest stock PyTorch / torch.compile baseline. One NVIDIA B200, CUDA 13.0, torch 2.13, bf16. Vendor-published; scripts and pinned environments public under MIT.
H200 megakernel: lead narrows from 17.4% to 4.1% as batch grows
Qwen3-4B decode, 8K input / 1K output, one H200 (141 GB, 132 SMs). Tokens/s per user by batch size; repo refuses other GPUs. Vendor-published, pinned comparison environments, Apache-2.0.
What the evidence says
a16z speedrun lists three founders — Momčilo Mrkaić, CEO (ex-Databricks, Microsoft ML); Pavle Padjin, CTO (ex-Tenstorrent AI compiler, Cambridge, Microsoft); Vladimir Zeljković (ex-Tenstorrent, JOANNEUM RESEARCH) — three employees, founded 2026, San Francisco. speedrun.a16z.com
Winning CUDA, Triton and CuTe kernels export as ordinary torch custom ops into otherwise unmodified PyTorch; Hoid says B200 measurements need no Hoid runtime at measurement time. runhoid.com
Reproducibility is unusually good for a startup this young: MIT-licensed repo with pinned uv environments, baseline and Hoid scripts, and correctness gates — 128/128 greedy-token parity, transcript identity, SNR/PSNR, retrieval-ranking checks. github.com
The H200 megakernel runs an on-GPU interpreter / device-side scheduler and integrates as a vLLM plugin — but is hard-coded for exactly one H200 configuration. github.com
Access is pilot-led: book a call, pick a model and target hardware, agree workload-specific performance targets; custom and private models accepted. No pricing, self-serve product, named customers or revenue is public. runhoid.com
Architecture, interpretation and risks
Architecture (Hoid's description)
A DAG graph IR serializes the model as an agent-readable document.
Rewrites have declared boundaries and trigger correctness plus whole-graph speed tests.
Nodes map to vendor libraries, generated kernels, or shape/card-specific hand-written kernels.
The agent reads profiles, forms hypotheses, writes diffs and rewrites, and iterates unattended for hours.
Deterministic numerical and speed verification decides keep vs discard; survivors ship as torch custom ops.
Analytical risks (our inference)
Generalizing beyond NVIDIA and beyond exact cards and shapes is unproven; treat “any model on any hardware” as aspiration.
Hoid itself names scheduling overhead and inefficient block kernels as failure modes, and says consistently generating performant megakernels remains a goal.
Optimization wall-clock time, dynamic / control-flow workloads, and transfer from batch-1 or controlled benchmarks to production serving are untested publicly.
Competition: PyTorch/Inductor, Triton, CUDA libraries, TensorRT-LLM, vLLM/SGLang, and other search/agent kernel optimizers.
Not disclosed / unknown
Legal entity
Financing amount, valuation, cap table
Customer names and revenue
Pricing, SLA and support terms
Security and compliance posture
Full hardware roadmap beyond B200/H200
Compiler access / API and training-workload support
Any independent benchmark
Diligence questions
How long does the agent loop take per model, and at what compute cost?
Can one optimization transfer across shapes, batch sizes, and GPU SKUs?
What fraction of attempted rewrites survive the verification gates?
Who runs pilots today, and against which production baselines?
What is the path from pilot services to a self-serve compiler?
Conclusion: technically credible early evidence and unusually good reproducibility for a startup this young — but still a narrow early proof (one B200 benchmark suite, one hard-coded H200 megakernel) rather than validation of the universal promise.
Source appendix — all 32 URLs, 12 hosts
#
Host
Title
Link
Method: company and technical review of Hoid (runhoid.com) as of 2026-10-20, built on a source audit of 32 URLs across 12 independent hosts (largest host: github.com, 9 of 32). Speedups are Hoid-published ratios vs the strongest stock PyTorch/torch.compile baseline per metric on one B200; H200 figures are tokens/s per user vs pinned vLLM, SGLang and TensorRT-LLM. All benchmarks are single-sourced / vendor-published; no independent reproduction exists. Raw latency/throughput values behind each ratio appear in the linked repos and are cut here for space.