OrchestrANT

OrchestrANT logo: an ant conducting with a baton

OrchestrANT#

👨🏻‍💻 Fast python AI prototyping 🐍

Linux x64 · build + test Linux arm64 · build + test Linux riscv64 · build + test Windows x64 · build + test Windows arm64 · cross build + test Lint gates Submodule pins Benchmarks CodeQL TopLang Donate Twitter YouTube

Docs site

OrchestrANT is the family’s Python AI package and its LLM benchmark lab. The orchestrant package orchestrates AI workloads: camera pipelines, detection with yolo-monitor, streaming, and system and GPU metrics. Beside it live the lab (benchmarks/ and the orchestrant-bench runner), the Flet viewer for its results, and the template a new Python AI project copies its uv/ruff/ty setup, Cython wheel build, Sphinx docs and CI wiring from. The four stay in one repository by owner decision (2026-09-15).

Highlights#

  • Installable packages with their own Python: MSI, MSIX and zip for Windows x64/arm64; .deb, AppImage and tarball for Linux x86_64/arm64.

  • yolo-monitor runs ModelANT’s DEIM-D-FINE-N (Apache-2.0) on CUDA, DirectML or CPU; --self-test needs no camera.

  • orchestrant-bench measures throughput, time-to-first-token, decode rate and correctness of any OpenAI-compatible endpoint.

  • GPU monitoring for two vendors through one GPUProbe: NVML for NVIDIA, ADL or amdgpu sysfs for AMD.

  • The shipped wheels compile every module with Cython; the free-threaded cp314t wheel runs them with the GIL off.

  • Static analysis is a gate: ruff, ty, bandit, vulture and codespell fail CI on any finding.

Quick start#

git clone --recurse-submodules --branch develop https://github.com/Kataglyphis/OrchestrANT.git
cd OrchestrANT
python third_party/ANTfrastructure/linux/scripts/lib/fetch-data-lock.py --lock resources/models/models.lock   # once: the model
uv run --extra app yolo-monitor --self-test    # one inference on a blank frame, a JSON report

The packages, the library install, a camera, the tests and the development setup: docs/source/getting-started.md.

Documentation#

Topic

Read

Getting started: packages, the library, demos, tests, development

docs/source/getting-started.md

The package, yolo-monitor, the model and the extras

docs/source/overview.md

orchestrant-bench, the LLM endpoint runner

docs/source/benchmark.rst

System and GPU monitoring

docs/source/monitoring.md

The LLM benchmark lab: tools, results, the gateway

benchmarks/README.md

The lab’s roadmap

benchmarks/docs/llm-benchmark-roadmap.md

Connecting a model to the NAS

benchmarks/docs/nas-document-ai.md

The model weights and their lock

resources/models/README.md

Rules for agents and contributors

AGENTS.md

Changes

CHANGELOG.md

Contributing and contact#

OrchestrANT is one of the Kataglyphis repositories, which share one way to contribute and one contact. Licence: MIT, see LICENSE.