Kinetic

Abstract

We present Kinetic, the first real-time physical movement intelligence system that perceives, reasons about, and enhances human physical capability across multiple domains: skill coaching, physical therapy rehabilitation, autonomous fall detection, and spatial monitoring. Kinetic introduces a 4-tier expert generation pipeline, a triple-metric scoring engine (Gaussian joint angles + cosine spatial similarity + COCO OKS), voice-first coaching via GPT-4o Realtime, and autonomous monitoring with Telegram alerts. Edge AI on NVIDIA DGX Spark provides sub-50ms latency. 44 MCP tools orchestrated through the Anthropic Claude Agent SDK with 3 sub-agents.

1. Introduction & Motivation

AI has transformed cognitive work. Yet human physical capability remains unaugmented. Movement literacy, rehabilitation, injury prevention, and safety monitoring still depend on expensive human experts or crude apps. 1.7 billion people want to learn physical skills. 55 million elderly Americans need fall monitoring. Millions more need accessible PT rehab.

Kinetic is the first system to unify skill coaching, physical therapy, autonomous monitoring, and hospital safety into a single Physical Movement Intelligence platform — powered by the same CV + AI stack across all four modes.

2. System Architecture

Kinetic's architecture spans 6 infrastructure pillars: (1) NVIDIA DGX Spark for edge AI pose estimation, (2) Modal + NVIDIA A100 for cloud-based motion generation, (3) Anthropic Claude Agent SDK for multi-agent orchestration, (4) OpenAI GPT-4o Realtime API for voice coaching, (5) Google MediaPipe + Ultralytics YOLO for computer vision, and (6) a custom triple-metric scoring engine. Data flows from camera frames through the CV pipeline to the scoring engine, with Claude orchestrating the coaching logic and voice delivering corrections.

3. Triple-Metric Scoring Engine

Traditional pose scoring relies on a single metric (typically joint angle difference), which fails to capture spatial relationships and overall pose shape. We propose a triple-metric approach:

• Gaussian Joint Angles (40%): Each of 16 key joint angles is scored using a Gaussian function with σ=15°, providing smooth, interpretable per-joint scores.
• Cosine Spatial Similarity (30%): Normalized skeleton vectors are compared using cosine similarity, capturing overall pose shape independent of body proportions.
• COCO OKS (30%): Object Keypoint Similarity, the standard metric in academic pose estimation (used in COCO benchmark), provides a weighted evaluation based on joint importance and localization accuracy.

4. AI Expert Generation Pipeline

The key innovation is generating expert references from text alone. Our 4-tier pipeline provides graceful degradation: Tier 1 (canonical templates) serves instant responses for common exercises; Tier 2 (Claude semantic mapping) resolves aliases and variations; Tier 3 (Claude angle generation) handles novel skills through biomechanical reasoning; and Tier 4 (HY-Motion 1.0-Lite on A100) provides state-of-the-art 3D motion diffusion for the most complex cases. Each tier is attempted in order, falling through to the next only when necessary.

5. Autonomous Monitoring

In monitoring mode, Kinetic accepts a goal (fall detection, desk security, posture watch) and runs a continuous perception→reasoning→action loop. Falls are detected via activity classification with temporal smoothing. Claude evaluates alert triggers. Telegram delivers photo alerts to caregivers instantly. The bot is bidirectional: caregivers send /status, /goals, /photo commands.

6. Edge AI & Latency

Real-time coaching demands sub-100ms feedback latency. We achieve this through edge AI inference on the NVIDIA DGX Spark's GB10 Superchip, running YOLOv8n-pose for 17-keypoint estimation at 15+ FPS with <50ms end-to-end latency. This is 4x faster than cloud-based alternatives and enables the tight feedback loop required for movement correction during active exercise.

7. Conclusion

Kinetic demonstrates that the same CV + AI stack that coaches a squat can detect a fall, guide PT rehab, and monitor a hospital room. By unifying four intelligence modes into one platform, Kinetic brings to physical capability what AI has already brought to cognitive work. AI has transformed how we think. Kinetic transforms how we move.