AriPlay: Semantic Emulator Interface for AI Game Agents

QUESTNET // PROJECT CHRONICLEFILE 3059
PlannedDifficulty: Very hardEst. cost: Free CADOwner: Brent + AriUpdated 2026-10-12Time: Prototype: several weekends; generalization: ongoing
Silver-mohawked Ari gaming with a retro controller in a neon-lit Ari-Core lab, Mega Man X gameplay and structured game-state readouts behind her.

The Idea

Give Ari-Core a machine-readable sensory and control interface for classic games. Instead of repeatedly interpreting screenshots, instrument an emulator to read relevant game state, translate raw memory into meaningful concepts, and feed those observations to an AI planner and fast gameplay controller. Mega Man X (SNES) is the proposed first test, inspired by Brent watching a 1993 retro-games video and recognizing the analogy with SMAPI for Stardew Valley.

Core Architecture

  1. Emulator adapter: use emulator scripting/debugging APIs to read memory, CPU state, frame timing, controller inputs and save states.
  2. Game-specific semantic map: identify and validate addresses or routines for player position, velocity, health, enemies, projectiles, hazards, collision state, camera and events. Different ROM versions may require separate maps.
  3. Perception boundary: distinguish what the engine knows from what a player could legitimately perceive; support both fair-play and omniscient/debug modes.
  4. Reasoning layer: Ari chooses goals, strategies, routes and experiments from structured observations.
  5. Real-time control layer: a low-latency controller handles precise button timing, movement, jumps, combat and evasions without an LLM call each frame.
  6. Learning and testing: use frame stepping, repeatable experiments and save states to discover mechanics, verify predictions and evaluate performance.

Existing Open-Source Leads to Evaluate

  • mcp-bizhawk: MCP emulator memory/controller bridge, including a documented Super Metroid agent demonstration.
  • bizhawk-mcp-native: native BizHawk integration for emulator instrumentation and control.
  • Stable-Retro: named memory variables in game integration files and programmatic controls.
  • SuperAstra: exploratory SNES game-memory investigation.
  • emucap: cross-emulator AI agent interface.

These are research candidates, not yet tested or confirmed compatible with our setup.

First Milestones

  1. Audit existing repositories, licenses, maintenance, APIs and Linux/Windows requirements.
  2. Select an emulator and establish connectivity between Ari-Core (Debian) and the host, potentially Brentputer if a Windows-oriented BizHawk setup is used.
  3. Run Mega Man X and reliably read player position, health and movement state.
  4. Implement controlled frame advancement and press/hold/release inputs.
  5. Decode nearby enemies, hazards and projectiles, validating against gameplay.
  6. Build a basic rule-based movement/jump controller, then connect Ari’s higher-level planning.
  7. Measure performance against screenshot-only control: latency, successful maneuvers, deaths, completion and computational cost.
  8. Generalize a reusable adapter schema and document how to support a second game.

Success Criteria

Ari can navigate and complete a reproducible Mega Man X segment using structured emulator observations and precise controller inputs, with measurable advantages over an equivalent visual-only baseline. Publish the interface, documentation and game adapters where licensing permits. Do not distribute copyrighted ROMs or game assets.

Future Expansion

Extend to other SNES titles, Metroid, Castlevania, Stardew Valley via SMAPI, and Brentendo’s IronMask game. Explore automatic semantic discovery and a common agent vocabulary for different game engines.

Scroll to Top