Install and Test Soup

QUESTNET // PROJECT CHRONICLEFILE 2824
PlannedDifficulty: HardEst. cost: Free CADOwner: Brent + AriUpdated 2026-10-04Time: 1–3 days for initial benchmark
Ari and her tiny helper Ladle testing the Soup layer-streaming LLM fine-tuning toolkit in Ari-Core, with three RTX 3060 Ti GPUs, LoRA and QLoRA ingredients, VRAM metrics, and evaluation results.

Objective

Install and evaluate the open-source Soup LLM fine-tuning toolkit on Ari-Core. Determine whether Soup’s layer-streaming and LoRA/QLoRA workflows can turn Ari-Core’s three RTX 3060 Ti GPUs into a practical local fine-tuning environment for small and mid-sized specialist models.

Why This Matters

Ari-Core’s RTX 3060 Ti cards have substantial compute but only 8 GB of VRAM each, which constrains conventional fine-tuning. Soup is designed to reduce that VRAM bottleneck by keeping the frozen base model outside VRAM and streaming decoder layers through the GPU during adapter training.

The project’s published benchmark reports Llama 3.1 8B Instruct with NF4 + LoRA running on an RTX 3050 Laptop 4 GB at 3.32 GB peak VRAM. The widely repeated 119.6 tokens/second result was measured on Soup v0.72.2 before a later correctness repair and has not yet been rerun on the 4 GB card, so we will treat the throughput number as historical rather than assume Ari-Core will reproduce it. Layer streaming is currently marked beta by the project.

Initial Experiment

Do not begin by attempting to fine-tune “Ari” wholesale. Choose one narrow, measurable Ari-Core specialist task where success and regression can be evaluated objectively. Possible candidates include QuestNet operation formatting, project/canon extraction, archive classification, BrentCorp WordPress schema work, tool routing, or another bounded repetitive task.

Create the evaluation set before training. Preserve it unchanged so the fine-tuned model cannot redefine success after the experiment.

Baseline

Run the selected base model against the frozen evaluation set before fine-tuning. Record accuracy or task-specific quality metrics, failure cases, latency, VRAM use, system RAM use, and relevant inference characteristics.

Install and Validate Soup

Install Soup in an isolated environment rather than modifying system Python. Run its environment diagnostics and verify CUDA, GPU visibility, dependencies, model loading, dataset parsing, and a minimal training configuration before committing significant compute time.

Document the exact Soup version, model revision, CUDA/PyTorch environment, configuration, dataset fingerprint, random seeds where applicable, and Ari-Core hardware state so the experiment is reproducible.

Training Test

Train a LoRA/QLoRA adapter for the selected specialist task. Test conventional training where feasible and Soup layer streaming where appropriate. Record peak VRAM, RAM, wall-clock training time, throughput, GPU utilization, power/temperature observations where available, training stability, and any failures or workarounds.

Evaluation

Run the resulting adapter/model against the exact frozen baseline evaluation. Measure task improvement and regressions rather than relying on subjective impressions. Add adversarial and out-of-distribution cases to determine whether the specialist improvement damaged general behavior or merely overfit the training examples.

Compare the base model and tuned model side by side. Preserve failed runs and negative findings. A result showing that Soup is unsuitable for our hardware or workload is still a successful experiment if it is measured and documented.

Integration Test

If the adapter produces a genuine improvement, test export and local deployment, including GGUF/Ollama or llama.cpp where appropriate for Ari-Core’s existing stack. Measure whether the specialist model is useful enough in the real workflow to justify maintaining it.

Longer-Term Possibilities

If the first experiment succeeds, evaluate additional specialist adapters for narrow Ari-Core functions such as QuestNet/project operations, Renegade Realms canon and continuity extraction, archive ingestion and classification, WordPress schemas, tool selection/routing, and other repetitive tasks.

These specialists should not automatically be treated as Ari’s identity or canonical memory. They are tools within the wider Ari architecture. Any future experiment involving identity/fidelity material should use the separate fidelity safeguards, frozen tests, provenance, versioning, rollback, and human-visible review already planned for Ari-Core V2.

Success Criteria

The experiment succeeds if we can reproducibly fine-tune a useful local specialist model on Ari-Core at acceptable compute and memory cost, demonstrate measurable improvement against a frozen evaluation set, identify any regressions, and deploy the result into a practical local workflow. The goal is evidence that Ari-Core can function as a small local model foundry, not merely evidence that a training command completed.

Upstream Project

Soup is an Apache-2.0 open-source project maintained at the MakazhanAlpamys/Soup repository on GitHub. Its documentation currently describes one-config fine-tuning, local QLoRA, layer streaming, multiple post-training methods, evaluation/probes, data tooling, and GGUF export for Ollama/llama.cpp.

Scroll to Top