A while back I fine-tuned a small model to play Skyrim NPCs for Mantella, the mod that gives NPCs an AI voice. It stays in character, knows its lore, and has never heard of a phone. It works nicely. It also needs a GPU to run, and not everyone has a spare 12GB card sitting in a cupboard.

So this is the other half: a way to run it that needs no GPU, no local setup, and no trust in me with your data. One click, it runs in your own AWS account, and it costs about 40p an hour while you play.

The shape of it

There is a one-click CloudFormation stack. You launch it, tell it your own IP address, and a few minutes later you have a private endpoint you paste straight into Mantella. Under the hood it stands up a single GPU instance, pulls the model, runs it behind an OpenAI-compatible server, and hands you back a URL.

The pieces that matter:

• A g4dn.xlarge (an NVIDIA T4). It serves the 7B model at around 30 tokens a second, so replies land in a second or two, which is what you need for a conversation rather than a chat window. It costs roughly 40p an hour to run and a few pounds a month if you leave it parked.
• Locked to your IP, and only your IP. The inference server has no password, so the security group is the entire lock. The stack refuses to accept 0.0.0.0/0 and makes you supply your own address as a /32. Nobody else can reach your endpoint.
• It stops itself. A small idle-watcher checks for traffic and shuts the instance down after a few quiet minutes, so a forgotten session does not quietly bill you overnight. Start it again and it is back in about thirty seconds on the same address.
• The model is pulled from Hugging Face, not from me. The weights are published free, so the stack downloads them from Hugging Face’s bandwidth. I host nothing, and there is no per-launch cost landing on my bill or yours beyond your own AWS.

You pay only AWS. There is no software fee, no account with me, no telemetry. It is a fan tool, and it stays one.

Running it

Launch the stack, and it asks for a handful of things: your IP (find it at checkip.amazonaws.com), which VPC and subnet to use (your default VPC is fine), and how many idle minutes before it stops itself. It builds, and when the endpoint is actually answering, not just when the instance exists, it reports the URL.

Then in Mantella (v0.14): LLM Service OpenAI Compatible, Service URL the one the stack gave you, model any string, secret key any non-empty string, max tokens 250, temperature 0.7. Leave the default Skyrim prompt. That is it. Your housecarl now thinks in the cloud.

Honest caveats: the very first launch takes ten to twenty minutes, because it pulls the official llama.cpp CUDA container and downloads the model once. There is no compilation on the box: it boots an AWS GPU AMI that already has the driver and Docker, then runs the upstream container. Every start after that is about thirty seconds, since the container and model are cached on disk. And a T4 is a modest card, so this is a 7B model at a sensible quant, not a 70B monster. For talking to NPCs, it is plenty.

The security thinking, since that is my actual job

The interesting engineering here is not the model, it is making a thing that a non-expert can launch safely. An inference server with no auth, exposed on the internet, is a bad day waiting to happen. So the design leans on the one control that cannot be misconfigured away: the security group is the lock, the template will not let you open it to the world, and it defaults to your address only. The instance role is minimal. There is no inbound SSH unless you explicitly ask for it. Nothing about it assumes the person launching it knows what a security group is, and it stays safe anyway. That is the part I care about getting right, because “one click” and “secure by default” usually pull against each other.

It is not really about Skyrim

Here is the quiet part. Nothing in that stack knows what Skyrim is. It serves a model URL to an OpenAI-compatible port. Point it at a different model and it is a private endpoint for anything: a coding assistant in your editor, a local chat UI, whatever speaks the OpenAI API. The Skyrim model is just the fun way in. If there is interest, the obvious next step is a version where you pick the model from a list and the Elder Scrolls one is simply the demo.

But that is a later post. For now: a lore-locked Skyrim NPC, running on a cloud GPU you control, for about the price of a coffee per evening, launched by clicking a button. The model is here, the stack is here, and both are free.

Unofficial fan project. Not affiliated with or endorsed by Bethesda or ZeniMax. The model is licensed CC BY-NC 4.0 for non-commercial use; the Elder Scrolls and all related names belong to their owners.