For now, it's only simple commands. The LLM has already proven it can do several tool calls in a single query. I remember now that those minecraft agents did create scripts themselves? It's definitely something I can explore, too.
No special training or tokenization at the moment. It's just a raw LLM. I think a finetuned LLM would improve quality, massively.
Yes, only certain characters. I'd hate for the "dragon slayer quest" giver to figure out they could kill the dragon themselves and go get killed. The agent has tools to interact with "fixed dialogue" npcs, though, like shops and quest givers. I've yet to see them do it efficiently, though.
I'm building a mmo-ish RPG with LLM agents that can (hopefully) complete player tasks, as an experiment. I've started documenting my progress here: https://huggingface.co/blog/neph1/rpg-llm-agents