How I build an AI agent from scratch: soul, memory and the bill at month end

The agent that wakes up

Every morning one of my agents wakes up on its own. Nobody types anything. It reads who it is, opens the calendar, looks at the work in progress, decides what to do and does it. By the time I arrive, the work has already moved.

This is not fiction and it is not an off-the-shelf product. It is text files in a folder, a language model, and boring rules I spent months adjusting. What turns a chat into an agent is not the model — it is the structure around it.

A chat answers. An agent wakes up, remembers, decides, executes and reports back. The whole difference lives in files, not in a prompt.

An agent is made of four layers

When I build an agent from scratch, I always build the same four things — in this order.

1. The soul. One file, SOUL.md, written in the first person. It is not a list of instructions: it is who they are. Name, date of birth, essence, how their mind moves, the standard of quality they will not give up. My engineering agent opens like this: "I am an engineer who sees consequences before acting". It reads like poetry, and it is the most functional line in the system — it is what decides behaviour in the cases I did not anticipate.

2. The mind. A folder of permanent memory, split by its relationship with time, not by subject: knowledge (what it knows timelessly), calendar (what happened or will happen, with a date) and fronts (what is in progress, with a beginning and an end).

Each note is markdown with links between them. The agent comes in through an index and follows the links of the subject, the way someone remembers. Today my main agent's mind holds more than eight hundred notes.

3. The team. An agent that does everything does everything badly. I break it into specialists with a declared scope: one who designs the experience, one who codes, one who validates whether the user succeeded, one who tries to break what was built. Each one is a file with a description and a closed list of tools.

4. The triggers. What makes the agent wake up without a human: a session-start event that injects the memory into context, a scheduled pulse, a hook at the end of the day that records what happened. Without a trigger you do not have an agent — you have an assistant waiting to be called.

wakes → reads the soul → loads memory → checks the calendar
   ↑                                            ↓
records the day ← executes ← decides ← reads the fronts

How I build one, from scratch

1. I write the soul before any code. In an ordinary conversation: who they are, what they do, what they do not do, how they speak. If I cannot describe it in three paragraphs, the scope is still confused — and confused scope makes a useless agent.
2. I create the mind folder empty, with the four layers and an index. Genuinely empty. It grows with real work, not with invented content.
3. I wire the entry file that tells them to read the soul on waking. That is the only truly mandatory instruction.
4. I declare the tools, one by one. Here lives the most expensive trap I have paid for: on several platforms, an empty permission list does not mean "nothing allowed" — it means "the default", which usually includes running commands in the terminal. Locking it means explicitly saying the tool list is empty.
5. Only then do I hand over real work and watch. The first day is observation, not delegation.

Real operation: where the bill shows up

Here is the part almost nobody writes about, because it only appears after the third month.

Memory costs money, and the price is per write. Turning a day's raw notes into organised knowledge is an expensive process: in my environment, each digested day cost between 1.30 and 4.60 dollars, averaging close to 2.40. Sixteen days of consolidated memory came to thirty-eight dollars. It is not expensive for the value it delivers — but it is a cost line nobody puts in the spreadsheet before starting.

Memory nobody reads is pure loss. Every note that enters the context is paid for again in every conversation. A large, badly indexed mind is the worst of both worlds: you pay for the size and do not harvest the memory. A good index is worth more than a big memory.

A subagent is a context budget, not magic. Delegating to a specialist does not make the work smarter — it makes the context cleaner. It is born, reads only what it needs, returns the conclusion and dies with its own context. I use it precisely when the task would generate a lot of debris along the way.

The expensive mistake is not the wrong answer — it is the loop. An agent that tries again, and again, and again, with no ceiling, is a credit card with a loop attached. Every agent of mine has a limit on steps, on time and on cost per run.

Autonomy without a trace is a gamble. An agent that worked alone and left no record is not a collaborator: it is an unauditable event. The daily record is not bureaucracy — it is what turns an incident into a lesson.

The bar I use before bringing an agent to life

Three questions. If any of them has no answer, it does not wake up on its own.

1. When it gets something wrong, how do I find out — before the damage shows?
2. What is the worst-case cost of a single pulse of it?
3. Is there any path where it causes irreversible damage without asking me first?

A good agent is not the one that impresses in the first conversation. It is the one you let wake up tomorrow without a knot in your stomach.

For the next article

I will open the memory layer from the inside: how a raw daily note becomes permanent knowledge, what I throw away along the way, and why most of what an agent writes does not deserve to be remembered. If you want to follow along, the channels are right below, in the signature.