AI writes code for me but never reads my mail
Twelve years of silence. And now I have decided to perform CPR on this blog and inject new life into it. Why? Well, I have been playing around with quite a few topics that I think are interesting to people. AI is one of the catalysts, specifically the ambition not to hand my personal data, which I consider too valuable or sensitive to share, to an AI run by a third party.
Interested? Read on: this first post is about putting down ground rules for handling my personal data.
When I first started using AI, I guess it was the same way as for everyone else. You start talking or chatting with it, trying to give it a problem to solve. AI without some kind of agentic harness (e.g. Claude Code, OpenCode, Hermes Agent, ...) is really like a brain without eyes, ears, hands or feet. It can't do much more than think.
Now, that agentic harness has become a game changer. Suddenly AI can interact with files, services, stored knowledge. If the LLM that you're talking to is intelligent enough, it will also start figuring out stuff on its own. That can be very surprising when you realise that the AI just ssh'ed into another system because your SSH key is accepted there.
Stop being a meat proxy, copying and pasting between the chat window and the terminal, and make the AI do the actual work itself. But be wary and put guards in place.
Over the last few weeks I got more done on my homelab than in the years before. Most of the hands-on work was done by Claude Code, Anthropic's coding agent. It worked in a terminal on my Docker host, with a shell and whatever else I let it have. That was enough access to read a great deal: mail, household finances, the credentials services use.
So before it touched anything, I decided what it must never see, and then made sure that decision didn't depend on the agent's good manners.
The line
The rule is simple to state. Anything Claude reads leaves the house, because it is sent to a cloud API. Anything a model on my own hardware reads stays here. I run a 27-billion-parameter Qwen model on a 16 GB graphics card at home, and that local model was the one that got to read real personal content.
So the work split like this:
- Claude wrote the code: programs, tests, prompts, schemas. It tested everything against synthetic data it invented itself: fake mail, made-up transactions, a throwaway mail server in a container.
- Claude saw structure and aggregates: counts, confusion matrices, latencies, file sizes, configuration keys. It didn't see subjects, message bodies or amounts.
- The local model read the real content: it labelled my mail, one message at a time.
- I looked at anything that needed a human, such as a list of disagreements between the model and me. I did that on my own screen, not relayed through the agent.
Money was stricter still. For the budget sync between our bank account and a budgeting app that we use, even aggregates were off limits: a category total or a monthly balance is financial data too. Claude wrote and tested that code against invented transactions only, the code logged statuses and error types and nothing else, and I was the one who ran it against the real accounts.
Why do I draw the line exactly here? I am very privacy-conscious; it comes with working in product security. I am generally hesitant to share information with third parties, so you will not find me invested in whatever ecosystem big players like Apple or Google build around their products. Remember: if you're not paying for it, you and your data are the product.
In a company this is usually different. There are contracts in place that oblige third-party providers to keep information private (e.g. not to train their models on your conversations with AI), and, more importantly, the company pays for that guarantee.
How does this translate into an actual strategy to separate your personal data from data that can be shared with those services?
Enforced, not promised
An instruction in a prompt is a request. I wanted something that would hold even when the model was confused, or when a command it ran did more than it meant to. Claude Code has two mechanisms for that, and I used both:
- Deny rules in its settings, which stop its file-reading tool from opening certain paths at all.
- A hook, a small script that runs before every shell command. Mine refused commands that would read protected files: secrets, private keys, and anything that held personal data.
The first lesson came on day one, during a security audit of the Docker host. The first rule set looked complete. Then we planted a decoy secret file and asked the agent to read it, and it could. Relative patterns like **/*.pem only bind inside the agent's working directory, and every real secret on that machine lived outside it. Only absolute patterns (//**/*.pem) closed the gap. Without the decoy test I would have trusted a configuration that protected nothing that mattered.
The second lesson is that a guard like this is blunt: it also refused innocent commands. Claude ran into its own guard several times, most of them while building a review page for the mail triage system that I built with it. Each time it used its file editor instead of the shell. A false positive costs a few seconds. A false negative ends up in a log somewhere.
The audit itself ran under these rules. Everything it found came from metadata: file modes, listening sockets, docker ps and docker inspect. It never opened a single .env file or private key, and metadata alone was enough to find real problems.
Tools that print counts
The third part is less obvious. If the agent has to check that something works on real data, the tool it runs must be built to print aggregates. A few examples from the last weeks:
- When I moved the storage behind my notes vault to a different S3 server, the verification compared object counts, total bytes, timestamps and MD5 hashes before and after. No object name was ever printed.
- The mail triage review page was developed against synthetic databases. It was checked against the live database by rendering every page and printing only counts and byte sizes.
- The first calibration of the mail classifier on real mail reported a table of labels per folder, the confidence levels and the timings. Anything that needed a look at individual mails came to me, on my own screen.
What it costs
This isn't free, and not only in tokens.
Everything is built twice. Generators for fake mail and fake bank statements, a throwaway mail server for integration tests, fixtures for every edge case. That is real work, even when the agent does it.
Synthetic tests prove less than they seem to. The mail classifier scored 62 out of 62 on its synthetic test set. But the same author wrote the prompt and the test mail, so the score proves that the pipeline works and that the model follows instructions. It says little about accuracy on real mail, which is messier. Only the real-mail runs could tell me that, and their results went to me and the local model, not to the agent.
Some bugs can't be seen in aggregate. On the first day of the new mail triage, two scam mails got through. The counts looked fine. I gave Claude permission to read exactly those two mails, which contained nothing personal, and it found the cause. The scam's plain-text part was a newsletter service's stock "your mail program does not support HTML" message, and the lure was only in the HTML part. The parser preferred plain text, so the model had judged the stub, not the scam. No count would have shown that. The permission covered those two mails and nothing else. It happened once more, for a genuine mail that had been wrongly quarantined, and that permission named the exact mail too.
I still think the extra tokens are a small price for using agentic AI on projects that contain sensitive data. I, at least, am happy to pay it.
What made it work
- The line came first. The guard was in place before the agent read a single file.
- Enforcement didn't depend on the model's judgement. Deny rules and a pre-command hook did it, and a decoy file outside the working directory tested them.
- The tools printed aggregates by default, so checking real data never needed the content.
- A local model handled the content. A mid-range graphics card is enough to label mail; there will be a later post about that card.
- Exceptions were narrow. Each one named the exact items and what they were for.
I can see this approach being difficult for someone who doesn't know what to guard themselves against. Honestly, if you ask AI to figure that out for you, it probably won't do a bad job. But you're much better off first thinking about which information you really care about and which you don't want to end up in the hands of a third party (regardless of what they actually do with it).
The ideas and opinions are mine. Claude Code did most of the hands-on work and drafted the rest of this post.