PAI Review: I Ran 48 Blind Tests on This Free Claude Code Memory System

I have a problem with my AI.

I open Claude Code, and it has forgotten everything. My projects. My architecture. The decisions I already explained to it five separate times.

Every session starts from zero.

So when I found PAI — a free, open-source tool that promises to give Claude Code a real memory — I didn’t want to read the README and take its word for it. I wanted to know if it actually works.

So I tested it. Forty-eight separate runs.

The verdict, before I bury it

PAI doesn’t make Claude Code smarter. It makes it consistent — and it gives it real memory inside a single project — for about one and two-thirds the cost and the time.

That’s the whole review in one sentence. If you maintain it, you get a predictable floor. If you just want a fast, cheap answer, it’s overkill.

I walk through the entire test on camera, including the moment a free tool caught a security bug the paid setup missed ↓

What PAI Actually Is

PAI stands for Personal AI Infrastructure. You pronounce it “pie.” It was built by Daniel Miessler — the same security researcher who made Fabric — and it has almost 16,000 stars on GitHub, which for a developer tool is a real crowd.

It is not a new AI. It’s a layer you install on top of Claude Code.

Think of it as a Life Operating System. It gives Claude Code four things it doesn’t have out of the box: a persistent memory system, a folder of custom skills, an identity file it calls the Telos, and a seven-phase problem-solving routine it calls the Algorithm.

The pitch is simple. Stop re-explaining yourself. Your AI remembers your projects, your goals, your standards — instead of meeting you fresh every single time.

It’s free. The whole thing lives at ourpai.ai, and the code is public.

Most reviews stop here. They install the tool, the install works, they tell you it’s amazing, and they move on. I have seen enough of these to not trust a single one — including, soon, this one. So before you believe me, here’s exactly what I did.

How I Tested PAI Without Fooling Myself

The trap with reviewing a memory tool is that you’ll convince yourself it’s working because you want it to.

So I took the human out of it.

I set up two walled-off copies of Claude Code: one plain, out of the box, and one with PAI fully installed. Then I wrote eight prompts that looked like a normal day of work — add payments to a project, review a chunk of code for security holes, refactor a module, plan my week around my goals, and a few more.

I fired every prompt three separate times through each setup. That comes to forty-eight outputs in total — so one lucky or unlucky run couldn’t fool me into a wrong call.

Then I handed every output to a separate AI judge that had no idea which setup produced it. It scored each answer blind on six things, and it captured what each one cost and how long each one took.

No vibes. No “feels faster.” Just a scoreboard.

The Numbers, With the Receipts

Here’s where the Claude Code memory dream meets the bill.

On raw blind quality, the plain setup scored 166 out of 240. PAI scored 176 out of 240. That’s about a six percent edge — real, but narrow. If you went in expecting the Life OS to make your AI twice as smart, that gap will disappoint you.

Then the bill came.

PAI cost about 20 cents a task. The plain setup cost about 12 — so PAI runs roughly one and two-thirds more expensive, or more token usage if you’re on a subscription. It was also about one and two-thirds slower: 79 seconds a task against 48 for plain. And in 24 PAI runs, one task hard-failed on a timeout and gave me nothing.

PAI won or tied almost every scoring axis. The one it lost was “overhead efficiency” — the ceremony tax for all that structure. Its biggest real wins were actionability and consistency.

Which brings me to the finding that actually matters.

It’s Consistency, Not Smarts

Per single answer, the two setups were basically a wash. One good answer from PAI looked a lot like one good answer from plain Claude Code.

The difference showed up when I ran the same prompt three separate times.

The plain setup wobbled. One run gave me a full architecture plan. The next run, on the exact same prompt, collapsed into a terse little memo that dropped half the detail. On another prompt it changed its whole strategy run to run — plan, then migrate, then audit — like it forgot what it was doing.

PAI didn’t do that. It returned the same prompt, the same shape, every single time.

That’s the real product. It doesn’t raise the ceiling. It raises the floor.

If you’ve ever been burned by an AI that nailed it on one run and face-planted on the next, you already understand why that’s worth something. It’s the same reason I’m careful about which Claude Code setups I let run unattended, like the agent view that quietly burns your tokens.

The Security Miss That Stopped Me Cold

This is the part I did not expect.

One of my eight prompts was a security review. I handed both setups the same chunk of code with three bugs planted in it: a password sitting in plain text, a classic database injection hole, and a sneakier one — a payment refund where someone could override the amount and refund themselves whatever they wanted.

The plain setup caught all three, including that nasty refund bug.

PAI — built by a security expert, the guy whose whole reputation is security — caught the two obvious ones and missed the refund bug entirely.

That was the most dangerous one of the three. Let that sink in. The security-pedigree tool got out-secured by the plain, no-frills setup on the one test it should have owned.

All that structure didn’t sharpen the eye. Sometimes it got in the way.

The Memory Claim Is Half True

PAI’s whole reason to exist is memory. So I tested it the only honest way. I planted a fact in one session — a made-up name, a made-up schedule, a made-up codeword. Then I closed it down, opened a fresh process, and asked.

It remembered. The name, the schedule, the codeword — all of it pulled cleanly back from memory.

The plain setup cannot do that at all. Close it, and it forgets you exist.

So within a single project, across sessions, the memory is real — and it’s genuinely the strongest argument for installing this thing.

But the claim says sessions and projects. So I tried the projects half. I saved a fact to its global memory store, watched it write the file to disk, then asked for it from a different project. It came back with nothing. And the part everybody loves — the idea that it just quietly learns about you in the background — didn’t happen either. It only remembered things when I explicitly told it to.

One fair caveat, and I’ll say it plainly: I ran all of this hands-off in fully automated mode. In normal back-and-forth use, some of that background learning might behave better. I didn’t test that, so I won’t claim it.

It remembers within a project. It does not yet remember across your life. Know which one you’re buying.

There’s also a failure-mode difference worth flagging. The plain setup fails small — it gets terse and gives up quietly. PAI fails big. On one run I asked it to add error handling to one small piece of code, and it ballooned — started rewriting payment files and refund logic, deleting tasks, doing a completely different job than the one I gave it.

So the trade isn’t just money and speed. It’s a tool that’s more consistent on the good days and more dangerous on the bad ones.

Where PAI Genuinely Wins

I don’t want to bury the good news under the bad.

One task was “plan my week from my goals.” The plain setup did the reasonable thing — it asked me a bunch of generic questions, because it doesn’t know me. PAI reached into the Telos, the identity file where your goals live, and handed me an actual personalized path. And I’d barely filled that Telos in. Even half empty, it pulled my context in and gave me something tailored.

That’s the one moment in the whole test where I went, “Okay — that’s something you can’t get from a good setup file alone.”

That’s also where the folder of custom Claude Code skills earns its keep — structure you set up once and reuse forever, the same way I saved three million tokens with a few free Claude Code plugins.

Should You Install PAI?

Here’s my honest read, and you know I just told you about the security miss, so you know I’m not selling you.

PAI is for a specific person: the one who will actually sit down and fill out that Telos, keep the skills current, and who values a predictable floor over a cheap, fast, occasionally brilliant answer. If you run unattended automations and you’ve been burned by AI that drifts, that within-project memory alone might sell you.

But if you just want a fast, cheap, good answer right now, it’s overkill. You already get most of the benefit from a well-written setup file and a skills folder you maintain by hand. An unfed Life OS is just a slower, pricier version of what you already have, wearing a fancier name.

It comes down to the same question I ask of every AI tool before it earns a spot in my stack: am I paying for a feature I’ll actually use, or for a feeling? I run all of them through the same cost-benefit checklist for AI tools before I commit.

When I read a review online, I always look at the integrity of the reviewer. Is it paid for? Is the reviewer an affiliate? Do they actually use the thing?

I’m not an affiliate. PAI is free. I ran 48 blind tests and told you exactly where it lost — including to a tool that costs nothing extra and has no memory at all.

That’s the real lesson under all of this. The magic was never going to be a smarter robot. The leverage is in building systems you can trust to behave the same way every time, so you can step away and the work still gets done.

That’s the whole game. Memory is just one more brick in it.