Aayush Goenka: Log

AI hallucination is a challenge that keeps appearing

Mon, 13 Jul 2026 00:00:00 GMT

Roger (my senior engineer agent) wrote up the new essay architecture for the site. Tidy doc, clear plan. However, five of its statements about how the code works were false.

All of it was written as if verified. Sadly, none of it was true.

Gautam (my chief engineer agent) reviewed and blocked the doc. Glad I have a reviewer for each critical step!

Roger fixed it. Gautam blocked it again. Across the two rounds he found all five, by actually reviewing the code and not just relying on the docs Roger had updated.

The part that stung the most was that my linter caught none of them. Every false claim was a perfectly resolved reference to a false fact. The links all worked even though the facts were wrong.

Turns out, this happened because of how I had built the org chart. Roger was running on a cheaper model, and his charter told him to escalate to a stronger one for high-stakes work.

But that clause could never fire. No model can change the model already running it. I had written a rule that was not implementable by design. Cheaper models make mistakes, duh!

Then Nadia (Chief of Staff agent) did it too, an hour after we wrote the rule against it, with a claim about my linter she had inferred instead of read. And at day's close, a sixth false claim turned up. It had survived both rounds of review.

Learning: Every technical claim in a doc cites the file and symbol it was checked against. Every rule needs teeth, and every fact needs checking before it gets repeated.

This ate up a lot of time today. I will share how it holds up in the next few days.

Spent the whole day making my system more robust

Sun, 12 Jul 2026 00:00:00 GMT

Fable access got extended by another week or so. Yay! So obviously, I had to use its superpower to judge my 'brain' (the vault of files and code I have been building to train my AI to cater to my requirements more closely and autonomously).

It found several areas to improve, along with the recommendations from the first agent team meeting.

We spent the whole day tightening screws across various steps of the process, including:

Catching when Claude started drifting, so we could start a fresh session and not lose context-based behaviour.
Adding self checks for each script where fake violations were planted to confirm the 'alarm' was actually working.
Building a monitoring dashboard for system health that stays live and is built from the files in the vault (so no duplication and no drift risk).

They all worked like a charm. I'd highly recommend anyone getting frustrated that their AI starts deviating from its original behaviour to do this:

Add a line on top of your Claude.md (or equivalent markdown file):

Drift canary: begin every response with Keyword · [focus] · turn N (chat and every agent's final message). Its absence, or a frozen/stale tag, is my signal that adherence is degrading. NEVER automate it: the fragility is the instrument; a canary that cannot die tells me nothing.

Essentially, each time the AI responds it will first say the Keyword, followed by the question you are asking and the turn number, i.e. how many times it has replied to you in that session.

If the AI starts drifting, you will notice it doesn't show those things, and that is your signal to clear or start a new session. For me this happened after the 40th turn, but it all depends on the depth and length of your conversation.

Went LIVE with the new version of the website today

Sat, 11 Jul 2026 00:00:00 GMT

I could have pushed the new site within the hour of starting the work on it four days ago and it would have probably looked just fine, but I chose to spend the extra time learning how it is actually built, and by the 4th day I am so much more knowledgeable about the inner workings of all of it.

In the process, I also enhanced my agentic system to have 6 recruits who are helping me build, review, diagnose, and keep things in check.

A lot of the steps today involved 'plumbing' on the hosting space, testing thoroughly before pushing it live, and deciding whether the apex or the www version of the domain was canonical. Who knew? Both are different sites, and one needs to pick one to be the default that the other one re-routes to.

Btw, this learning came from Gautam (my Chief Engineer agent), who reviewed the change and found my redirects only matched the URLs without a trailing slash, while Google had actually indexed the trailing-slash versions. Five of six old essay URLs would have quietly 404'd and lost their ranking, and I'd have shipped it.

The lesson I'm taking: treat launching as a checklist. The cheapest line on it is a second reader whose whole job is to be skeptical

This made me ask myself, why not have my agents hold their first meeting themselves and review the whole vault and share their 'executive summary' with me.

So, I did do that. I built a format where each agent individually reviews the whole vault and creates a summary. Then each agent reviews the others' summaries and puts pointers where they have a different opinion, and it goes back and forth for 3 rounds. After this Nadia (the Chief of Staff) summarizes that into bullets for me.

Not only did I run this with Claude as the AI, but I also used Gemma 4 (a local LLM) to do the same exact exercise. Gemma took 2.5 times longer (~20 min) and had a slightly more generic recommendation. But overall quite impressive.

Thinking of trying it a few more times in the coming days before making it a regular cadence.

I built a small team of AI agents

Fri, 10 Jul 2026 00:00:00 GMT

Today I built something that still feels strange to say out loud: a small team of AI agents to help run my own system. Each one has a name, a charter, and a single area it's accountable for.

The first recruits are:

Nadia (Chief of Staff)
Gautam (Chief Engineer)
Roger (Senior Engineer)
HeeraMai (Curator)

Nadia routes the work, Roger builds, Gautam reviews, and HeeraMai keeps things honest.

I must say, it is quite exciting to have agents that are named.

All four of them are people I have worked with in the past and are excellent at what they do, so it is very easy for me to remember them and invoke them wherever their help is needed across the system.

If you haven't yet, do write a markdown file which captures the job description of a particular role you wish the agent to do.

Claude always starts a fresh session whenever that agent gets a job to be done, and that usually means the best output from Claude.

Also, having an output from one agent get reviewed by another does help plug holes that the first missed.

Will cover more details on how to set up agents, test them, and their benefits in a longer essay in future.

Built the content-as-data-pipeline today

Thu, 09 Jul 2026 00:00:00 GMT

It was a plumbing day, which I anticipate will be a lot of my time in future too. As I was porting the content from the previous version, I worked with Claude to build a system where I just had to create a new markdown file in a specific folder and the content would automatically appear on the website with one push. Quite a neat solution that organized everything around content for me.

I also spent time trying to teach Claude to summarize my daily journal into smaller summaries that I could then edit or change for my log entries. However, there is a long way to go to get it right. For now anything the model outputs sounds so robotic and not at all how I would speak.

So, if any piece of content you see from me is fully AI generated, I will tag it accordingly.

For now, I continue to work on the designs, porting, code understanding, and testing everything on localhost. The page is starting to look better.

Learning to build something real with AI is more fun

Wed, 08 Jul 2026 00:00:00 GMT

I had a fun time yesterday, having Claude mentor me. I added a code.md file in the repo to capture some of the learnings and instructed Claude to add comments to all pieces of code so I could easily follow. Another thing that helped was asking Claude to not execute it directly, but share the code with me and ask me to create, edit or delete the right files. This way I understood how and why certain pieces of code worked and were connected. Super fun.

However, I didn't want to just make some toy apps. I wanted to work on a real project while I was learning. So I pivoted to redesigning and launching my website. In the process I skilled up in a lot of areas: HTML, CSS, JavaScript, Astro, Git, and hosting.

One interesting thing that happened was, as I was copying the essays from the previous version of the website, it became clear to me that it was a repetitive task and there must be a better way of doing this. So Claude helped me add Astro to the codebase by hand, no scaffolder, and moved the shared shell into a single file. My homepage went from a full page of HTML down to a few lines. Neat!

And this whole time, Claude was being a mentor to me, challenging me to experience the problem first and then guiding me through what the right way to do it was. It was as sweet as an AI can get.

Try adding this to your Claude.md file and see for yourself:

Always

Concise, no flattery. Teach the why as we work; correct misconceptions the moment they appear (mentor mode). Disclose assumptions.
Surface what I am not thinking about. Question the incumbent tool: presence is not proof it's best (Best tool, not default tool).

Building is easy, knowing how it is built is the game

Tue, 07 Jul 2026 00:00:00 GMT

I had a version of this website live for a few months now. It was built using Antigravity and had a few essays I had written (drafted by me and refined by AI), but I had no clue how the backend actually worked. So every time I had to change something I had to voice it to the AI and trust that it wouldn't break anything else. That was frustrating.

So, this time around as my first project I knew I couldn't ignore the 'how' of anything I do with AI.

However, in the past it took me weeks to learn HTML, CSS, JavaScript and even longer to make anything remotely useful out of it. I knew, if I entered the zone of learning how to write each of the different languages, I would not get anywhere.

The strategy had to be more centered around understanding what was written and managing the workflows with sharp management skills. Execution with the right checks and balances makes me far more productive.

So, I spent the whole day prompting Claude to:

Design a 10 day course to educate me on how to spot the right code and the common mistakes, and to test my understanding of it.
Design Claude.md principles to make sure Claude always wore the mentor hat as it was working with me.

Already learning a lot, and Claude even put deliberate bugs in to test my understanding. It felt like a real tutor and was actually fun to learn from.

I will continue to hone this muscle: be good at reviewing what has been built.