by Chris Hunter

Share

A rubber stamp marked with a red X beside a stack of social post proofs, one crossed out, on a desk at night, with the title "Build log, Oct 13–20: an AI caption that claimed work nobody did"

TL;DR

  • An AI caption said a client’s crew installed a pressure tank. They don’t install tanks. Captions can now only claim the work the job folder names, and the check fails closed at publishing.
  • My assistant told me “triaged” when it hadn’t. It now reports honest counts and never says done when it isn’t.
  • The marketing engine shipped 13 releases in 7 days, and most of them taught a tool to refuse the wrong thing.

This one covers October 13 through 20. Thirteen releases of my marketing engine, a new weekly job for photos, and a lot of work teaching tools to refuse the wrong thing. It starts with a caption that said a client did work they don’t do.

AI Marketing Department - An AI caption claimed work nobody did

One of our home-services clients gets social posts written by AI from the job photos their crew uploads. One of those posts went out on all four platforms saying the crew installed a pressure tank.

They don’t install tanks.

The homeowner’s old tank was sitting in the photo. The step that describes the photos saw the equipment and treated it as the client’s work. We caught it after it posted.

Here’s what changed. Photo descriptions now say what’s visible, never what was installed. A caption can only claim the service the job folder names, and each client can list work they never do, which gets blocked no matter what the folder says. The check runs at the publishing step itself and fails closed. If a caption breaks the rule, it can’t go out, even if an earlier step skipped the check.

It shipped as its own release the same day the problem reached me, with the engine’s full suite passing 3,282 of 3,282 tests.

The gap: it only knows the equipment each client’s settings list. Older photo sets with vague folder names get plain, generic captions until the crew names the equipment in the folder title.

If you’re automating content for a business, the AI can’t be allowed to put words in the owner’s mouth about services they don’t sell. Check the claim at the last step before it publishes, not the first.

Hope (AI executive assistant) - My assistant said “triaged” when it hadn’t

I’d ask my assistant agent to triage my inbox, then ask again. Some days three or four times. When I finally had it diagnosed, there were four separate problems stacked on top of each other. It only looked at a small batch of unread mail, once. It ignored my setting to include mail I’d already read. One bad message ID made it reject the whole batch. And after the call failed, it told me “triaged” anyway.

That last one is the dangerous one.

On-demand triage now runs in the background on the same engine as the scheduled triage. It skips a bad item instead of failing the batch, and it reports honest counts: moved, skipped, failed, left. It never says done when it isn’t. It shipped Sunday after review, with 2,446 tests passing.

An agent that reports success it didn’t earn is worse than one that fails loudly. You stop checking, and that’s when it costs you.

AI Marketing Department - Lead verification got brakes

Every week an AI reads each account’s new contacts, call recordings included, and proposes which are real leads and which are spam, vendors or noise. I review the proposals, and the approved tags get written back to the CRM. This week’s review was approved and applied for every account on the weekly cadence.

New rules this week:

  • A tag a person put on by hand wins. If someone on the team marked a contact “not a lead,” the AI can’t call it a lead.
  • If the transcript pass didn’t finish, the review and the apply both refuse to run. One account’s review had to be re-run after a fix landed, which is the exact case this rule covers.
  • A missed call goes on a call-back list for a person instead of into the CRM as a tag. A missed call doesn’t prove anything.
  • Every verdict records which model made it, so a bad call can be traced later.
  • When a recording can’t be transcribed, the review quotes the actual error instead of guessing.

After each apply, we re-read sample contacts straight from the CRM to confirm the tags landed. 37 of 37 checks came back clean.

Your lead count is only as good as the tags behind it. Count spam and vendor calls as leads and every report you read lies a little.

AI Marketing Department - Job photos now move on a schedule

Job photos are the best content a contractor has, and they’re worthless sitting in a folder nobody opens. Crews upload them to a shared drive. Until this week, copying them into each client’s content workspace was a manual job, and it had last run in early September.

Now a weekly job copies every new job’s photo set over, where the caption pipeline above picks it up. It went from 1 account to 6 in a day: 62 job sets and 585 photos on the first runs, last 90 days only. The first scheduled run went Monday, October 26.

It broke on day one. Two photos in the same set with the same file name overwrote each other, and one account lost 8 photos that way. We re-copied them, fixed the bug with a test behind it, and rechecked another account’s 53 sets. Nothing else was lost.

The limit: it needs one folder per job. Some accounts drop loose files or sort by region, and those need a folder change before they can be wired in.

The first run on live data is the real test. Have a recheck ready for it, because a test folder of five clean photos will never show you a name collision.

AI Marketing Department - A weekly check on whether Google is indexing your pages

“Is Google even showing my pages?” usually gets asked once, during an audit, and never again.

Now, once a week per site, a tool checks every page in the sitemap against Google Search Console and writes one review card. It lists the pages Google isn’t indexing, grouped by likely cause and fix, what got worse since last week, and how things compare to a dated baseline.

It’s read-only. The one thing that writes anything, resubmitting a sitemap, only runs when a person starts it. If it can’t reach a site’s data, the card says “not collected.” It never reports a blank as “all indexed.” It ships switched off for each client and gets turned on once access is granted and a clean baseline exists. 3,111 tests when it shipped.

AI Marketing Department - Thirteen releases, and a bug sweep that came back fixed

The marketing engine plans, writes, checks and publishes content and client reports. It shipped 13 releases from October 13 to 19. Besides what’s above:

  • A dry run that’s actually dry. The sweep found that one tool’s “dry run” still pushed an image to a live site. Now it makes zero writes, and a test proves it.
  • No duplicate posts. If a publish result is uncertain, it adopts the post that’s already there or stops. It never creates a second copy blind.
  • Ad-change undo still works when the config file is broken, but only against the one ad account it already wrote to.
  • A publishing board. Every site the engine posts to writes a daily queue snapshot, so I can see in one table what’s scheduled and what’s running dry.
  • Podcast thumbnails put the episode badge in a corner with no face in it.

That sweep was a fleet of AI agents going through the engine on audit day. It found 1 high and 5 medium bugs, and all 6 were fixed and released the same day, for about 10 million tokens.

A dry run that writes to a live site is worse than having no dry run at all, because you trust it.

Friday (my AI chief of staff) - Friday got an agent desk

Friday is my chief-of-staff agent, and it changed in three ways this week.

An agent desk. I tag a card on my task board with an agent’s name, and the desk routes it to that agent’s session, waits for the acknowledgement and tracks the result. It has its own budget: 20 million tokens a week, at most 3 builds a day and 2 per agent. For now it runs dry. It reads live cards and logs what it would do, but writes nothing. It was built and merged in one day after two review-and-fix rounds, with 2,104 tests passing.

Pacing on actual usage. Friday checks my live AI plan usage every hour and decides whether to start new build work. At 85% of the weekly limit, it stops starting. Right now that’s a rule Friday follows each hour, not a coded gate.

Approvals in one place. Every project session sends its “I need Chris’s go” to Friday. Friday brings them to me in one batch, and my answer goes back word for word. I stopped hopping between sessions to say yes.

Running agents in parallel only works if you aren’t the switchboard. I was the bottleneck, and nobody could see it.

AI Builders Labs - Up next: a 10-question AI scorecard for agency owners

I looked at how 10 AI agencies and platforms bring in clients. I didn’t find any selling a fractional Head of AI to agency owners, and my own site had nothing a visitor could try before booking a call.

So I’m building a free self-assessment for the AI Builders Labs site. An agency owner answers 10 questions, gets a score in one of three bands, and sees the first fix to make. A breakdown email follows if they want it.

It’s built, with 75 tests passing. It went live October 20. More on it next week.

Need help building something like this?

Everything I write about here runs in my own business first. If the AI job landed on your plate, start with an assessment: where AI will make or save you the most money, and what it'd take to run it right.

About the Author: Chris Hunter

I'm Chris Hunter, a fractional Chief AI Officer. I've spent 23 years in marketing and co-founded Marketing Heroes, an agency for roofers and home-service companies. I've spent the last four years building AI into the work, and today an AI system I built runs a big piece of it: an AI sales rep, an executive assistant, and a marketing department that ships content and runs ads every week. One operator and those agents now cover two of our six production seats, and the agents cost about $460 a month to run. Now I do the same for other companies, so the founder doesn't have to be the head of AI on top of running everything else. Before any of this I was an Air Force crew chief, working on the first stealth fighter under zero-defect standards. That's still how I build. If it breaks in front of a customer, it wasn't done.

Share

Related Posts