How We Built an AI Comment Generator That Doesn't Sound Like AI

Most AI-generated writing has a tell. It's the em-dash overdose. The “delve into” opening. The three-sentence rule-of-three closer that wraps everything in a neat bow no human would actually tie. When we built Replya — a Chrome extension that generates LinkedIn comments — our first draft was guilty of every sin in the book. Here's how we fixed it.

The Problem: Every AI Comment Smelled Like ChatGPT

LinkedIn is a graveyard of AI-generated comments. Scroll any viral post and you'll spot them immediately: “Great insights, thanks for sharing!” “This is spot on.” “Couldn't agree more.” They're the commenting equivalent of elevator music — pleasant, frictionless, and instantly forgotten.

The root cause isn't the model. Gemini 2.5 Flash, which powers Replya, is perfectly capable of writing sharp, specific prose. The problem is how you ask it. The default conversational posture of an LLM is helpful, agreeable, and structurally tidy — three traits that are absolute poison for authentic LinkedIn engagement.

Our first version of Replya took a straightforward approach: feed the post text to Gemini, ask for five comments in a specific tone, return the JSON array. The output was technically correct. It was also terrible. Every comment started with “Great post” or some variant. Every comment used em-dashes. Every comment ended with a polite question the poster would never actually answer.

Step 1: Ban the Obvious Tells

We started with what we call the BAN list — a set of hard constraints we pass to the model with every request. These aren't suggestions. They're prohibitions:

The impact was immediate. Removing the crutch of “Great post” forced the model to actually engage with the content. Without em-dashes, sentences became more natural and varied in structure. Without hedging filler, comments gained conviction. The BAN list alone took our output from “obviously AI” to “plausibly human.”

But “plausibly human” isn't the same as “good.” We needed more structure.

Step 2: Give the Model a Real Persona

Most prompt engineering stops at rules. We went further and gave the model an identity. Our system persona isn't “you are a helpful assistant” — it's this:

This persona does three things simultaneously. First, it establishes expertise — this isn't a general-purpose writer, it's someone who understands LinkedIn dynamics. Second, it sets a quality bar — comments should earn replies, not just fill space. Third, it creates tension — the persona is aware that AI-generated comments are reputation poison, so it has a stake in avoiding detection.

That last part is subtle but crucial. By telling the model that “AI comments destroy reputations,” we trigger a kind of self-censorship of the very patterns that make AI output recognizable. The model is now actively trying not to sound like itself.

Step 3: Force Archetype Variation

The single biggest failure mode of AI-generated comments is homogeneity. Ask for five comments and you often get the same idea paraphrased five ways. Our fix was to define five distinct comment archetypes and require the model to embody a different one for each suggestion:

This approach solved the diversity problem. A VALUE-ADD comment shares a concrete technique. A CHALLENGE comment respectfully disagrees with a specific claim. A WITTY comment lands a sharp observation. The key instruction is “embody them, don't label them” — the output should feel like five different people with five different communication styles, not five labeled paragraphs.

Step 4: Speed and Natural Language

One of our most effective constraints is invisible to users: lead with substance. We require that the first eight words of every comment contain the actual point — not a compliment, not a warmup, not an acknowledgment. This forces the model to open with a take rather than a nicety.

We also enforce speed through structural constraints. “One concrete noun beats three abstract ones.” “Start sentences mid-thought where natural.” “No bullet points inside comments.” These rules push the output toward the cadence of someone typing on their phone between meetings, not someone composing a formal response.

Step 5: Handling the Short Post Problem

One edge case nearly broke us: short posts. When someone posts “Happy Monday!” with a photo of their coffee, most of our rules become nonsensical. You can't add tactical depth to a coffee photo. You can't challenge a claim that doesn't exist.

Our solution is a post length check we pass to the model at the start of every prompt:

This is conditional prompt engineering — the model evaluates the input first, then adjusts its output strategy. It prevents our sophisticated rules from producing absurdly overthought comments on casual posts.

The Results: Comments People Actually Reply To

The difference between our first version and the current one is dramatic. Here's a real comparison on the same post:

The “before” comment could be pasted under any post on LinkedIn and nobody would notice. The “after” comment references a specific detail from the post (“retention numbers you cited”), adds a personal data point (“we saw a similar pattern”), and ends with a question the author would actually want to answer. It would take a real human a minute or two to write something comparable.

The Cost of Quality

All of this prompt engineering comes at a literal cost. Our prompts are roughly 1,200-1,800 tokens each, depending on the user's profile data and whether they're in continuation mode. That's 3-4x larger than a naive implementation would require. For a free product with a 100-request daily limit per user, this is a non-trivial expense.

But engagement quality is the entire product. Every comment Replya generates reflects on the user who posts it. A single “Great post!” comment can damage the credibility someone has spent years building on LinkedIn. The token budget is the cheapest reputation insurance we could buy.

What We Learned

Three principles emerged from this process that apply to any product generating text with LLMs:

1. Negative constraints matter more than positive ones. Telling the model what not to do eliminates the most common failure modes. Bans are more reliable than instructions. “Don't start with a compliment” produces better output than “be engaging.”

2. Persona design creates quality incentives. A well-designed persona doesn't just describe a role — it creates a reason to care about output quality. Our persona fears destroying reputations, so it self-polices.

3. Structure begets diversity. Without explicit archetypes, the model converges on a single communication style. Forcing it to inhabit different voices produces genuinely varied, useful output.

The most surprising lesson: prompt engineering isn't about finding clever tricks. It's about designing constraints that leave the model with no good option except writing something that sounds like a real person.