AI Chatbots on Marketing Sites: Useful, or Just Another Popup

We pulled the analytics on a B2B client’s support widget last year, three months after their growth team installed it. Roughly 2% of sessions opened the chat. Of those, just under a third sent a message. Of those, the bot resolved maybe half without a human. The widget was shipping about 290 KB of compressed JavaScript to every single visitor, including the 98% who never touched it, and it had quietly pushed mobile INP past 200ms on the pricing page.

The bot wasn’t bad. The deployment was. That distinction is the whole argument, because an ai chatbot website install is one of those decisions that gets made in a procurement meeting and never gets audited afterwards.

Here’s the honest version, from someone who has shipped these on client sites and also ripped a few out.

What actually happens when you bolt one on

The pitch is always deflection plus capture: fewer support tickets, more qualified leads. What we see in session recordings is narrower and more mundane.

Visitors use chat for three things. They ask a question the page should have answered and didn’t. They try to skip the form because they want a human. Or they poke at it out of curiosity and close it. That third group is noise, and it inflates every dashboard the vendor shows you.

The failure pattern that keeps repeating: the widget gets configured to proactively open after 5 seconds with “Hi! Any questions?” on every page. Engagement goes up. Conversions do not. What you have built is a popup with a friendlier avatar. On mobile it covers the CTA you spent three weeks designing. If your website chatbot is proactively interrupting a first-time visitor who is still reading the hero section, you are measuring interruption, not intent.

The performance cost nobody puts on the invoice

Chat SDKs are heavy. Realistically you are looking at 150 KB to 400 KB compressed across the loader, the main bundle, fonts and an avatar image, plus a persistent websocket and often an analytics beacon. The loader usually injects itself synchronously-ish from a snippet in <head>, and even when it’s async it competes for main-thread time during hydration.

On a Canvas-based marketing site we had down to an LCP of 1.1s on a throttled 4G profile, adding one popular widget took it to 1.6s and introduced a 240ms long task on scroll. None of that is the vendor being incompetent. It’s just physics: you added a second application to the page.

The fix is a facade. Ship your own button in HTML and CSS, and only load the SDK when someone actually intends to chat.

<button id="chat-launcher" class="chat-launcher" aria-haspopup="dialog" aria-expanded="false">
  <span class="visually-hidden">Open chat</span>
  <svg aria-hidden="true" width="24" height="24"><use href="#icon-chat"></use></svg>
</button>

const launcher = document.getElementById('chat-launcher');
let sdkPromise = null;

function loadChatSDK() {
  if (sdkPromise) return sdkPromise;           // never load twice
  sdkPromise = new Promise((resolve, reject) => {
    const s = document.createElement('script');
    s.src = 'https://widget.example.com/loader.js';
    s.async = true;
    s.onload = resolve;
    s.onerror = reject;
    document.head.appendChild(s);
  });
  return sdkPromise;
}

// Warm the connection on hover/focus, but only on devices with a pointer
// and only when the user is not on a metered or slow connection.
const conn = navigator.connection || {};
const canPrefetch = matchMedia('(hover: hover)').matches
  && !conn.saveData
  && !/2g/.test(conn.effectiveType || '');

if (canPrefetch) {
  launcher.addEventListener('pointerenter', loadChatSDK, { once: true });
}

launcher.addEventListener('click', async () => {
  launcher.setAttribute('aria-expanded', 'true');
  launcher.classList.add('is-loading');          // show a spinner, clicks feel slow otherwise
  try {
    await loadChatSDK();
    window.ChatWidget.open();                    // vendor API, name varies
  } catch (e) {
    location.href = '/contact?reason=chat-unavailable'; // always have a fallback
  } finally {
    launcher.classList.remove('is-loading');
  }
});

Two details matter here. The pointerenter prefetch hides the 300ms to 800ms SDK load behind the user’s hand movement, so the panel feels instant on desktop. And the catch branch means a blocked third-party domain (corporate proxies, aggressive blockers, EU consent rejection) degrades to your contact page instead of a dead button. That’s the same thinking we apply to render-blocking resources on static sites: the page should be fully useful before any vendor script arrives.

When an ai chatbot website install actually earns its place

There is a clean test. Does your product generate specific, repeated, answerable questions that your pages don’t answer well?

Cases where we have seen real returns:

• Complex pricing or eligibility. “Does the Pro plan include SSO for 12 seats?” is a one-line answer buried in a comparison table. A bot that answers it correctly removes a real blocker.
• Large catalogues and documentation. If you have 400 docs pages and search is bad, the bot is effectively a better search UI with citations.
• Logistics questions on ecommerce. Shipping to Norway, return window, customs. High volume, low ambiguity, fully in your policy pages.
• Qualification on high-ticket services. Not a form replacement, a form pre-filler. Two questions, then route to the right human with context attached.

Cases where it’s usually the wrong choice: a five-page agency site, a landing page with one conversion action, anything where your real bottleneck is that the copy doesn’t explain the product. A bot does not fix unclear positioning. It just gives people a place to ask what you should have written down. And if your site is a single scroll, the chat bubble is often competing with your own nav for the same corner, which is its own mess (we wrote about the related tension in one-page versus multi-page sites).

Retrieval quality is the whole game

Every vendor demo in 2026 is a model demo. The model is not your problem. Frontier models are more than good enough to paraphrase a shipping policy. Your problem is that the bot is answering from a crawl of your site taken six weeks ago, including a deprecated pricing page that’s still live at an old URL.

What we do before switching a bot on:

1. Curate the source set explicitly. Not “crawl the domain”. A list of URLs, or better, a content export you control. Exclude the blog archive, old release notes and anything with pricing you no longer offer.
2. Add refusal content. Write short canonical answers for the questions you do not want improvised: contract terms, SLAs, discounts, security questionnaires, anything legal. The retrieved chunk should say “contact sales” so the model doesn’t invent a number.
3. Re-index on deploy. Hook the reindex to your publish pipeline, not a weekly cron. A stale index is the single most common source of confidently wrong answers.
4. Log every unanswered question. Weekly, read them. This is the most valuable content brief you will ever get, and it usually shows that four questions make up a third of the volume. Answer them on the page.

If you are building the retrieval layer yourself rather than buying it, the honest answer is that most marketing sites don’t need a dedicated vector store at all. A few hundred chunks fit comfortably in Postgres with pgvector or even in memory. We laid out the decision in vector databases: do you need one.

Escalation is the feature, not the fallback

The moment that decides whether a chatbot helped or hurt is the moment it doesn’t know. Get that wrong and you have built a trap.

Rules we now write into every spec:

• A human handoff option is visible in the first bot message, not hidden behind three failed attempts.
• Two consecutive low-confidence or off-topic turns trigger automatic escalation. Don’t let it loop.
• When no human is available, the bot says so with a timezone and collects an email. It does not say “a specialist will be right with you” at 3am on a Sunday.
• The transcript goes with the handoff. Asking the customer to repeat everything to a human is the fastest way to make the bot a net negative.
• Tone constraints are enforced in the system prompt and verified in review, not assumed. Our full approach is in guardrails for customer-facing AI.

If you cannot staff a human path, do not ship a chatbot. Ship a good form instead. On static sites we route those through WebForms so the submission lands in email and Slack with no backend to maintain, and nobody is left talking to a wall.

Design rules we apply by default

Triggers

No time-based auto-open on first visit. Ever. Acceptable triggers: user clicks the launcher; user has viewed the pricing page twice in a session; user scrolls to the bottom of a docs page without finding an answer; exit intent on a checkout, and even then only once. Intent-based beats time-based by a wide margin on every test we have run.

Dismissal memory

If someone closes the panel, respect it for the session at minimum, and for seven days if they closed it twice. Store it in localStorage, not a cookie you then have to declare.

Mobile

The launcher must not sit on top of a sticky CTA, and the open panel needs to handle the virtual keyboard. Use 100dvh rather than 100vh, or the input disappears under iOS Safari’s chrome. Test with a real device, because the simulator lies about keyboard behaviour.

Accessibility

The panel is a dialog: focus trapped while open, Escape closes, focus returns to the launcher, new messages announced via an aria-live="polite" region. Most vendor widgets get maybe 70% of this right. Audit it the same way you would audit your own form markup, because to a screen reader user it is a form.

Measuring chatbot conversion without fooling yourself

Vendor dashboards optimise for the vendor’s renewal. They lead with conversations and engagement rate. Those go up when you make the thing more annoying.

Track these instead, over a window of at least four weeks:

• Open rate by page template. Tells you where chat is genuinely wanted. Usually pricing and docs, rarely the homepage.
• Containment rate. Conversations closed with no human and no repeat visit within 48 hours. Repeat visits mean you didn’t actually resolve it.
• Escalation rate and time to human.
• Assisted conversion. Sessions that touched chat and converted, compared against a holdout group where the widget is disabled entirely. Without a holdout you are measuring selection bias: people who open chat were already more interested.
• Core Web Vitals, split by widget loaded or not. This is the one nobody runs, and it is the one that most often kills the business case.

The holdout matters more than anything else on that list. Run it at 10% of traffic for a month. On one ecommerce build the holdout converted marginally better on mobile, entirely because of the layout shift and the covered CTA. We moved the launcher, raised the trigger threshold, and the gap closed. That finding was worth more than a year of dashboard screenshots.