AI-Crawler Tracker

Many AI and search crawlers request server-rendered pages without running browser analytics. Tailwin detects supported crawlers at the edge or origin and sends successfully delivered visits to your dashboard.

What it does, and what it doesn't

• Passes the site request through and keeps delivery outside the response path.
• Reads the user agent and path without the query string. Status and country are included only when the integration provides them.
• No cookies, no request bodies, no visitor IP addresses, nothing about human visitors.
• Every version is short enough to read before you trust it.
• Bot list syncs from our registry hourly; new AI agents appear without a redeploy.
• Delivery is best effort. A failed batch is dropped instead of retried because a duplicate would overstate crawl counts.

Which one to install

Install one, not several. The Worker sees every request including those served from cache, so pick it whenever Cloudflare is in front of your site.

Install: Cloudflare

npx degit tailwin/crawler-worker tailwin-tracker && cd tailwin-tracker
npx wrangler secret put TAILWIN_SITE_KEY   # from your Tailwin dashboard
npx wrangler deploy
# Cloudflare dash → Workers Routes → yourdomain.com/* → tailwin-crawler-tracker

Install: WordPress

1. Upload the plugin folder to /wp-content/plugins/ and activate it.
2. Copy your site key from your Tailwin dashboard, under Site → Crawler tracker. It starts with twk_.
3. In WordPress, open Settings → Tailwin Tracker, paste the key, and save.
4. Press Send a test hit. A confirmation means the connection works.

One caveat worth knowing before you install it: a full-page cache that serves a crawler from disk without starting WordPress is invisible to every WordPress plugin, including this one. If most of your traffic is served from cache, use the Worker instead.

Install: Next.js or Express

npm install @tailwin/tracker

// Next.js — middleware.ts
import { NextResponse } from "next/server";
import { createNextTracker } from "@tailwin/tracker/next";

const track = createNextTracker({ siteKey: process.env.TAILWIN_SITE_KEY! });

export function middleware(request) {
  track(request);
  return NextResponse.next();
}
export const config = { matcher: "/((?!_next/static|_next/image).*)" };

// Express
import { tailwinTracker } from "@tailwin/tracker/express";
app.use(tailwinTracker({ siteKey: process.env.TAILWIN_SITE_KEY }));

Next.js middleware runs before the response exists, so those hits carry no status code. Express reports the real one. Nothing is awaited on the request path, and a failed batch is dropped rather than retried, because a duplicated hit would overstate your crawl counts.

Why some crawlers show as unverified

On Cloudflare we use their verified-bot signal. Off Cloudflare there is no single equivalent that this software currently implements for AI crawlers. The WordPress plugin can use documented, forward-confirmed reverse DNS for Googlebot, Bingbot, and Applebot. The Node middleware has no stronger verification signal. Other matches are reported as unverified instead of claiming certainty from a user-agent alone.

Questions? sgirard@girardmedia.com