Skills

All Skills

crawler

Skills tagged with #crawler

@utkarsh

nextjs-seo-2026

Complete SEO and Answer Engine Optimization (AEO) skill for Next.js 15 apps. Covers metadata, structured data (JSON-LD), sitemap, robots.txt, Core Web Vitals, dynamic OG images, and AI crawler optimization. Use this skill when building or auditing SEO for any Next.js project.

SEOSearch Engine Optimization
5mo ago
1
@sharozdawa
MCP

Robotstxt Ai

Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.

mcpgithubai
sharozdawa/robotstxt-ai
5mo ago
0
@zubair-trabzada

geo

GEO-first SEO analysis tool. Optimizes websites for AI-powered search engines (ChatGPT, Claude, Perplexity, Gemini, Google AI Overviews) while maintaining traditional SEO foundations. Performs full GEO audits, citability scoring, AI crawler analysis, llms.txt generation, brand mention scanning, platform-specific optimization, schema markup, technical SEO, content quality (E-E-A-T), and client-ready GEO report generation. Use when user says "geo", "seo", "audit", "AI search", "AI visibility", "optimize", "citability", "llms.txt", "schema", "brand mentions", "GEO report", or any URL for analysis.

zubair-trabzada/geo-seo-claude+13 more
5mo ago
2.5K0
@ai-analyst-lab

/notion-ingest — Notion Workspace Crawler

> Crawls a Notion workspace to extract business terms, metrics, product docs, > and team structure. Populates the organization knowledge system.

ai-analyst-lab/ai-analyst+30 more
5mo ago
990
@piiiico
MCP

AEO Audit

AEO audit: score any website 0-100 for AI visibility. Checks schema, meta, content, AI crawlers.

mcpaiweb
piiiico/aeo-mcp-server
5mo ago
0
@houtini-ai
MCP

Io.Github.Houtini Ai/Seo Crawler Mcp

Crawl and analyse websites for SEO errors using Crawlee with SQLite storage

mcpgithubaiwebragsqlite
houtini-ai/seo-crawler-mcp
5mo ago
0
@apify
MCP

Apify Mcp Server

Extract data from any website with thousands of scrapers, crawlers, and automations on Apify Store ⚡

mcpapiweb
apify/apify-mcp-server
5mo ago
0