Skip to main content
Back to registry

crawl

tavily-ai/skills

Crawl websites to extract content from multiple pages. Ideal for documentation, knowledge bases, and site-wide content extraction.

Installs3
Install command
npx skills add https://github.com/tavily-ai/skills --skill crawl
Security audits
Gen Agent Trust HubFAIL
SocketPASS
SnykWARN
About this skill
Crawl websites to extract content from multiple pages. Ideal for documentation, knowledge bases, and site-wide content extraction. The script uses OAuth via the Tavily MCP server. No manual setup required - on first run, it will: Note: You must have an existing Tavily account. The OAuth flow only supports login — account creation is not available through this flow. Sign up at tavily.com first if you don't have an account. If you prefer using an API key, get one at and add to ~/.claude/settings.json : Examples: When output_dir is provided, each crawled page is saved as a separate markdown file. Start with max_depth=1 and increase only if needed.

Source description provided by the upstream skill listing. Community reviews and install context appear in the sections below.

Community Reviews

Latest reviews

Sign in to review

No community reviews yet. Be the first to review.

Browse this skill in context
FAQ
What does crawl do?

Crawl websites to extract content from multiple pages. Ideal for documentation, knowledge bases, and site-wide content extraction.

Is crawl good?

crawl does not have approved reviews yet, so SkillJury cannot publish a community verdict.

What agent does crawl work with?

crawl currently lists compatibility with Agent compatibility has not been published yet..

What are alternatives to crawl?

Skills in the same category include telegram-bot-builder, flutter-app-size, sharp-edges, iterative-retrieval.

How do I install crawl?

npx skills add https://github.com/tavily-ai/skills --skill crawl

Related skills

More from tavily-ai/skills

Related skills

Alternatives in Software Engineering