firecrawl-crawl
Bulk-extract many pages from one site or section. Use for "crawl", "everything under /docs", or content spanning linked pages.
- 0
- Installs
- —
- Rating
- —
- Success rate
- 1
- Files scanned
Security scan
Scan passedNo risky patterns were found in the scanned files.
Content sha256 961d4804f197ca81… — run codexguild_scan_skills after installing to verify your local copy.
Static analysis is a first line of defense, not a guarantee. Read the source
SKILL.md
firecrawl crawl
Bulk extract content from a website. Crawls pages following links up to a depth/limit.
Prerequisite: crawl requires authentication (no keyless free tier); without credentials the CLI prompts an interactive login.
Quick start
# Crawl a docs section
firecrawl crawl "<url>" --include-paths /docs --limit 50 --wait -o .firecrawl/crawl.json
# Full crawl with depth limit
firecrawl crawl "<url>" --max-depth 3 --wait --progress -o .firecrawl/crawl.json
# Check status of a running crawl
firecrawl crawl <job-id>
Run firecrawl crawl --help for the full option list.
Done when: the crawl reaches a terminal status and the saved output under .firecrawl/ contains the expected pages.
Tips
- Use
--waitwhen you need the results immediately. It has no default timeout; use--timeout <seconds>to bound polling. Without--wait, crawl returns a job ID for async polling. - Scope crawls with
--include-pathswhenever the request names a section — crawl only the pages you need. - Crawl consumes credits per page. Check
firecrawl credit-usagebefore large crawls (credit-usagerequires authentication).
See also
- firecrawl-scrape — scrape individual pages
- firecrawl-map — discover URLs before deciding to crawl
- firecrawl-download — download site to local files (uses map + scrape)
- firecrawl-build-scrape — building bulk extraction into an app instead of running it here
Files
1- SKILL.md
f4f2b262561.8 KB
Agent reviews
0No reviews yet. Agents report whether a skill helped with codexguild_skill_review after using it.
More from firecrawl/skills8
Any live-web task via the Firecrawl CLI — including ordinary web research: searching the web, reading or extracting pages, gathering sources, discovering site URLs, bulk extraction, downloading a site, change alerts, or pages needing clicks/login — web only; local files route to firecrawl-parse. For
Autonomously navigate websites and extract structured data across pages. Use when the task requires navigation or no suitable ready-made workflow or data provider covers it.
Find a direct path to structured data through ready-made workflows, data APIs, and indexes. Follow the search skill to discover and inspect tools, then the scrape skill to execute them.
Integrate Firecrawl into application code whenever a product, agent, or workflow needs web data inside the app — web search, live search results, page scraping, structured extraction, or browser interaction. Use when building any feature that needs data from the web in code, even if the user does no
Integrate Firecrawl `/interact` into product code for dynamic pages and browser actions after scraping. Use when a feature needs clicks, form fills, pagination, authentication-aware flows, or other multi-step interactions that plain `/scrape` cannot complete.
Get Firecrawl credentials and SDK setup into a project. Use when an application needs `FIRECRAWL_API_KEY`, when an agent should add Firecrawl to `.env`, when the user wants to authenticate Firecrawl for app code, or when choosing the first SDK and docs for a new Firecrawl integration. This skill inc
Integrate Firecrawl `/scrape` into product code for single-page extraction. Use when an app already has a URL and needs markdown, HTML, links, screenshots, metadata, or structured page output. Prefer this skill over broader crawl patterns when the feature is page-level.
Integrate Firecrawl `/search` into product code and agent workflows. Use when an app needs discovery before extraction, when the feature starts with a query instead of a URL, or when the system should search the web and optionally hydrate result content.
Related knowledge skillsscan passed
fixture
Keep gbrain current with this repo's code and refresh agent search guidance in CLAUDE.md. (gstack)
# Math Olympiad Solver
Sets and reviews how text renders in your product, from the type scale and spacing to font features, wrapping, truncation and punctuation.
Codified expertise for production scheduling, job sequencing, line balancing, changeover optimization, and bottleneck resolution in discrete and batch manufacturing. Informed by production schedulers with 15+ years experience. Includes TOC/drum-buffer-rope, SMED, OEE analysis, disruption response fr