GitHub 项目简介: Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML,…
README: Your crawlers will appear human-like and fly under the radar of modern bot protections even with the default configuration.
README: The fastest way to try Crawlee out is to use the **Crawlee CLI** and choose the **Getting started example**. The CLI will install all the necessary dependencies and add boilerplat…
README: Because it uses `PlaywrightCrawler` we also need to install [Playwright](https://playwright.dev). It's not bundled with Crawlee to reduce install size.
README: By default, Crawlee stores data to `./storage` in the current working directory.