The web transformed into data for your AI, in France
Crawl4AI crawls websites and converts them into clean, structured Markdown ready for your LLMs. The ideal engine to power a RAG. Installed and maintained by DINAO, on our French servers.
What is Crawl4AI?
Crawl4AI is an open-source web crawler and scraper specifically designed for artificial intelligence. Where traditional tools produce raw HTML that is difficult to exploit, Crawl4AI cleans pages, removes noise, and generates clean, structured Markdown, directly digestible by large language models and RAG pipelines.
The engine relies on a headless browser capable of executing JavaScript, managing sessions, hooks, and interactions, allowing it to crawl entire dynamic sites reliably and in parallel, with proxy management. For structured extraction, it offers strategies using CSS selectors, XPath, or LLM-driven schemas, returned in JSON format.
Exposed via an REST API and a Python client, Crawl4AI integrates naturally into AI processing chains to build knowledge bases or power agents. Hosted by DINAO in France, it provides you with clean raw web material while keeping targets and extracted data within the territory.
Host Crawl4AI at DINAO
Resource tiers compatible with Crawl4AI prerequisites (minimum 2 vCPU / 2 Go / 5 Go). Hosted in France, fully managed.
- 2 dedicated vCPU
- 4 Go RAM
- 40 GB NVMe
- Daily backups
- Managed & monitored by DINAO
- 4 dedicated vCPU
- 8 Go RAM
- 80 GB NVMe
- Daily backups
- Managed & monitored by DINAO
- 8 dedicated vCPU
- 16 Go RAM
- 160 GB NVMe
- Daily backups
- Managed & monitored by DINAO
1 hidden tier(s) (insufficient resources for this app) : Découverte
This application uses AI
The container hosts the application, not the AI engine (which requires dedicated GPUs) : inference is handled externally, with your own provider key. Prioritize a sovereign engine — Mistral AI, NumSpot, Scaleway, or OVHcloud AI Endpoints (France, GDPR) ; an international provider (OpenAI, Anthropic…) only if a specific capability requires it. Inference subscriptions are not included in hosting.
Technical details
You might be wondering…
What is Crawl4AI used for?
Crawl4AI crawls websites and transforms them into clean, structured Markdown, directly usable by LLMs. It is the ideal engine to build the knowledge base of a RAG system or power an AI agent.
Does it handle dynamic JavaScript sites?
Yes. Crawl4AI uses a headless browser that executes JavaScript, manages sessions and interactions, allowing it to crawl dynamic pages that traditional scrapers cannot see.
How to extract specific fields?
You define an extraction strategy using CSS selectors, XPath, or an LLM-driven schema. Crawl4AI then returns the structured data in JSON format.
Do my extracted data remain in France?
Yes. The engine runs on DINAO infrastructure in France and extracted content is stored on the territory. See the sovereignty note below regarding the use of external LLM APIs.
Do I need an LLM to use Crawl4AI?
No for crawling and CSS/XPath extraction, which do not require any model. An LLM is only involved if you choose schema-driven extraction by AI; it can then be local (sovereign) or external.
