Share This Article
The EDPB web scraping guidelines finally set out how the GDPR applies to the data that trains generative AI — and they raise open questions that could determine whether AI models can still be built in Europe.
On 7 July 2026, the European Data Protection Board adopted Guidelines 03/2026 on web scraping in the context of generative AI, opening them to public consultation until 30 October 2026. They came out of the same plenary that produced the new anonymisation guidelines and the final blockchain guidance, which tells you something about the direction of travel: the EDPB is systematically mapping the GDPR onto every layer of the AI stack.
And this layer is the foundation. No scraped data, no training dataset. No training dataset, no model. Whatever the guidelines say about scraping, they say about the feasibility of developing artificial intelligence in the European Union.
The document is only 22 pages. Do not let that mislead you. The level of granularity expected from controllers is remarkable, and the practical consequences for anyone training or fine-tuning a model, not just the large language model developers, but any company that scrapes, directly or through a vendor, are significant.
What the EDPB web scraping guidelines actually say
Let me start with the uncontroversial part, because the useful debate is elsewhere.
The GDPR applies to web scraping whenever the operations involve personal data: extraction, cleaning, structuring, storage. “Publicly available” is not a magic exemption, and the EDPB repeats it with no ambiguity. The guidelines cover two scenarios — an organisation that scrapes itself or instructs a third party to do so, and an organisation that reuses a dataset scraped by someone else. Pure data brokers who scrape and resell without training models are left out, as is public sector scraping.
From there, the guidelines walk through the principles:
- Roles. The scraper is not automatically the controller. Where a scraper builds a dataset on the documented instructions of an AI developer, it may be a processor, with the developer as controller. Where a developer reuses a dataset already scraped, each party answers for its own processing, and the original scraper is not, in principle, responsible for the reuse. Joint controllership arises where collection criteria are set jointly.
- Transparency. Individual privacy information notice is often impossible or disproportionate, and Article 14(5)(b) GDPR can be relied on but only after a real balancing exercise on the dataset as a whole, weighing the number of data subjects, the age of the data and the safeguards adopted. The fallback is not silence: the controller must publish the privacy information notice, including a precise indication of the sources, ideally with domain names, URLs in searchable format and collection dates.
- Data minimisation. A long list of expected measures, before and after collection: consider synthetic data first, define precise collection criteria, run a data mapping exercise, filter out categories that are not needed, exclude websites structurally likely to contain sensitive data or data of minors, exclude sites that oppose scraping via robots.txt, ai.txt or CAPTCHA, apply syntax-based filters during collection, and anonymise or pseudonymise where feasible.
- Accuracy. Scrape from reliable, maintained sources, timestamp the data, validate before training. And, this is easy to miss, accuracy also bites on the output: if the model is expected to generate personal data once on the market, that output must be accurate too.
- Legal basis. Consent is off the table for indiscriminate collection, and the EDPB is explicit that publishing data online is not consent to scrape it. Legitimate interest under Article 6(1)(f) remains the realistic route, subject to the three cumulative conditions of the EDPB Guidelines 1/2024 and Opinion 28/2024 on AI models: a real, lawful, precisely articulated interest; necessity, meaning no equally effective and less intrusive alternative; and a balancing test.
- Special categories. Prohibited in principle. Scraping that captures Article 9 data needs both an Article 6 basis and an Article 9(2) derogation. The EDPB accepts that the CJEU’s reasoning in GC & Others (C-136/17) developed for search engines may apply to incidental and residual collection, within the controller’s “responsibilities, powers and capabilities“, provided technical and organisational measures prevent collection and dissemination. But there is no blanket exemption and no blanket assessment: case by case, every time.
On balance, the EDPB web scraping guidelines are a serious, well-constructed document, and considerably more pragmatic than some feared. The recognition that residual sensitive data does not automatically make training unlawful matters enormously. So does the acknowledgement that data minimisation does not prohibit training on large volumes of data.
But improvements and difficulties can coexist. And a consultation is exactly the moment to say where the difficulties are.
Why the EDPB web scraping guidelines matter for the growth of AI
Here is the strategic point. Europe has spent the last two years debating whether it can host AI development or merely consume it. The Digital Omnibus, the competitiveness agenda, the Draghi report — the whole conversation assumes that European companies can actually build models here.
The training dataset is where that assumption is tested. If the compliance cost of assembling a lawful dataset is high but predictable, European AI development survives and the guidelines become a genuine competitive asset, because trustworthy data provenance is worth something. If the cost is unquantifiable, the rational decision for any board is to buy a model trained elsewhere and deploy it which achieves nothing for the protection of data subjects, since the same data was scraped anyway, only by someone outside the reach of the EDPB.
This is a point worth raising constructively in the consultation, because it is one the EDPB is well placed to address: guidance of this kind is naturally most visible to the organisations already committed to complying with it.
There is also a structural point that deserves attention. The measures the EDPB expects, precise collection criteria, source exclusion, filtering, mapping, pseudonymisation, memorisation controls, are not merely documentary. They are engineering. Larger developers are generally equipped to absorb them. A European scale-up fine-tuning an open-weight model for a vertical use case may find the same expectations proportionately heavier. Some calibration of the guidance to the size and scope of the processing would help ensure that the framework supports innovation across the market, and not only at the top of it.
What to do now: a practical checklist
Before the debate on the open points, there is work that can start immediately, because none of it depends on how the consultation ends. If your organisation scrapes, commissions scraping, or buys scraped datasets:
1. Map who you are. Controller, joint controller or processor, per processing activity. If you instruct a vendor, check the instructions are documented and specific on sources and categories — that is what makes them a processor rather than a joint controller.
2. Re-open your legitimate interest assessment. A generic LIA will not survive. Articulate the interest precisely (commercial or research, internal or external), document why less intrusive alternatives — synthetic or pseudonymised data, narrower criteria — were not equally effective, and record the balancing test with the mitigating measures actually implemented.
3. Write down your collection criteria before you crawl. Untargeted crawling is the hardest position to defend, because the EDPB links it directly to the controller not knowing what it holds.
4. Honour opposition signals. robots.txt, ai.txt, CAPTCHA and authentication walls. Log the exclusions — the evidence matters more than the practice.
5. Exclude structurally risky sources. Sites used mainly by minors, and sources structurally likely to carry financial, location or Article 9 data.
6. Timestamp and validate. Record collection dates, prefer official and maintained sources, spot-check samples before training.
7. Publish an Article 14(5)(b) notice that actually works. Categories of data, purposes, legal basis, crawler characteristics, and the source list — domain names and URLs in searchable format where possible, with the reasons for any omissions.
8. Address the model, not just the dataset. Memorisation and regurgitation controls, output filtering and accuracy of generated personal data all feed back into the balancing test.
9. Paper the supply chain. For purchased datasets: sources, exclusion criteria, audit rights, warranties on how the dataset was assembled, and a contact point for the originating controller.
10. Run a DPIA — and consider publishing it. The EDPB lists publication among the appropriate safeguards where individual notice is disproportionate.
Five open questions before 30 October 2026
1. How wide is the “responsibilities, powers and capabilities” door? GC & Others was about a search engine responding to referencing requests. Extending it to the untargeted crawling of the open web is a real analytical leap. If it works, incidental sensitive data is manageable. If a data protection supervisory authority reads it narrowly at enforcement stage, most large-scale scraping is unlawful under Article 9 no matter what filters were applied. Which reading will prevail?
2. Does robots.txt now have legal effect? The guidelines treat opposition signals, robots.txt, ai.txt, CAPTCHA, both as a data minimisation expectation and as a factor in reasonable expectations, with Examples 4 and 5 turning on precisely that. A technical convention that carries no legal status of its own is therefore acquiring real weight in a GDPR balancing test. That is a meaningful development and one worth examining openly during the consultation. What happens where the signal is ambiguous, changed after collection, or set by a platform on terms its own users have not chosen?
3. What is the temporal scope? Models already trained cannot easily unlearn, the EDPB says so itself when discussing the impossibility of deleting personal data from a trained model. Do the EDPB web scraping guidelines apply to datasets assembled before July 2026? To fine-tuning a model whose base training predates them? The document is silent, and silence here is expensive.
4. How does this interact with the AI Act and the Digital Omnibus? The AI Act has its own data governance requirements for training data. The Digital Omnibus reopened the definition of personal data before the contested amendment was dropped following the EDPB and EDPS Joint Opinion 2/2026. Controllers now face parallel regimes, drafted by different institutions, on the same dataset. Who reconciles them, and when?
5. Is the source list workable? Publishing domain names and URLs in searchable format, with collection periods, is transparency in its purest form. It is also, for an untargeted crawl across millions of URLs, a substantial disclosure of what may be a competitive asset. Is there a proportionate middle ground between a meaningless list of source categories and a full crawl inventory?
My view
The EDPB web scraping guidelines are not the obstacle some will claim. They are closer to a roadmap than a wall, and the flexibility on transparency and on residual sensitive data shows an institution that has listened. The direction is sound. What remains is uncertainty at exactly the points where a general counsel has to give a clear answer to a business that wants to invest.
That uncertainty is addressable, and the consultation is the right forum in which to address it, before divergent national practice develops around it.
So I will put the same question to you that I have been putting to colleagues over the last week: which of these open questions is the one your organisation cannot live with? I suspect the answers differ enormously depending on whether you train, fine-tune or merely deploy, and that difference is itself something the EDPB should hear before 30 October 2026.
On a related topic, you may find of interest the article “EDPB Anonymisation Guidelines: What They Mean for AI Systems” on this blog, which covers the companion guidance adopted at the same plenary.
The implications of the EDPB web scraping guidelines depend heavily on whether your company trains, fine-tunes or deploys AI models, and on how your training data was assembled. If you would like to discuss what they mean for your organisation — or to contribute to the public consultation before 30 October 2026 — you can reach me at giulio.coraggio@dlapiper.com, and our team is happy to help.