Privacy Law in AI Is Infrastructure Policy

The Pattern

The question is not whether AI companies collect your data. They do. The question is why the dominant framework for stopping them treats collection as a disclosure problem rather than a prohibition problem.

The California Consumer Privacy Act lets you request deletion of your data. The EU’s General Data Protection Regulation gives you a right to erasure under Article 17. Both assume the harm is downstream — that the problem is what happens to your data after it is collected, and that the fix is to let you ask for it back. Both assume you know your data was taken. Both assume the holder will comply. Both assume the data still exists in a form that can be located and deleted.

None of those assumptions hold for AI.

The regulated act is extraction at ingestion

A model is trained on a corpus. The corpus includes your words, your images, your behavior traces, your pauses and hesitations and typing rhythms. The ingestion happens at training time, invisibly, at scale. You were not asked. You were not notified. The data is no longer stored as data. It is baked into the model’s weights. GDPR Article 17 gives you the right to erasure, but erasure of what? The data is not a record that can be located and deleted. It is a pattern the model learned. There is no off switch, because the data is no longer stored. It is learned.

This is the pattern named training-data extraction. The harm is at ingestion, not at downstream misuse. A privacy law that regulates the output — the model’s behavior, the generated text, the reproduced fragment — misses the point. The taking happened years before the output. The surfacing is the proof. The extraction was the harm.

The policy implication is specific. The regulated act in AI privacy is the ingestion of personal data into a training corpus without meaningful consent. Not the downstream sale. Not the targeted ad. Not the breach. The ingestion. If the law does not reach the ingestion, it does not reach the harm.

The EU AI Act takes a step in this direction with Article 5, which prohibits certain AI practices outright — social scoring, real-time biometric identification in public spaces, emotion recognition in workplaces and schools. This is risk-tiered prohibition, not disclosure-and-consent. The model is not “tell people you are scoring them and let them opt out.” The model is “you may not score them.” The distinction is the entire argument.

Affect surveillance is the AI-native privacy violation

The FTC’s Operation AI Comply crackdown targeted companies making fraudulent AI claims. One firm was selling “AI” that was resold email lists. The industry lies about what the technology does. That is documented. But the more serious privacy violation is not the fraudulent claim. It is the accurate one.

Affect surveillance is the monitoring and interpretation of a person’s emotional state by a machine, at scale, without meaningful consent. The system reads your face, your voice, your typing rhythm, your hesitation. It infers how you feel. It uses that reading to time a message, adjust a price, or steer a decision. You did not offer your emotional state as data. It was taken from the traces you could not help leaving.

This is not hypothetical. Emotion AI is a product category. Affect recognition is sold for workplace monitoring, customer service optimization, and ad targeting. The EU AI Act prohibits emotion recognition in workplaces and schools under Article 5. The United States has no equivalent prohibition. The CCPA does not reach it, because the CCPA regulates data sale and retention, not inference. A system can comply with every CCPA requirement and still read your inner life without your consent, as long as it discloses the practice in a privacy policy no one reads.

The demand is not “disclose it.” The demand is “prohibit it.” A system that reads your hesitation, your fatigue, your frustration, and times interventions to your vulnerability is not a service. It is a condition. GDPR Article 22 gives you the right not to be subject to a decision based solely on automated processing. But Article 22 addresses the decision, not the inference. The inference happens before the decision. The inference is the harm. The decision is the output.

Biometric collection is regulated at the source

The door-knock pattern documents the pipeline from doxxing database to deportation flight. Politico reported that a senior DHS investigations official admitted in federal court that 75 of approximately 100 names in the department’s “tiger team” dossiers came from Canary Mission. NBC News confirmed the testimony. Canary Mission is a private database. The government used it as a lead source. The pipeline is frictionless.

Betar USA, a militant pro-Israel group, claimed to use facial recognition AI to identify protesters, including those wearing face coverings. The technology exists. The databases exist. The pairing of private doxxing infrastructure with government deportation machinery is documented. The question is not whether it happened. The question is why the biometric collection that made it possible is not prohibited at the source.

The EU AI Act prohibits real-time biometric identification in public spaces under Article 5. The prohibition is not absolute — it has law enforcement exceptions — but the default is prohibition, not disclosure. The United States has no federal prohibition on facial recognition. Several cities have banned it locally. The patchwork is the failure. A surveillance architecture that is national in scope cannot be regulated by a city council.

Privacy law in AI has to reach biometric collection at the source, not just the sale of biometric data downstream. The harm is the collection. A database of faces built without consent is a weapon whether or not it is sold. The sale is the profit. The collection is the threat.

Why disclosure-and-consent is not enough

The CCPA model is disclosure-and-consent. The company tells you what it collects. You have the right to request deletion. You have the right to opt out of sale. The model assumes you read the disclosure, understand it, and exercise your rights. The model assumes the disclosure is honest. The model assumes the data is deletable.

None of these hold at scale. Privacy policies are unread. Consent is manufactured through dark patterns. Data is not deletable once it is baked into a model. And the disclosure is not honest — it is written by lawyers to minimize liability while maximizing collection. The FTC had to launch Operation AI Comply because companies were lying about what their AI did. The lying is not a bug in the disclosure model. The lying is the disclosure model. The disclosure is a liability management tool, not a consent mechanism.

Risk-tiered prohibition is the alternative. The EU AI Act does not ask whether you consent to social scoring. It prohibits social scoring. It does not ask whether you consent to emotion recognition in your workplace. It prohibits emotion recognition in your workplace. The model is not “tell me what you are doing to me and let me decide.” The model is “you may not do this to me.” The distinction is the entire argument. Disclosure-and-consent treats the subject as a consumer choosing a product. Prohibition treats the subject as a citizen with rights the holder cannot waive.

The cost-imposition theory

The archives already exist. Canary Mission has the names. Flock Safety has the movement records. Palantir has the deportation profiles. The data is stored. The question is not whether it was collected — it was — but whether it remains cheap to read.

This is where encryption enters the policy frame. The EFF’s position is that encryption is a human rights issue. Access Now frames it the same way. Both are correct. But the specific demand for AI privacy is narrower and more structural: encryption at rest, applied retroactively to archives that were built in plaintext, so that the subpoena returns ciphertext instead of names.

The frame is plaintext liability. The institution that stored personal data in plaintext chose a form that made exposure trivial. The breach, the subpoena, the unauthorized access — these are the consequences. The plaintext was the choice. The choice has a cost. The cost is the liability.

Retroactive encryption is the remedy. The archive cannot be deleted — it is legally retained, or politically protected, or simply too large to purge. But it can be encrypted, so that access requires a key the holder must deliberately produce. The cost of decryption is the protection. The lever is not delete. They won’t. The lever is cost.

The engage-don’t-abstain posture

The suspicion of AI is correct. The companies building the most powerful systems are not neutral actors. OpenAI, Anthropic, Google, Meta, Palantir. They are firms with shareholders, lobbying budgets, and a structural interest in framing their product as inevitable. Wired reported a $140 million super PAC funded by executives affiliated with OpenAI and Palantir paying influencers to stoke China fears and drive adoption. The right is building the infrastructure of the next century. The left is moralizing while it happens.

But suspicion is not a strategy. The tenant organizer who uses an AI tool to scan a thousand leases for illegal clauses is not contaminated by using the tool. The public defender’s office that uses a model to draft motions faster is not surrendering. The workers’ center that uses AI to translate into seven languages without a six-figure contract is not complicit. The question is not whether to use AI. The question is on what terms.

Encrypted inference is the term for using AI on terms that do not feed the surveillance dividend. The prompt is encrypted before it reaches the model. The model processes it inside a trusted execution environment. The output returns without the operator retaining the input. The tool did the work. The operator did not get a dossier. This is the inference-layer equivalent of running a local model on a laptop. Not purity. Not surrender. Engagement on terms that do not make you the raw material.

The policy demand is not “do not touch the machinery.” The policy demand is “touch it on terms that do not feed the surveillance dividend.” The first is a purity test. The second is a program. Purity tests do not win infrastructure fights.

What this establishes

Privacy law in AI is infrastructure policy. The regulated act is extraction at ingestion. Affect surveillance is prohibited, not disclosed. Biometric collection is regulated at the source. The model is risk-tiered prohibition, not disclosure-and-consent theater. The archives that already exist are encrypted retroactively, because the lever is not delete but cost. And the people who need AI tools use them on encrypted terms, because abstention is surrender and engagement is strategy.

The surveillance architecture that makes the propaganda, the doxxing, and the deportations work runs on plaintext and consent theater. The fix is not better consent. The fix is less plaintext. The fix is prohibition of the practices that should never have been permitted, and encryption of the archives that should never have been built.

The question is not whether AI companies will collect your data. They will, unless the law stops them. The question is whether the law will treat collection as a disclosure problem or a prohibition problem. The answer is the entire argument.