The EU AI Act started applying in phases from August 2024, with the high-risk AI system requirements hitting full force in August 2026. If your team is deploying AI in or for the EU market, you're now operating under the most detailed AI regulatory framework in the world. Unlike GDPR, which focused on data, the EU AI Act focuses on AI systems themselves: how they are built, where they run, how decisions are logged, and what oversight mechanisms exist. For teams already weighing on-prem vs cloud cost and control tradeoffs, the EU AI Act adds a structured compliance layer on top.
This guide covers the infrastructure-level implications: where you run compute, what you log, how you govern models across their lifecycle, and how to pick a GPU cloud provider that doesn't create compliance headaches.
What the EU AI Act Requires for GPU Cloud Deployments
The regulation (Regulation (EU) 2024/1689) operates on a four-tier risk framework. The tier your AI application falls into determines your compliance obligations. In operational terms for a team running workloads on GPU cloud, this is what each tier means.
| Risk Tier | Examples | GPU Cloud Implication |
|---|---|---|
| Unacceptable | Social scoring, real-time biometric identification in public spaces for law enforcement (with narrow exceptions) | Cannot deploy regardless of infrastructure |
| High-risk | Medical diagnosis AI, CV screening tools, law enforcement AI | Full documentation, audit logging, EU data residency preferred |
| Limited risk | Chatbots, deepfake detection tools | Transparency notices required; no infrastructure mandate |
| Minimal risk | LLM developer tools, internal productivity AI | No mandatory compliance; best practices encouraged |
For most engineering teams, the practical reality is this: internal developer tooling, coding assistants, and research workloads fall into minimal or limited risk. Customer-facing AI that touches hiring, healthcare, credit scoring, or legal decisions is high-risk and requires the full compliance stack.
One important carve-out: General Purpose AI (GPAI) models trained above 10^25 FLOPs are classified as systemic risk under Article 51, with obligations defined in Articles 53 and 55. This is the "systemic risk" threshold. Models below this threshold still have GPAI obligations but lighter ones. If you are fine-tuning or deploying open models like Llama, Qwen, or Gemma, those obligations fall primarily on the base model provider. Your downstream modifications have lighter requirements.
Enforcement timeline:
- 2 February 2025: Prohibited AI provisions applied
- 2 August 2025: GPAI model obligations applied
- 2 August 2026: High-risk AI system obligations apply in full
- 2 August 2027: High-risk AI obligations for systems that are safety components in products regulated under existing EU product safety legislation (medical devices, civil aviation) take full effect
Data Residency: Where Your Model Weights and Inference Data Must Live
This is where GPU cloud decisions have the most direct compliance impact. GDPR still governs personal data used in training and passed through inference endpoints. The EU AI Act adds obligations on top for high-risk applications.
A few things worth being precise about:
Model weights are not personal data in most cases. But if they encode personal information from training on medical records or HR data without proper anonymization, data protection authorities treat them with caution. The ICO and national DPAs have been increasingly specific about this.
Inference inputs from EU users are often personal data. A user querying an LLM with their name, situation, or medical history creates a personal data processing event. Where those inputs are processed matters for GDPR Chapter V (international transfers). You need either adequacy decisions, Standard Contractual Clauses (SCCs), or Binding Corporate Rules to transfer that data outside the EU.
The practical rule: for high-risk AI systems serving EU users, keep inference traffic within the EU. For internal tooling, anonymized workloads, or non-personal-data pipelines, there is more flexibility.
Here is a breakdown of what "data" means across a GPU deployment:
| Data Type | EU AI Act Relevance | GDPR Relevance | Recommended Approach |
|---|---|---|---|
| Model weights | Store in auditable location for high-risk | Low unless personal data encoded | EU-region object storage or GPU node local storage |
| Training data | Must be documented in technical file | High if personal data | Process in EU, document provenance |
| Inference inputs | Feed into audit logs for high-risk systems | High if personal data | Keep in EU region, encrypt in transit |
| Inference outputs | Logged for audit trail | Depends on content | Retain per your risk tier requirement |
One way to simplify data residency compliance is running self-hosted inference where you control the entire data path. See self-hosted inference with OpenAI-compatible APIs for how to set that up without giving up the model ecosystem your team relies on.
Risk Classification: How Your GPU Workload Type Affects Your Compliance Tier
Classification determines everything downstream. Here are the most common AI workload patterns and how they map to the EU AI Act risk tiers.
LLM fine-tuning for enterprise apps: The fine-tuned model's risk tier depends on the deployment use case, not the training workload itself. A Llama 4 fine-tune used for internal knowledge management is minimal risk. The same model fine-tuned for triage recommendations in a clinical setting is high-risk.
RAG pipelines for healthcare or HR: If the downstream application makes or assists consequential decisions about individuals (access to healthcare services, hiring outcomes), it is high-risk regardless of whether the model is a foundation model or a specialized one. The application determines the tier.
Computer vision for access control or surveillance: High-risk or potentially prohibited depending on specific use. Real-time biometric identification in public spaces for law enforcement purposes is prohibited, with narrow exceptions for searching for victims of serious crime, preventing imminent terrorist threats, and locating suspects of specific serious offenses. Post-hoc identification in specific law enforcement contexts is high-risk with additional conditions.
Research and experimentation workloads: Minimal risk. No deployment obligations until the system goes into production use for EU users.
Is Your AI System High-Risk? A 3-Question Test
- Does the system make or directly inform decisions about: healthcare, hiring, credit, education, law enforcement, or border control?
- Is the output used by humans to make decisions about individuals?
- Is the system deployed for EU residents or businesses operating in the EU?
If you answered yes to questions 1 and 2, you almost certainly have a high-risk system. If yes to question 3, the EU AI Act applies to you.
This matters for agentic AI systems in particular. Agents that take autonomous actions on behalf of users, especially in regulated domains, face heightened oversight requirements precisely because a human is not reviewing each individual output before it has effect.
Model Governance: Logging, Auditing, and Transparency on GPU Infrastructure
Governance is where compliance becomes an engineering problem. Here is what the EU AI Act actually requires in technical terms, and how to implement it.
What Audit Logging Looks Like in Practice
For high-risk systems, the EU AI Act requires logs that allow authorities to reconstruct how the system behaved when a particular decision was made. Specifically:
- Timestamps of when the system was used
- Input context sufficient to reconstruct decisions
- Output records
- Model version active at time of inference
- Session or user identifiers (where legally permissible under GDPR)
In practice, this means enabling request logging at the inference server layer. For vLLM deployments:
vllm serve meta-llama/Llama-4-Scout-17B-16E-Instruct \
--enable-log-requests \
--enable-log-outputsFor structured log shipping, OpenTelemetry works well. Pipe logs to an S3-compatible store in an EU-region bucket, with access controls limiting who can read or delete log data.
A self-hosted LLM observability stack gives you the detailed per-request audit logs Article 12 requires - see the self-hosted LLM tracing infrastructure guide for a reference deployment on Spheron.
For AI systems that require runtime content enforcement to meet EU AI Act Article 9 risk management requirements, NeMo Guardrails on GPU cloud covers the full deployment pattern including audit-ready rail decision logging.
Access Controls and Model Versioning
High-risk systems require documented model versions in production, controlled rollback capability, and access control logs showing who modified the model deployment. In a Kubernetes environment, this means Kubernetes RBAC controls who can update the inference deployment, combined with image digest pinning to ensure the exact model version is recorded. For a detailed walkthrough of how DRA, KAI Scheduler, and namespace-level isolation work together, see the Kubernetes GPU orchestration guide.
Model version tracking should be as simple as:
# Pin to exact digest, not just a tag
containers:
- name: inference
image: your-registry/llama4:sha256-abc123...This way, every deployment event in your Kubernetes audit log references a specific, immutable artifact.
Human Oversight Requirements
High-risk AI systems must include mechanisms for human oversight. The EU AI Act does not specify implementation details, but the intent is clear: humans must be able to understand, override, and correct the system's outputs. In GPU infrastructure terms:
- Include confidence scores or uncertainty outputs in your inference API responses where feasible
- Build override or feedback endpoints that route edge cases or low-confidence outputs to a human review queue
- Set alert thresholds that escalate to human review rather than acting autonomously
For agentic systems, this typically means a confirmation step before consequential actions, or a human-in-the-loop queue for outputs above a risk threshold.
Beyond logging and oversight mechanisms, Article 9 also requires documented evidence of technical robustness testing against reasonably foreseeable misuse. Teams building the technical robustness layer required by Article 9 can follow the AI red teaming guide for PyRIT, Garak, and Inspect AI to set up a full adversarial testing pipeline on Spheron GPUs.
Transparency for GPAI Models
If you are fine-tuning or deploying a GPAI model above the 10^25 FLOPs threshold, you need to publish a summary of training data and maintain a copyright compliance policy. Below that threshold, GPAI obligations are lighter. For most teams working with open models from Meta, Alibaba, or Google, the base model provider handles the heavy GPAI compliance obligations. Your responsibility covers your fine-tuning data and any modifications you make. Keep your fine-tuning dataset provenance documented: source, license, any filtering or anonymization applied.
Teams handling cross-border training data can satisfy Article 10's data residency requirements via federated learning on GPU cloud, which keeps raw data at each site and shares only model updates across organizational boundaries.
Choosing a GPU Cloud Provider That Meets EU AI Act Standards
Infrastructure selection is a compliance decision. Here are the properties that matter, and what to look for in each.
| Property | Why It Matters for EU AI Act | What to Look For |
|---|---|---|
| EU data center nodes | Data residency for inference traffic and model storage | Provider lists EU-region nodes clearly; you can select them before provisioning |
| Data Processing Agreement (DPA) | GDPR Chapter V transfer compliance | GDPR-compliant DPA available; SCCs for non-EU transfers if needed |
| Root access to instances | Custom logging, audit trail agents, network isolation | Full SSH root; not container-only restrictions |
| Tier 2+ certified facilities | Physical security and uptime requirements | ISO 27001, SOC 2, or Tier 3/4 certification visible |
| Audit trails for provisioning | Who accessed what infrastructure and when | Dashboard logs and API access logs available and exportable |
| No vendor lock-in | Portability if compliance posture changes | Standard protocols (SSH, Docker); portable images with no proprietary runtime |
Hyperscalers have EU regions, but they also have complex data sub-processing chains and DPAs that can be difficult to verify in detail. For some regulated workloads, the opacity of what happens inside a managed container service creates a compliance gap that is hard to close.
For workloads where the threat model includes privileged cloud provider access to GPU memory during inference - a specific concern for high-risk AI systems under the EU AI Act - see our confidential GPU computing guide for NVIDIA CC mode deployment on H100 and B200.
For vendor lock-in implications and what happens when you need to migrate workloads under a compliance deadline, a detailed provider comparison covers those tradeoffs.
For building production-grade reliability on top of GPU marketplace infrastructure, the patterns are the same regardless of provider.
GPU Pricing for EU-Region Deployments
Running inference in EU-region nodes costs the same as any other region on Spheron. Here are current on-demand prices for the GPUs most commonly used in regulated enterprise deployments:
| GPU | On-Demand (per GPU/hr) | Spot (per GPU/hr) | Common Use |
|---|---|---|---|
| H100 SXM5 | $2.90 | $0.80 | Large LLM inference, training |
| A100 80G SXM4 | $1.64 | $0.45 | Mid-scale inference, fine-tuning |
| L40S PCIe | $0.72 | $0.32 | Cost-efficient inference |
Pricing fluctuates based on GPU availability. The prices above are based on 17 Apr 2026 and may have changed. Check current GPU pricing for live rates.
Step-by-Step Compliance Checklist for AI Teams Using Cloud GPUs
This checklist maps to the howToSteps in the structured data above. Run through it before and during deployment for any AI system targeting EU users.
Pre-Deployment
- Classify your AI application against the EU AI Act risk tiers
- Determine whether GDPR applies to your training data and inference inputs
- Select a GPU provider with EU-region nodes and a signed DPA
- Define data retention policies for inference logs
Infrastructure Setup
- Deploy inference server (vLLM, SGLang, or Triton) with request logging enabled
- Route inference logs to a persistent, access-controlled store (S3-compatible in EU region)
- Configure Kubernetes RBAC or equivalent access controls for your GPU deployment
- Set up model version pinning and rollback capability
- Document your GPU instance locations and provider compliance certifications
For High-Risk AI Systems Only
- Write the EU AI technical file (model purpose, performance metrics, known limitations)
- Implement human oversight endpoints in your inference API
- Conduct a conformity assessment (internal or third-party depending on system type)
- Register the system in the EU AI Act regulatory framework operated by the European Commission before deployment
- Establish a post-market monitoring plan and incident reporting process
Ongoing
- Monitor model output quality and flag drift
- Retain audit logs per your risk tier obligation (minimum 6 months per Article 26(6); technical documentation retained for 10 years per Article 18)
- Review compliance status when updating model weights or deployment configuration
- Stay current with enforcement guidance from the European AI Office
How Spheron Supports Compliant AI Deployments
Three things make Spheron a practical fit for teams building EU AI Act-compliant infrastructure.
Geographic node selection. When you provision compute on Spheron, you can filter GPU nodes to those hosted in EU data centers. You see where your compute runs before you rent it. This is not a black-box "EU region" designation where you have to trust that your data stays in-region. You pick nodes with explicit location visibility, which means your compliance documentation can reference actual facility locations and certifications rather than provider promises.
Full root access for custom governance. Unlike container-only platforms, Spheron gives bare metal and VM deployments full SSH root. You install your own audit logging agents, configure network isolation to prevent data egress, and apply security policies that match your organization's requirements. For a direct comparison of what full-VM access means versus container-only restrictions, see Spheron vs Vast.ai. This matters for high-risk AI compliance because the EU AI Act requires you to demonstrate control over your AI system. A platform where the provider controls the runtime layer makes that harder to document.
No vendor lock-in for portability. If your compliance posture requires switching providers or repatriating workloads on-premise, Spheron's deployment model uses standard tooling: Docker, SSH, standard GPU drivers and CUDA toolchain. No proprietary SDKs, no migration barriers. For teams evaluating the long-term on-prem vs cloud tradeoff through a compliance lens, the portability factor matters more than the headline price comparison.
Spheron's infrastructure runs through vetted data center partners across EU regions, with Tier 2/3/4 compliant facilities and support for ISO 27001, SOC 2 Type I/II certifications where required.
Building compliant AI infrastructure is a one-time engineering investment that pays forward as enforcement matures. EU teams that establish audit logging, governance workflows, and compliant data residency now avoid forced rewrites when inspectors arrive. The frameworks are in place; the question is whether your infrastructure makes compliance straightforward or difficult.
EU AI Act compliance starts with controlling where your AI runs. Spheron lets you select GPU nodes in EU data centers, get full root access for custom audit logging, and move workloads without vendor lock-in.
Quick Setup Guide
Map your AI application to one of the four EU AI Act risk categories - unacceptable risk, high-risk, limited risk, or minimal risk - based on the intended use case. Most LLM-based developer tools and internal productivity AI fall into minimal or limited risk.
Identify whether your model weights, training data, or inference inputs contain personal data subject to GDPR. For high-risk systems, document where model artifacts are stored and which data centers process inference traffic.
Choose a provider that can guarantee your GPU nodes are hosted in EU Tier 2+ data centers. Verify that the provider's data processing agreements (DPAs) cover your use case and that you have contractual clarity on data handling.
Deploy an inference logging layer that captures model inputs, outputs, timestamps, and request metadata. For high-risk systems, logs must be retained and made available to regulators. Tools like vLLM's request logging, OpenTelemetry, and Prometheus exporters cover the basics.
High-risk AI systems require a technical file covering intended purpose, performance metrics, training data sources, known limitations, and human oversight mechanisms. Keep this documentation version-controlled alongside your model release process.
If your system qualifies as high-risk, register it in the EU AI Act database operated by the European Commission before deployment. The registration requires your technical file, conformity assessment results, and provider information.
Set up model performance monitoring to detect drift, unexpected outputs, and potential harms. High-risk AI systems must have a post-market monitoring plan and a process for reporting serious incidents to national authorities.
Frequently Asked Questions
The EU AI Act requires companies to classify their AI systems by risk tier, maintain technical documentation, implement logging and audit trails, and for high-risk systems, register in the EU AI database. Infrastructure running in EU data centers simplifies data residency compliance.
Yes. Both training and inference workloads are in scope if the model will be deployed for use in the EU market. Data used for training is subject to GDPR if it contains personal data, and the trained model itself falls under the EU AI Act based on its intended application.
High-risk systems include AI used in critical infrastructure, education, employment decisions, access to essential services, law enforcement, border management, and administration of justice. General-purpose LLMs used internally for productivity or developer tooling typically fall into lower risk tiers.
It depends on your workload. Inference data processed for EU users must comply with GDPR data transfer rules. Model weights for high-risk systems should be stored in auditable locations. Using a provider with EU-region nodes and transparent data handling is the cleanest path to compliance.
Spheron lets operators select GPU nodes hosted in EU data centers, giving explicit control over where compute and data reside. Full root access enables custom logging, audit trail configuration, and network isolation - all requirements for high-risk AI system governance.