15. The provider landscape

The market splits into four groups: hyperscalers with hundreds of services, GPU-focused clouds, serverless and inference platforms, and European providers. The concepts are identical across all of them; only the names differ, which the table below translates.

Hyperscalers

• AWS - the largest provider with the broadest catalog and the most job postings. Its own AI chips are Trainium and Inferentia. Bedrock offers model APIs from several vendors, including Anthropic; SageMaker AI covers training and deployment.
• Google Cloud - strongest in data and analytics (BigQuery), the only home of TPUs, and GKE is widely regarded as the most polished managed Kubernetes. Vertex AI offers Gemini plus partner models.
• Microsoft Azure - deep enterprise integration through Entra ID and Microsoft 365, and the main cloud for OpenAI models, alongside a growing catalog of others. Very common in large German companies.
• Oracle Cloud - smaller overall but a major seller of very large GPU clusters.

GPU clouds, marketplaces and inference platforms

• GPU clouds (“neoclouds”) such as CoreWeave, Lambda, Nebius and Crusoe focus on GPUs. They often price lower, hand you Kubernetes or Slurm clusters with fast interconnects, and offer far fewer surrounding services.
• Marketplaces such as RunPod and Vast.ai rent individual GPUs, some hosted by third parties. They are the cheapest option, with variable reliability and security, so keep sensitive or client data off community-hosted machines.
• Serverless GPU and inference platforms such as Modal, Baseten, Together AI and Fireworks AI run your code or open models on demand and bill by the second or by the token.

European options

• Hetzner (Germany) - very cheap CPU servers and some dedicated GPU servers. Excellent for learning and side projects.
• STACKIT (Schwarz Group, Germany) and IONOS (Germany) - sovereignty-focused clouds aimed at German businesses and the public sector.
• OVHcloud and Scaleway (France) - broad clouds with GPU offerings.
• Open Telekom Cloud (T-Systems) and Nebius (headquartered in Amsterdam, with European GPU data centers).

The trade-off: easier GDPR and sovereignty stories and often lower prices, in exchange for fewer managed services and a smaller ecosystem.

Translation table

Concept	AWS	Google Cloud	Azure
Isolation boundary	Account (in an Organization)	Project (in folders and an organization)	Subscription (in management groups)
Virtual machines	EC2	Compute Engine	Virtual Machines
Object storage	S3	Cloud Storage	Blob Storage
Block storage	EBS	Persistent Disk / Hyperdisk	Managed Disks
Shared or parallel file storage	EFS / FSx for Lustre	Filestore / Managed Lustre	Azure Files / Managed Lustre
Private network	VPC	VPC	Virtual Network (VNet)
Firewall rules	Security groups	VPC firewall rules	Network security groups
Load balancing	Elastic Load Balancing (ALB, NLB)	Cloud Load Balancing	Load Balancer / Application Gateway
DNS	Route 53	Cloud DNS	Azure DNS
CDN	CloudFront	Cloud CDN	Front Door
Identity and access	IAM, IAM Identity Center	Cloud IAM	Entra ID with Azure RBAC
Secrets and keys	Secrets Manager, KMS	Secret Manager, Cloud KMS	Key Vault
Managed Kubernetes	EKS	GKE	AKS
Container registry	ECR	Artifact Registry	Azure Container Registry
Serverless functions	Lambda	Cloud Run functions	Azure Functions
Serverless containers	Fargate (with ECS or EKS)	Cloud Run	Container Apps
Queues and messaging	SQS, SNS	Pub/Sub	Service Bus, Event Grid
Workflows	Step Functions	Workflows	Logic Apps, Durable Functions
Relational database	RDS, Aurora	Cloud SQL, AlloyDB	Azure Database for PostgreSQL
Key-value / document database	DynamoDB	Firestore, Bigtable	Cosmos DB
Data warehouse	Redshift	BigQuery	Fabric, Synapse
ML platform	SageMaker AI	Vertex AI	Azure Machine Learning
Model API platform	Bedrock	Vertex AI (Model Garden)	Azure AI Foundry
Custom AI chips	Trainium, Inferentia	TPU	Maia (mostly internal)
Monitoring and logs	CloudWatch	Cloud Monitoring, Cloud Logging	Azure Monitor
Audit log	CloudTrail	Cloud Audit Logs	Activity Log
Native infrastructure as code	CloudFormation, CDK	Infrastructure Manager	Bicep, ARM templates

Which to learn first

Learn the concepts once and one provider deeply; the second provider then takes weeks, not months. AWS appears most often in job postings overall, Azure is strong in large German enterprises, and Google Cloud is popular with data- and ML-heavy teams. For AI infrastructure roles specifically, the strongest combination is Kubernetes, one hyperscaler, and hands-on experience with a GPU cloud. Check the postings for the roles you want and let them decide.

Check yourself

• Translate this sentence into Google Cloud terms: “We run vLLM on EKS with Karpenter, weights in S3, secrets in Secrets Manager.”
• When would you choose a GPU cloud over a hyperscaler for training? And for serving?
• What is the risk of processing a client’s data on a community-hosted marketplace GPU?