Shark Tank's Kevin O'Leary says if he were 25 today, he’d chase these two booming opportunities in the world of AI

When Shark Tank's Kevin O'Leary says he'd chase "two booming opportunities" in AI if he were 25 today, most people expect him to name the next viral chatbot or a flashy generative AI startup. But O'Leary, a venture capitalist who has seen multiple tech cycles, took a dramatically different stance: young entrepreneurs shouldn't build the AI apps that grab headlines-they should build the backbone that makes those apps possible. The real billions are buried in the infrastructure that most AI founders ignore.

This contrarian wisdom comes at a pivotal moment. More than 90% of AI startups today are Building on top of foundation models like GPT-4, Claude. Or Llama. They compete on prompt engineering, fine-tuning. And UI polish-increasingly commoditized layers that offer diminishing returns. Meanwhile, the companies that sell the shovels in this gold rush-data center operators, chip designers, networking engineers. And MLOps tooling-are quietly capturing the majority of economic value. O'Leary's advice, if he were 25, would be to go deep into the hardware and software that powers artificial intelligence at scale.

This article isn't a rehash of a TV soundbite. It's a technical and strategic deep dive for engineers and founders who want to understand where the high-use opportunities truly lie. We'll dissect the two specific opportunities O'Leary alludes to, back them with production-level engineering realities. And map a path for a 25-year-old to build a career-or a company-that rides the infrastructure wave, not the application hype cycle.

Understanding O'Leary's Infrastructure-First Mindset

Kevin O'Leary's portfolio history reveals a preference for businesses with hard assets, recurring revenue, and defensible moats. Software AI applications often lack all three: they rely on rented API calls, can be copied in a weekend. And offer zero differentiation. O'Leary's framing suggests two categories: physical infrastructure (data centers, energy, cooling) system software infrastructure (orchestration, model serving, monitoring). Both require deep engineering and capital-exactly the barriers that keep most 25-year-olds from competing but also create durable value.

Consider the sheer scale: training a single frontier model like GPT-4 required roughly 25,000 NVIDIA A100 GPUs running for months, consuming an estimated 50 GWh of electricity. The cost of the data center buildout alone exceeds $1 billion. Every dollar spent on GPUs needs another dollar for networking, storage, cooling, and power. The margins on those components are thin. But the volumes are staggering-and growing at 70% CAGR. O'Leary recognizes that young founders can't build the next H100 chip. But they can build the software and services that improve every watt and every dollar of that infrastructure.

This echoes an earlier wave: during the 1999 dot-com boom, the fortunes were made by Cisco (routers), Oracle (databases), and EMC (storage), not by the myriad Pets com failures. AI is no different. The "backbone" includes everything from liquid cooling systems to scheduler algorithms for GPU clusters. Internal link: compare AI infrastructure to cloud infrastructure in 2008.

Opportunity One: AI Hardware and the Data Center Supply Chain

The first booming opportunity O'Leary identifies is the physical infrastructure that hosts AI compute. This isn't just about building data centers-it's about rethinking power delivery, cooling, interconnect, and security for AI workloads. A hyperscale data center today needs up to 150 MW of power, more than many small towns. getting that power permitted and delivered is a multi-year ordeal. Young entrepreneurs can focus on solving this bottleneck: modular substations, grid-scale batteries, or software that optimizes power usage across clusters.

One real-world example is the explosion of "colocation" startups that lease AI-ready rack space. Companies like CoreWeave (which started as a crypto miner) pivoted to GPU hosting and became a billion-dollar business. CoreWeave's secret wasn't a better GPU-it was better density: they pack more GPUs per rack using custom liquid cooling and high-speed InfiniBand networking. A 25-year-old could start a niche consulting firm that helps mid-sized companies deploy private AI clusters, leveraging open-source tools like Kubernetes for GPU scheduling and NVIDIA NCCL for multi-GPU communication.

On the software side, the opportunity is in orchestration and observability. Managing a fleet of 1,000 GPUs is harder than managing 1,000 CPUs because of topology constraints (which GPUs are physically close to reduce latency) - thermal throttling. And failure modes unique to AI hardware. Startups like Run:ai (acquired by NVIDIA) built a Kubernetes scheduler that optimizes GPU utilization from 40% to 90%-a massive cost saving. A 25-year-old with deep knowledge of Linux, GPU drivers, and scheduling algorithms can build the next generation of cluster management tools. Internal link: building a GPU-aware scheduler.

Opportunity Two: MLOps and the Software Backbone of Model Lifecycle

The second opportunity is less about clicks and mortar and more about the software that makes AI production-ready? O'Leary has explicitly mentioned that building yet another ChatGPT wrapper is a dead end. Instead, young engineers should focus on the pipeline between raw data and deployed model: data versioning, training orchestration - model evaluation, and inference serving at scale. This is the MLOps (Machine Learning Operations) stack, and it's fragmented, painful. And poorly documented-perfect for a hungry engineer,

Consider the problem of model servingWhen you deploy a 70-billion-parameter model, you can't just throw it on a single GPU. You need to shard the model across multiple GPUs, handle batching, manage memory, and achieve sub-100-millisecond latency. Tools like vLLM, TensorRT-LLM. And TGI are open-source but require deep surgery to work in a custom enterprise environment. Startups that offer managed inference services-or embeddable SDKs for edge devices-are growing fast. A 25-year-old can fork vLLM, add support for a rare architecture. And sell support contracts to regulated industries.

Another gap is evaluation and observability. How do you know your LLM isn't hallucinating in production? How do you track drift across thousands of prompts? The current state of AI evaluation is primitive-most teams rely on human raters or automated metrics that correlate poorly with real-world performance. Building a robust evaluation framework, similar to how APM tools like Datadog work for web apps, is a massive opportunity. The DeepEval open-source project is a start. But enterprise-grade solutions with custom guardrails are years away.

O'Leary's insight is that these infrastructure problems are boring but sticky. Once a company integrates your data versioning tool or monitoring dashboard, switching costs are high. Compare that to a chatbot startup that can be replaced by a single API key change. Young entrepreneurs who build for the backbone are building moats, not traps.

Why Most AI Startups Are Building on Quicksand

O'Leary's disdain for flashy AI apps is data-driven. According to CB Insights, over 70% of AI startups fail within the first two years. And the majority are "wrapper" apps that provide no underlying technology. The collapse of Jasper AI (once valued at $1. 5B) and the struggles of companies like Copy ai demonstrate the fragility of being a thin layer over a foundation model. When OpenAI drops prices by 50% (as it did in 2023), wrappers lose their already thin margins.

Furthermore, the regulatory environment is shifting. The EU AI Act and looming US regulations may require auditing of training data, bias testing. And explainability-costs that infrastructure providers can absorb but that eat into application margins. No matter what the law says, every AI deployment will need monitoring, version control. And compliance logs. Those tools are infrastructure. And they will be mandatory purchases for large enterprises.

A 25-year-old should ask: Do I want to compete on marketing spend (apps) or on engineering excellence (infrastructure)? The latter has a longer sales cycle but vastly higher retention and lifetime value.

Concrete Skills a 25-Year-Old Needs Today

Pivoting from O'Leary's advice to actionable career steps, here are the skills that build the backbone:

• Distributed Systems & Kubernetes: AI training is a distributed computing problem. Understanding cluster scheduling, Pod networking. And persistent volume claims for GPUs is non-negotiable. Contribute to the Kubernetes SIG-Network for real-world experience
• GPU Programming (CUDA, ROCm): You don't need to design chips. But you must understand memory hierarchy, tensor cores. And kernel launch overhead. Optimizing a naive attention mechanism in Triton vs. CuDNN can make a 10x difference in inference cost.
• High-Performance Networking: InfiniBand and RoCE aren't taught in CS curricula. Learning RDMA, congestion control, and topology-aware job scheduling will set you apart.
• Observability and Monitoring: Prometheus, Grafana. And custom exporters for GPU metrics (temperature, power, utilization) are bread and butter. Build a dashboard that caught a pod-level memory leak, not a business KPI chart.

These skills are transferable to hyperscalers (AWS, Azure, GCP) or to startups building vertical infrastructure solutions.

The Economics of AI Infrastructure: Numbers That Matter

To appreciate O'Leary's point, look at the numbers. NVIDIA's data center revenue in fiscal 2024 was $47, and 5 billion, up 217% year-over-yearThe entire "enterprise AI application" market was maybe $10 billion. Infrastructure captures the majority of spend, while even within infrastructure, the margins vary: GPU hardware is capital intensive and low margin for resellers. But the software tooling (like Weights & Biases, $1. 25B valuation) enjoys gross margins above 80%. The sweet spot is the vertical software layer that sits between the raw hardware and the end user-MLflow, DVC, Kubeflow. And newer entrants like Baseten or Modal.

A young engineer could build a specialized deployment platform for small models (e g., 7B-13B parameters) on consumer-grade GPUs with quantized weights. That pricing would undercut cloud providers by 5x, eat into the long tail of inference. And become a necessary tool for startups that can't afford $10/hour H100s. Internal link: building a low-cost inference server with llama. And cpp

Power is another overlooked lever. Data centers now consume 2% of global electricity, projected to hit 8% by 2030. Software that matches AI jobs to times of low carbon intensity or renewable availability can save millions in energy costs. Some startups are building "carbon-aware schedulers" as Kubernetes extensions-a perfect intersection of environmental impact and engineering challenge.

Risks and Why Infrastructure isn't a Guaranteed Win

O'Leary's advice isn't without caveats. Building infrastructure requires capital, patience. And deep domain expertise that a 25-year-old may lack. The data center market is dominated by deep-pocketed incumbents like Equinix - Digital Realty. And hyperscalers, and competing on physical real estate is unrealisticHowever, the software layer around that infrastructure is still up for grabs. The real risk is building a tool that becomes obsolete when a hyperscaler adds the same capability for free (e g. And, AWS SageMaker adding experiment tracking)

Another risk: AI model architectures are evolving rapidly. The current dominance of transformer-based models could be supplanted by state-space models (like Mamba) or hybrid architectures. Infrastructure built for dense attention may not transfer. The lesson is to build for abstractions, not APIs: focus on orchestration, data pipelines. And telemetry that work with any model architecture. Stay architecture-agnostic.

Finally, the engineering talent shortage in infrastructure is real. Many developers prefer building user-facing features over debugging kernel panics. That scarcity is good for those who explore-higher pay, less competition, more impact, and o'Leary's "two opportunities" aren't for everyone,But for the engineer who enjoys the gritty, unglamorous work of making AI cheaper and faster, the runway is incredibly long.

How to Start Tomorrow Without 20 Million Dollars

You don't need a billion-dollar data center to enter this space. Start with open-source infrastructure projects. Clone the vLLM repository and improve its PagedAttention for a specific AMD GPU. Write a Helm chart that deploys a production-grade LLM serving stack on a single node Kubernetes cluster. If you can reduce inference latency by 20% for a popular model, you'll attract attention from the open-source community and eventually from hyperscalers hiring.

Another path: contribute to the CNCF AI/ML Working Group