AI Infrastructure 101: Semiconductor - The Industry That Powers Everything

From the chip in your phone to the servers running AI — here's how the semiconductor industry actually works.

You've heard that chips are in short supply. That Taiwan matters geopolitically. That NVIDIA is the most valuable company in the world. All of it traces back to the same industry: semiconductors.

This is the explainer for what that industry actually is, how chips are made, who runs the supply chain, and where the next wave of opportunity lies.

What Is a Semiconductor?

A semiconductor is a material — almost always silicon — that conducts electricity only under certain conditions. That "sometimes on, sometimes off" property is exactly what you need to build a transistor: a microscopic switch that flips between 1 and 0.

A modern chip packs billions of these transistors onto a piece of silicon smaller than a fingernail. Arrange them cleverly, and those simple on/off switches can run any computation — from sending a text to training an AI model.

If you tried to count every transistor on a single advanced chip, it would take you over 6,000 years.

Everything digital runs on this: your phone, your laptop, your car's braking system, the data centres running ChatGPT. The semiconductor industry designs, manufactures, and distributes those chips.

By the numbers: global semiconductor sales hit $791 billion in 2025 — up 25% year-on-year — and are projected to cross $1 trillion in 2026, driven almost entirely by AI infrastructure. AI chips alone are estimated to account for roughly 50% of industry revenue in 2026, despite being less than 0.2% of chips sold by volume. The money is concentrated at the very top of the technology stack.

Moore's Law: Why Chips Keep Getting Better

For 50 years, the industry has run on a single observation: the number of transistors on a chip roughly doubles every two years. This is Moore's Law — not a law of physics, but a relentless engineering target the industry has hit decade after decade.

Smaller transistors mean you can fit more of them on a chip. More transistors mean more computing power, lower cost per calculation, and better energy efficiency. This is why your phone today outperforms a supercomputer from the 1990s.

Transistor size is measured in nanometres (nm) — referred to as the "node." The smaller the node, the more advanced the chip:

The catch: each node shrink is exponentially harder and more expensive than the last. We are now approaching the physical limits of how small a transistor can get, which is forcing the industry toward new tricks — new transistor shapes (called gate-all-around or nanosheet designs) and stacking chips in 3D rather than just shrinking them.

The Types of Chips

Not all chips are the same. The industry breaks into four broad categories.

1. Logic chips — the "thinking" chips

These process instructions and run computations. This is the highest-value, most AI-relevant category — and it's where the crucial CPU / GPU / TPU distinction lives:

CPU (Central Processing Unit) — the general-purpose brain. A handful of powerful cores (8–64) optimised for doing complex tasks one after another: running your operating system, handling logic and decisions. Brilliant at sequential work, poor at doing millions of things at once. Made by Intel, AMD, Apple, Qualcomm.

GPU (Graphics Processing Unit) — originally built to render graphics, which means doing the same simple maths across thousands of pixels simultaneously. That parallel design turned out to be perfect for AI, which is essentially millions of matrix multiplications at once. A GPU has thousands of smaller cores built for exactly that. Nvidia dominates; its data-centre GPUs are the gold standard for AI training.

TPU / NPU (Tensor / Neural Processing Unit) — chips built specifically for AI, even more specialised than a GPU. They strip away everything not needed for the matrix maths that neural networks use, making them faster and more power-efficient for AI workloads — but less flexible. Google's TPUs and the AI accelerators inside your phone are examples.

The simple analogy:

• CPU = a few expert consultants who can handle anything, one task at a time.
• GPU = a large team of generalists all doing the same task in parallel.
• TPU = a purpose-built factory line doing one specific job extremely fast.

Logic chips command the most investment by far: logic semiconductors attracted 89% of all venture capital in the sector over the past 12 months, despite being only 67% of deals.

2. Memory chips — the "remembering" chips

DRAM holds data in active use (your computer's RAM). NAND flash stores data long-term (your SSD). HBM (High Bandwidth Memory) is premium, ultra-fast memory stacked right next to AI processors to feed them data quickly enough to keep up. HBM is one of the hottest segments in the entire industry — memory will drive 31% of near-term industry growth, with almost half of that tied to HBM alone.

3. Analogue and power chips

Less glamorous, everywhere. Analogue chips translate real-world signals (sound, light, temperature) into digital data. Power chips regulate electrical current. Every EV, industrial machine, and smartphone depends on them.

4. Sensors and MEMS

Microelectromechanical systems that detect motion, pressure, and light — the chips behind your phone's accelerometer, your car's radar, and medical devices.

How a Chip Is Actually Made

This is the part most explainers skip — and it's the key to understanding why the supply chain is so concentrated.

Step 1 — Design. Engineers design the chip using specialised software (EDA, more on this below). A leading-edge design can cost hundreds of millions before a single chip exists.

Step 2 — The wafer. Chips are built on thin discs of ultra-pure silicon called wafers, usually 300mm across. One wafer yields hundreds of chips.

Step 3 — Lithography. This is the heart of fabrication. Light is projected through a stencil (a "mask") to etch circuit patterns onto the wafer, layer by layer. Advanced chips need EUV (Extreme Ultraviolet) lithography — and only one company in the world, ASML, makes the machines that can do it.

Step 4 — Etching and deposition. The patterns are chemically etched and layered, repeated dozens of times to build a 3D structure of transistors and connections. A single chip can require over 1,000 process steps and take 3–4 months to make.

Step 5 — Packaging. The finished chips are cut from the wafer, tested, and packaged into a usable form. For AI chips, this increasingly means 3D packaging — stacking multiple chips (logic + memory) into a single package, using techniques like TSMC's CoWoS.

Each of these steps is a specialised industry of its own — which is why no single company makes a chip end-to-end.

The Value Chain: Who Does What

Making a chip is a tightly choreographed global supply chain split across distinct layers.

1. EDA Software (Electronic Design Automation) — the tools used to design chips. Synopsys and Cadence hold a near-duopoly; no chip gets made without them. Synopsys recently acquired Ansys to expand into a $31 billion design market.

2. Chip Design (Fabless) — companies that design but don't manufacture: Nvidia, AMD, Apple, Qualcomm. ARM is a special case — it licenses the underlying architecture (the instruction set) that powers ~99% of smartphones and a growing share of data-centre chips.

3. Manufacturing (Foundries) — the factories that physically make chips. TSMC dominates, Samsung is second, Intel is rebuilding. A modern fab costs $20+ billion and takes years to reach full output.

4. Equipment — the machines that make fabrication possible. ASML's near-monopoly on EUV lithography is the single biggest chokepoint in the entire industry.

5. Advanced Packaging — assembling and stacking finished chips. Now a critical bottleneck, since every AI chip needs 3D packaging that even TSMC can't supply fast enough.

Three business models

A quick but important distinction in how companies operate:

• Fabless — designs chips, outsources manufacturing (Nvidia, AMD, Apple)
• Foundry — manufactures chips for others, doesn't design its own (TSMC)
• IDM (Integrated Device Manufacturer) — does both, designs and makes its own chips (Intel, Samsung)

The fabless-foundry split is the defining structure of the modern industry. It let companies like Nvidia focus entirely on design while TSMC focused entirely on manufacturing — and both became dominant by specialising.

The Dominant Players and Why They're Hard to Displace

The common thread: each dominant player controls a chokepoint. TSMC owns the fab. ASML owns the machine that makes the fab possible. Nvidia owns the software that makes the GPU useful. This is an industry that compounds monopolies.

The Startup Landscape

Despite incumbent dominance, there's a genuine wave of startup activity — mostly in AI chips and infrastructure. These are the challengers worth knowing:

AI accelerator designers — building chips specifically for AI, competing with Nvidia on cost and efficiency. Cerebras makes a chip the size of an entire wafer, eliminating the memory bottleneck that slows GPUs. SambaNova builds reconfigurable dataflow chips aimed at large enterprise models. Tenstorrent, led by legendary architect Jim Keller, pairs its chips with an open-source software stack and licenses its designs. Groq pioneered ultra-fast inference chips (and was acquired by Nvidia in a landmark deal). d-Matrix focuses on in-memory compute for inference.

Inference optimisation — making AI models run faster and cheaper at the point of use, through hardware, software, or both. Etched is building a chip hard-wired specifically for transformer models; Fractile (UK) and a wave of inference-software startups like Baseten and Fireworks attack the same cost problem from the software side.

AI-native EDA — embedding AI into chip-design software to compress the time and cost of designing a chip, chipping at the edges of the Synopsys–Cadence duopoly.

Advanced packaging — new entrants targeting the 3D-stacking bottleneck that even TSMC can't fully resolve.

Alternative lithography — Swedish startup AlixLabs is developing an etch-based process that achieves sub-10nm features without extra EUV exposure, potentially reducing dependence on ASML.

Sovereign / regional fabs — government-backed entrants like Japan's Rapidus (targeting 2nm), Korea's Rebellions (AI inference chips), and new EU Chips Act fabs, building domestic capacity for strategic resilience.

Here's the quick-reference roster of who's attacking what:

Bottlenecks and Where the Opportunity Is

The industry is structurally constrained in several ways — each one a startup opportunity:

1. The HBM shortage. AI chips need huge amounts of high-bandwidth memory, and only three companies make it at scale. Suppliers are prioritising HBM for AI servers, tightening conventional memory supply and pushing prices up across the board. Opportunity: new memory architectures, near-memory compute.

2. Advanced packaging. Every frontier AI chip needs CoWoS packaging, and TSMC controls most of the capacity. Leading AI chip designers consumed ~90% of global CoWoS and HBM supply in 2025 while using only 12% of advanced logic production. Opportunity: new packaging capacity, substrates, assembly tech.

3. Power and cooling. AI data centres consume enormous electricity, and cooling is becoming a crisis. Grid constraints may soon be a more severe deployment bottleneck than silicon supply itself. Opportunity: liquid cooling, co-packaged optics, power-delivery chips.

4. Geographic concentration. Critical inputs cluster in a few regions; a single political event in Taiwan disrupts everything. Opportunity: sovereign fabs, supply chain diversification.

5. Chip design cost. Designing a leading-edge chip costs hundreds of millions before manufacturing begins. Opportunity: AI-native EDA tools that make advanced design accessible to more teams.

6. Inference cost. Running AI models in production is expensive, and GPUs are power-hungry. The more AI is deployed, the more acute this becomes. Opportunity: inference-optimised chips, edge hardware, hardware-software co-design.

The big shift: for decades, phones and PCs drove the industry. Now AI and data centres are the growth engine, which is reshaping which chips matter and who profits.

Find my post about Cerebras in here, or SambaNova in here.