Why Python remains the AI industry’s biggest performance bottleneck

This article is about the strangest trade-off in modern engineering: the industry that computes faster than anything in human history conducts those computations in one of the slowest languages in the history of programming. And about what happens once we stop treating that as a law of nature.

1. Introduction: the oddity everyone got used to

Let us begin with a scene that plays out every day in thousands of companies.

A data centre. Racks of accelerators, each performing hundreds of trillions of operations per second. Electrical power on the scale of a small city: in 2024 the world’s data centres consumed about 415 TWh, and by 2030 the International Energy Agency expects roughly 945 TWh — more than all of Japan consumes today, with AI as the main driver of that growth [48]. The engineers running all of this are, quite possibly, the highest-paid programmers on the planet.

Now let us peek at their code. It is written in a language that placed 26th out of 27 in the classic study of programming-language energy efficiency [14]. A language that, until October 2025, was fundamentally incapable of executing bytecode on two threads at once [20]. A language whose own creator candidly described it as “glue” for other people’s fast components [1].

⚠️ A common misconception. “If all of AI is written in Python, then Python must be technically the best fit for AI.” It is not — and we will show why. Python fits AI historically and socially: it had a ready-made scientific ecosystem and a low barrier to entry. Technically, the whole industry is built like a matryoshka doll: Python on the outside, C++ and CUDA within. The question this article asks is what we pay for that doll — and what happens if we open it.

Let us agree on honesty right away — it matters more than anything else. We will not claim that Python is bad: it is an outstanding language whose victory was earned. We will not propose “rewriting everything in Swift by Friday”: that is not how the world works, and such attempts have already failed — we will dissect exactly how.

We will argue three things, each of which rests on verifiable numbers:

1. Python’s dominance is an entrenched monopoly of habit, not a law of nature. The king reigns deservedly, but the crown rests on network effects, not physics.
2. The “Python tax” is real, measurable, and grows with the scale of the industry. It is paid in microseconds of dispatch, gigawatts of electricity, and doubled engineering labour.
3. On AI’s new territories — local, mobile, energy-efficient — the choice is already being made, and not in Python’s favour. Swift is one of the most underrated candidates for the language that closes the gap.

2. How Python became king (and why it deserved it)

Let us give the king his due — without that, calling him overrated would be a cheap shot.

Python’s victory in machine learning was not an accident. It was planned — by the language’s creator himself. Back in January 1998, in a position paper titled “Glue It All Together With Python,” Guido van Rossum described the development model we now know as NumPy and PyTorch: let C/C++ programmers write the efficient computational components, and let scientists “glue” them together with Python scripts [1]. This was not a post-hoc excuse for slowness — it was a strategy, and it worked brilliantly.

By the time the deep-learning boom struck, Python had spent a decade and a half growing a scientific stack: Numeric — 1995, SciPy and IPython — 2001, NumPy — 2005–2006, pandas — 2008, scikit-learn — first release in 2010 [2]. When Google open-sourced TensorFlow in 2015 and Facebook released PyTorch in 2016, the choice of frontend language was obvious: the scientists already lived in Python. From there the network effect closed on itself: NumPy ended up in the software stack behind the discovery of gravitational waves and the first image of a black hole [3], every textbook and every course was written in Python, and every new engineer learned it first.

The numbers of dominance are impressive:

Arguing against this is pointless — and we won’t. We merely note two cracks in the marble. First: the peak, it seems, has passed — from the record 26.98% in July 2025, the TIOBE index slid to ~19% by mid-2026 (Python is still #1, but the lead is melting) [6], and in Octoverse 2025 TypeScript took first place by contributor count [5]. The second crack is more interesting — the next section is devoted to it.

📜 A bit of history: “the second-best language for everything.” The Python community has long circulated a self-deprecating formula attributed to Van Lindberg: “Python is the second-best language for everything.” It is a compliment: universality beats specialisation, because the cost of switching between niches is zero. But the compliment has a flip side that is voiced far less often: if for every task there exists a better language, then as the price of the task grows — in money, watts, and milliseconds — sooner or later a moment arrives when “second-best” stops being good enough. The AI industry, as we are about to see, has reached that moment.

3. The king’s secret: Python in AI isn’t Python

Brace yourself — this is going to be disappointingly simple.

Open the repository of PyTorch — the primary tool of modern AI — and look at the official language statistics. Python is only 64% of the codebase. The rest is C++ (28.4%), CUDA (2.6%), C, Objective-C++, Metal, assembly [9]. The entire performant core of eager mode — the ATen tensor library, the CUDA kernels — is written in C++. NumPy tells the same story: 61% Python, 33% C. In other words, the “Python libraries for machine learning” are libraries written in C++, with Python as the polite receptionist at the front desk.

🧪 Try it yourself (right now): open github.com/pytorch/pytorch and click the coloured language bar under the file list. Then do the same with ollama/ollama (Go) and ggml-org/llama.cpp (C/C++). Three of the most popular tools of the AI era — and in none of them is the performant part written in Python. This takes two minutes and replaces ten paragraphs of our assurances.

This arrangement of the world has a name — the two-language problem. It was canonically formulated by the creators of Julia: first in their 2012 manifesto (“We want the speed of C with the dynamism of Ruby” [10]), then in their programmatic SIAM Review paper, where among the “laws of nature” they refused to accept was this one: “One must prototype in one language and then rewrite in another language for speed or deployment” [11].

Here is what that “law” looks like in 2026:

• A researcher writes a model in Python.
• The speed comes from C++/CUDA kernels written by other people in another language.
• For production, the model is converted into a third environment — a C++ runtime, GGUF, Core ML.
• When something is slow at the seams (and it regularly is — see Section 5), only an engineer fluent in both worlds can fix it.

📐 Analogy. Python in AI is the conductor of an orchestra. The conductor plays no instrument, and that is fine: his job is the baton; the orchestra (C++ and CUDA) does the playing. The analogy is accurate in another way that is mentioned less often: if the conductor waves the baton ten times slower than the orchestra can play, the orchestra starts waiting for the baton. Very expensive musicians sit and stare at the conductor. Below we will show this is not a metaphor but a profiler trace: unoptimised LLM engines were spending up to half their time on the “baton waving” [30].

It is important to understand: this construction is neither a conspiracy nor stupidity. It is a rational trade-off of its era. But it has a price, and the price can be measured.

4. The price of habit: measuring the tax

4.1. What the interpreter costs

The most-cited measurement is the study by Pereira et al., “Energy Efficiency across Programming Languages” (SLE 2017): 27 languages, 10 tasks from the Computer Language Benchmarks Game, with energy, time, and memory measured [14]. Normalising everything to C (1.00), the authors produced a table worth hanging on the wall of anyone who utters the words “AI energy crisis”:

Python ranks 26th out of 27 on energy (thanks to Perl for bringing up the rear). Across the study, compiled languages spent on average 120 J per task, interpreted ones — 2,365 J: a gap of roughly 20× [14]. The extended journal version of 2021 confirmed the findings [15].

The gap between Python and Swift — the subject of this article — is ~27× on energy and ~17× on time.

Fig. 1. Normalised energy consumption of programming languages (C = 1.00), logarithmic scale. Data: Pereira et al., SLE’17, Table 4 [14].

⚠️ Two honest caveats (as always). First, the measurements are from 2017: Swift was version 3.x back then, and Python has sped up since (more on that below — spoiler: not by 70×). Second, a recent paper by van Kempen et al. (ASE 2025) found methodological flaws in Pereira et al. and showed that a language’s energy consumption is almost entirely explained by its execution time — “energy-efficient” languages do not exist separately from “fast” ones [16]. Note that this correction does not rescue Python. It merely collapses two of its failures into one — time. On current Benchmarks Game measurements, the best Python program solves n-body in ~6 minutes versus 2.10 s for C and 5.45 s for Swift (~170× and ~66×); on spectral-norm — 90.4 s versus 0.40 s for C and 1.45 s for Swift (~226× and ~62×) [17, 18]. Yes, the fastest C versions there use hand-written SIMD intrinsics — but the Swift versions get by without assembly-level magic.

Fig. 2. Wall time of the best programs on the n-body and spectral-norm tasks (Computer Language Benchmarks Game, mid-2026), logarithmic scale [17, 18].

4.2. The GIL: three decades of solitude (population: one thread)

The second line item of the tax is parallelism. The GIL (Global Interpreter Lock), introduced in 1992, allows only one thread at a time to execute Python bytecode. For over thirty years, the language of the world’s foremost parallel-computing industry could not do parallelism across CPU cores.

In fairness: the ice has begun to move. Python 3.13 (October 2024) introduced an experimental GIL-free build — at the cost of ~40% slowdown of single-threaded code [19]. In Python 3.14 (October 2025) it became officially supported, with the penalty down to 5–10% [20]. But per the PEP 703 plan this is only phase 2 of 3: free-threading has still not become the default build, and the world still downloads Python with the GIL.

4.3. “We’ll make CPython five times faster”

Perhaps we should just wait — and Python will become fast on its own? The industry ran that experiment on itself, on Microsoft’s budget.

In 2021, at Guido van Rossum’s initiative, the Faster CPython team assembled at Microsoft; the “Shannon Plan” promised roughly a 5× speedup over four releases. The results: Python 3.11 delivered +25% on average, 3.12 — ~5%, and the new tail-call interpreter in 3.14 — another 3–5%. Team member Ken Jin summed it up: Python 3.14 is “roughly 20–40% faster” than 3.10 — that is 1.2–1.4× instead of five [21]. And in May 2025 Microsoft laid off most of the team, including tech lead Mark Shannon, and the project moved to “community stewardship” [21].

💡 Key idea of this section. A fortyfold gap does not get closed by heroic twenty-per-cent gains. This is no reproach to the CPython engineers — they did brilliant work. It is a property of the problem itself: a dynamic interpreted language pays by construction, on every single operation, for type checks, attribute lookups, and dispatch that a compiled language pays for once — at compile time. An empty function call in CPython costs ~37 ns; in a compiled language it either costs single nanoseconds or inlines away to zero.

5. “But the GPU does all the math!” — when that’s true, and when it isn’t

This is the strongest counterargument, and it must be addressed without straw men. It goes like this: Python is merely the orchestrator; 99% of the time is spent in CUDA kernels, so you could write the rest in BASIC for all anyone cares.

When the model is big and the batch is fat — this is the plain truth. Asynchronous dispatch hides Python’s overhead: while the GPU spends a minute grinding matrices, Python has ample time to leisurely prepare the next command. The JAX documentation describes this mechanism explicitly — and Patrick Kidger puts the upshot neatly: JAX uses Python as a metalanguage in which you write a program for the real executor — the XLA compiler [12, 13].

The problem is that the boundary of “when this is true” is shrinking fast. Here are the measurements:

• Launching a single CUDA kernel costs microseconds of host time: in NVIDIA’s classic example, a kernel executes in 2.9 µs, but with launch and synchronisation it comes to 9.6 µs — the overhead exceeds the useful work [24]. GPUs have become ~15× faster at compute in six years [22] — while CPUs and Python stayed put. The gap widens with every hardware generation.
• PyTorch eager on small tensors manages only ~280 thousand operations per second — roughly a hundred times fewer than the same loop in plain Python without tensors: nearly everything is eaten by framework dispatch [25].
• Data. An analysis of millions of ML jobs in Google’s infrastructure (VLDB 2021): 30% of all compute time goes to the input pipeline — reading, decoding, augmentations — and 20% of jobs spend more than a third of their time on it [26]. The FFCV project showed that with a standard Python data loader, the “idealised” training time is almost 30× smaller than the time needed merely to process the images [27].
• LLM serving. The very composition of the engines is telling: in the original vLLM paper, the authors describe the system as “8.5K lines of Python and 2K lines of C++/CUDA,” with the 2–4× throughput win delivered precisely by custom CUDA kernels and memory management [28]. The SGLang authors write that an unoptimised inference engine can spend up to half its time on CPU overhead — batch scheduling, allocations, prefix matching [30]. The vLLM team, rewriting the architecture for V1, stated the goal outright — near-zero CPU overhead: a Llama-8B generation step on an H100 takes ~5 ms, and the Python subsystems simply cannot keep up with the GPU; the result — up to 1.7× higher throughput [29].

But the most eloquent admission came from the creators of PyTorch themselves. The PyTorch 2.0 announcement opens with the words “From day one, we knew the performance limits of eager execution” — and explains that substantial parts of PyTorch had to be moved into C++, taking it further from “hackability” [22]. The answer was torch.compile: intercepting Python bytecode and compiling it to Triton and C++ — a geomean speedup of 2.27× on inference and 1.41× on training on an A100 across 180+ real-world models (ASPLOS 2024) [23].

💡 Key idea of this section. Notice the pattern: every time the “Python stack” gets faster, the speedup consists of removing Python from somewhere. From the math kernels (C++/CUDA), from data loading (tf.data, DALI, FFCV), from the serving scheduler (vLLM V1, SGLang), from the very execution of the model (torch.compile turns your Python into not-Python). The syntax stays — the execution semantics leave. The king reigns, but no longer governs.

6. The industry is already voting with its feet

Now let us look not at benchmarks but at behaviour — at what the industry actually uses when performance goes from “nice to have” to “mandatory.”

llama.cpp. In March 2023, Georgi Gerganov published an LLM inference engine in pure C/C++ — with no Python in the runtime at all. By July 2026 it had 119K GitHub stars and the status of the foundation of the entire local-AI ecosystem: it powers Ollama (175K stars — more than PyTorch itself; Ollama is written in Go), LM Studio, and dozens of other tools, while the GGUF format became the industry standard — the Hugging Face Hub alone hosts about 45K GGUF checkpoints [31, 32]. In February 2026, Hugging Face — the heart of the Python ML ecosystem — brought Gerganov’s team onboard, calling llama.cpp “the fundamental building block for local inference” [31].

Rust inside the Python world. That same Hugging Face is systematically rewriting its own infrastructure in Rust: the tokenizers library (a gigabyte of text in under 20 seconds [33]), the safetensors format, the candle framework. The Python user never sees any of this — and that is precisely the point: to keep Python fast, its internals are made non-Python.

Apple MLX. Apple built its ML framework for Apple Silicon with a C++ core and APIs in four languages, with Swift as a first-class citizen. The wording of Apple’s engineers in the MLX Swift announcement deserves a quotation: Swift “combines the ease of use and high-level syntax of a language like Python with the speed of a compiled language like C++” [43].

Edge. ExecuTorch 1.0 — PyTorch’s runtime for devices — is a “lightweight C++ runtime” with a 50 KB base footprint, serving, by the project’s account, billions of Meta users [34]. Python does not live on a microcontroller — and never has.

Mojo. Chris Lattner — the creator of LLVM and Swift — considers the problem real enough to build an entire language around it: Python syntax atop an MLIR-grade compiler, with demonstrations of a Mandelbrot speedup of up to 68,847× over CPython (after typing, vectorisation, and parallelisation across 88 cores — a sporting result, but a telling one) [35]. By May 2026 Mojo reached its 1.0 beta; the compiler remains closed for now [36].

In fairness: Swift is neither alone in this movement nor first. The alternatives hold real trump cards. Rust is the champion of Pereira’s table among languages with safety guarantees (1.03× versus C [14]): memory without a garbage collector and without segfaults — exactly what infrastructure needs; it is no accident that Hugging Face builds tokenizers, safetensors, and candle on it, while the Burn framework proves that a complete ML stack in Rust is possible. JAX is the most elegant realisation of the “Python as metalanguage” idea: the composition of jit/grad/vmap atop the XLA compiler gives research code brevity and speed at once [13]. Mojo strikes at the very heart of the problem — a compiled language for Pythonistas with no retraining — and behind it stands a man who has already built industrial compiler ecosystems twice [35].

So why does this article bet on Swift? Because of a combination the others do not yet have. Rust is brilliant, but the borrow checker and the learning curve are the opposite of “reads like pseudocode”; JAX remains a Python frontend, with all the deployment consequences; Mojo is still a beta with a closed compiler [36]. Swift alone, today, combines a mature industrial toolchain, almost-Python syntax, compiled-language speed, and the systemic on-device niche with an entire platform behind it. For the article’s main thesis, though, this rivalry is secondary: a win for any of the four is a defeat for the two-language problem.

📐 Analogy. Picture a city with one official language, where all the engine rooms — the power plants, the waterworks, the metro — de facto speak another. That is exactly what the AI stack looks like in 2026: Python upstairs; C++, CUDA, Rust, Go, and Swift in the engine room. The debate over “whether AI needs a compiled language” has been settled by practice: it is already there. The open question reads differently — must a human write in two languages, or can one language walk the whole way from the idea to the engine room?

7. The Swift case: the speed of C, with almost-Python syntax

And so we arrive at the second half of the thesis. Why Swift, specifically? Let us go point by point — with numbers, and where there are no numbers, we will say so honestly.

7.1. Speed and energy

In Pereira’s table, Swift stands at 2.79× on energy and 4.20× on time versus C [14]; recall that Python stands at 75.88× and 71.90×. On the current numeric tasks of the Benchmarks Game, the best Swift programs run close to C (n-body: 5.45 s versus 2.10 s; on fannkuch-redux — near parity), sagging on string-heavy tasks [17, 18]. Swift compiles through LLVM to machine code; there is no interpreter here and no JIT warm-up.

Memory is managed via ARC (automatic reference counting) — reference counting resolved at compile time instead of a tracing garbage collector. This is not only about speed but about predictability: there are no GC pauses. How much that matters in production, Apple demonstrated on its own Password Monitoring service, rewritten from Java to server-side Swift: +40% throughput, sub-millisecond latency for 99.9% of requests at billions of requests per day, per-instance memory down from tens of gigabytes to hundreds of megabytes, freeing about half of the Kubernetes capacity [42].

And where Python suffered under the GIL, Swift 6 went in the opposite direction: a strict-concurrency mode in which data races are diagnosed at compile time as errors [44]. Consider the contrast: one language spent three decades serialising threads behind a global lock; the other statically proves your concurrent code correct before the first run.

7.2. Syntax: spot the difference

“Compiled means verbose and scary”? Here is the attention mechanism — the heart of the transformer — in both languages:

# Python (NumPy style)
def attention(q, k, v):
    scores = q @ k.T / math.sqrt(k.shape[-1])
    return softmax(scores) @ v

// Swift (MLX)
func attention(_ q: MLXArray, _ k: MLXArray, _ v: MLXArray) -> MLXArray {
    let scores = q.matmul(k.T) / sqrt(Float(k.dim(-1)))
    return softmax(scores, axis: -1).matmul(v)
}

The difference is a typed signature. But that is not a tax — it is a gift: the compiler now knows these are tensors and will not let you accidentally pass a token index where a probability was expected. Types are comments that never go stale and are checked by a machine.

🧪 Try it yourself. If you have a Mac, the Swift compiler is already installed, out of the box. Save a couple of lines into hello.swift, run swift hello.swift — and you are already writing in a compiled language with the REPL-like ease of a script. Ready-made, line-by-line-annotated implementations of language models in Swift — from n-grams to a mini-GPT on MLX — live in the open repository github.com/asaptf/swift-language-models: clone and run.

7.3. The lesson of Swift for TensorFlow: obituary or prologue?

Here the most inconvenient fact for our thesis must be spoken, and we will volunteer it ourselves: Google already tried to make Swift the language of machine learning — and failed.

📜 A bit of history. The Swift for TensorFlow project was announced in March 2018; it was led by Chris Lattner, the creator of Swift and LLVM. The idea was radical: build differentiable programming — automatic gradient computation — directly into the compiler of a general-purpose language [38]. Jeremy Howard of fast.ai called it “the first serious effort I’ve seen to incorporate differentiable programming deep in to the heart of a widely used language that is designed from the ground up for performance,” and moved part of his famous course onto Swift [37]. Lattner left Google in January 2020; in February 2021 the project was put into archive mode [39]. The cause of death was not the technology but the ecosystem gap: not even Google could rewrite the NumPy universe from scratch.

So — obituary or prologue? Three circumstances argue for the latter.

First, the legacy of S4TF did not die: automatic-differentiation support remained in the mainline Swift compiler as an experimental feature, and industry keeps developing it. PassiveLogic builds autonomous building-control systems on differentiable Swift and publishes measurements: a physical heat-transfer simulation on a CPU — ~0.03 ms per forward+backward cycle versus 8.16 ms for PyTorch (238×) and 11.0 ms for TensorFlow (322×) [40]; in 2024 the same company claimed an energy-efficiency record for autodiff — 34 J/GOps against tens of thousands for the big frameworks [41]. The honesty caveat: this is a CPU-class workload with small, irregular computations — that is, precisely the territory where Python dispatch is murderous — and the measurements are the company’s own.

Second, the context has changed. In 2018, Swift in ML was a stranger with no hardware and no niche. In 2026 it has both: the entire Apple Silicon line with unified memory, MLX with a full-fledged Swift API [43], swift-transformers 1.0 from Hugging Face [47], and the Foundation Models framework — the system Swift API to the on-device LLM introduced at WWDC 2025, where the @Generable macro guarantees structurally correct model output straight into Swift types via constrained decoding [45]. Note what is absent here: a Python layer. A 3-billion-parameter model, quantised to 2 bits, runs in your pocket — and Swift talks to it directly.

Third, Swift has stopped being “Apple-only”: official Linux and Windows support, an Android workgroup (2025), Embedded Swift for microcontrollers, deep C++ interop [44].

💡 Key idea of this section. S4TF failed by storming the Python fortress head-on — on the territory of server-side training, where Python’s ecosystem advantage is absolute. The new game is being played on a different board: on-device, edge, local inference — territories where Python has no army, because its runtime simply does not exist there. It is exactly there that a compiled language with Python-grade readability is not an “alternative” but the shortest path.

8. Honest counterarguments (and what they actually prove)

Scientific integrity demands that the prosecution be heard — that is, Python’s defence. Here are the four strongest arguments and our replies.

“The ecosystem decides, and Python has it.” True. 582 thousand new AI repositories in a year [5], every textbook, every job posting. Our reply: we are not proposing a battle for server-side training — Python will hold it for a long time, and that is fine. The thesis is different: the new niches (see Section 9) are free of that network effect, and there the language choice is open again. And inside the citadel itself the performant code is no longer Pythonic — the only remaining dispute is about which language the storefront is written in.

“Julia already tried — and where is it now?” The lesson of Julia is real: a language’s speed does not beat ecosystem quality (Kidger describes leaving Julia after months of fighting incorrect gradients [12]). But note what exactly that lesson proves: technically fast languages exist in abundance; the deficit is in engineering maturity and a sponsor. Swift’s sponsor is a company with a multi-trillion capitalisation for which on-device AI is existentially necessary, plus a living industrial toolchain and millions of developers. That is not a guarantee — but it is a difference.

“Python is speeding up too: free-threading, JIT, torch.compile.” It is — and we honestly cited the numbers: +20–40% over five releases [21], free-threading with a 5–10% penalty [20], torch.compile at 2.27× [23]. But read the structure of these victories closely: free-threading fixes something compiled languages never had; torch.compile speeds Python up by ceasing to execute it. This is brilliant engineering that reinforces our thesis: the future of Python performance lies in Python not being executed.

“The overhead is pennies next to the GPU.” Already dissected in Section 5 with numbers; let us add the economics of scale. ChatGPT handles 2.5+ billion requests per day [55]; by Sam Altman’s estimate, the average query costs 0.34 Wh [54] (methodology undisclosed — keep your scepticism). Multiplying the two gives ~0.31 TWh per year just for the answers of a single product. At such volumes, even 5–10% of software overhead is 15–31 GWh annually — the yearly consumption of a small town. The pennies stopped being pennies when the meter started spinning in the billions.

⚠️ A caveat you did not expect from this article. AI’s energy efficiency per task is improving — at a pace the IEA calls unprecedented in the history of energy [49]. And one should be sober about loud AI-energy numbers in general: the famous training-carbon estimate from Strubell et al. [52] was later corrected downward by Google engineers by a factor of 18–88 [53] — a lesson in caution we try to absorb. Our argument is not that AI will wreck the power grid and only Swift can save it. It is humbler and firmer: when inference is up to 90% of the cost of an ML workload (AWS [50]) and ~3/5 of its energy (Google [51]), software overhead stops being a matter of taste and becomes a budget line. An engineering culture that tunes kernels down to microseconds yet shrugs at a 70× gap in its orchestration language simply has not yet converted its habits into megawatts.

9. New horizons: what opens up beyond Python

Everything said so far would be mere grumbling at a successful language, were it not for the main point: the most interesting territories of the coming years lie where Python physically cannot reach.

Horizon 1: AI in your pocket. Apple Intelligence’s ~3-billion-parameter model generates 30 tokens per second right on an iPhone — offline, private, free [46]; the third generation (June 2026) added a sparse 20-billion-parameter model [56]. There is no CPython here and there never will be: the budgets of battery, memory, and latency leave room only for a compiled stack. This entire world is programmed from Swift — including guaranteed-typed LLM output via @Generable [45].

Horizon 2: local and server-side inference. The llama.cpp/GGUF ecosystem has already proven that an inference engine does not need Python [31]. Server-side Swift, with ARC and no GC pauses, is a ready-made candidate for the API layer of the same problem class Apple closed with Password Monitoring [42]. More and more often the bottleneck is not the matrix on the GPU but the orchestration around it: agentic loops, tool calls, millions of small decisions on the CPU. That is exactly the territory where an interpreter loses to a compiler by a crushing score.

Horizon 3: the end of the two-language problem. The deepest shift is methodological. Today’s path — “prototype in Python → rewrite in C++ → convert for the device” — is doubled labour and walls between teams. A language with the readability of a prototype and the speed of production lets one person walk the path from idea to device — as Julia dreamed in 2012 [10], as S4TF attempted in 2018 [38], and as the Swift stack — MLX, differentiable Swift, Foundation Models — makes possible today, for the first time, in its natural niche. The ability to read and write the whole stack top to bottom is not just speed. It is understanding instead of incantations.

📐 One last analogy. The Python ecosystem resembles imperial units — feet and pounds: historically earned, grown into the infrastructure, backed by the habit of hundreds of millions of people. One can live with them (America does). But every new calculation requires conversion coefficients, every junction of systems is a source of error, and every so often a Mars probe shatters against it. Nobody will abolish feet by decree — but every new bridge is quietly designed by engineers in metres.

10. What we’ve learned

• Python won deservedly — through the “glue” strategy [1], the head start of its scientific ecosystem, and network effects. Arguing with this is pointless; understanding it is mandatory.
• Python in AI is the storefront, not the engine: 36% of PyTorch’s codebase is compiled languages [9], and every successful “Python-stack accelerator” works by removing Python from execution — from tf.data to torch.compile [22, 23, 26].
• The tax is real and measured: ~70× on time and energy in language benchmarks [14, 17], up to half of an inference engine’s time on CPU overhead [30], 30% of Google’s ML-fleet compute time on input pipelines [26]. At the scale of billions of requests per day, percentages turn into gigawatt-hours [54, 55].
• The industry is already voting with its feet: llama.cpp and GGUF [31, 32], Rust inside Hugging Face [33], MLX [43], ExecuTorch [34], Mojo [35] — wherever performance is mandatory, compiled languages are already at the helm.
• Swift is a rare combination: speed within a small factor of C (not a couple of orders of magnitude) [14], syntax honestly comparable to Python, ARC and compile-time race checking [44], production evidence [42], and the industry’s only systemic integration with an on-device LLM [45]. The lessons of S4TF [39] and Julia [12] are taken on board: not a storming of the citadel, but new territories.
• The main horizon is one language from idea to device. The two-language problem [11] is not a law of nature but a historical compromise. Compromises get revisited.

The king is not dead, and nobody is planning to depose him. But a crown is not an argument in an engineering dispute. Next time you choose a tool for a model that must live on a device, inside an agentic loop, or within an electricity budget — just do the math. The numbers in this article are at your service.

Better yet — check for yourself. Nothing teaches like an experiment.

This article was originally published by Andrey on HackerNoon.