Compilers and Programming Languages

A modern compiler is one of the most complex software systems ever built. It reads a text file written by a human, parses it, analyses it, optimises it, and emits machine code that runs at a small fraction of the speed of hand-written assembly. The story of how compilers went from crude one-pass assemblers to multi-stage optimisers that produce better code than most humans can write is, in a real sense, the story of how software came to dominate every other form of human artefact.

The first compilers and the bootstrap problem

The first compiler is usually credited to Grace Hopper and her team at Remington Rand in 1952, in the form of the A-0 system, which translated symbolic mathematical notation into machine code. The first complete high-level language compiler is generally credited to John Backus and his team at IBM, who released the Fortran compiler in 1957. The Fortran project was an explicit attempt to make programming “more like mathematics”: to free the programmer from the tyranny of register allocation, instruction scheduling, and address calculation, and to let them write in a notation closer to the problem being solved.[1, 2, 14]

The Fortran compiler was a major technical achievement. It was the first compiler to do serious optimisation (in particular, the first to do loop-nest optimisation and register allocation automatically), and it produced code that was, on average, only 20 percent slower than hand-written assembly. The fact that this was possible at all was, at the time, deeply controversial; many computer scientists, including the influential John von Neumann, believed that automatic optimisation was impossible in principle. Backus’s success in producing an optimising compiler at all was, in retrospect, a decisive moment in the history of programming.[1, 14, 15]

The first Lisp compiler, written by John McCarthy and his students at MIT in 1958, took a different approach. Lisp is a much higher-level language than Fortran: it is interpreted, has automatic memory management (garbage collection), supports functions as first-class values, and has a syntax that is essentially a parse tree. The first Lisp compiler was written in Lisp itself, which raised the bootstrap question: how do you run a program written in language L on a computer that has no compiler for L? The answer, in McCarthy’s case, was to write a small interpreter for Lisp in assembly (a few hundred lines), then write a more complete Lisp system in Lisp, then compile the latter with the former. The technique — known as bootstrapping — is now standard.[3, 14, 16]

The language explosion: Fortran, LISP, ALGOL

The 1960s were a period of extraordinary experimentation in programming language design. The most influential development was the design of ALGOL, a project led by an international committee of computer scientists (including Backus, Naur, McCarthy, Wirth, Dijkstra, and others) and intended to be a universal algorithmic language. The first version, ALGOL 58, was influential; the second, ALGOL 60, was perhaps the most influential language design of the century. ALGOL introduced the Backus–Naur form (BNF) for describing language syntax, the concept of nested lexical scoping, the concept of block structure, and the use of structured control flow (if-then-else, while, for). Nearly every language designed since 1960 has borrowed something from ALGOL.[2, 4, 17]

The ALGOL family tree is enormous. From it descend BCPL (1967), from which descend B (1969), from which descends C (1972). C is, directly or indirectly, the ancestor of C++, Objective-C, Java, C#, Go, Rust, and most other modern systems languages. The other major family is the LISP family: LISP, Scheme, Common Lisp, Clojure, Racket. The third is the ML family, which descends from the ML language designed by Robin Milner in 1973, and which includes Standard ML, OCaml, F#, and (more distantly) Haskell. The fourth is the Prolog and logic-programming family. The fifth is the Smalltalk family, which includes Ruby, Python, and (in design philosophy) JavaScript.[2, 4, 17]

Each family embodies a different set of design choices. C prioritises efficiency and direct access to hardware, at the cost of safety. Lisp prioritises expressiveness and metaprogramming, at the cost of performance. ML prioritises type safety and formal reasoning, at the cost of complexity. Smalltalk prioritises object-orientation and late binding, at the cost of predictability. The trade-offs are not absolute, and modern languages have increasingly borrowed ideas from multiple families. Rust, for example, combines C-like efficiency with ML-like type safety, and adds a sophisticated ownership system that solves the long-standing problem of memory management without garbage collection.[4, 5, 17]

Static versus dynamic typing

The most consequential design choice in any modern language is the type system. A type system is a discipline for classifying the data a program manipulates (integers, floating-point numbers, strings, lists, records, functions) and for rejecting programs that misuse them. A type error — adding a number to a string, calling a non-existent method, indexing past the end of an array — is the most common class of bug in any large program, and a good type system catches most of them at compile time, before the program ever runs.[5, 6, 18]

Statically typed languages (C, C++, Java, Rust, Go, Haskell) require the programmer to declare the type of every variable (or, in the case of type inference, allow the compiler to deduce the type from context). The compiler checks types at compile time and rejects programs that misuse them. The cost is verbosity: the programmer has to write more, and the type system is sometimes more rigid than the problem requires. The benefit is safety: most type errors are caught at compile time, and the resulting program is much less likely to crash at runtime. Dynamically typed languages (Python, JavaScript, Ruby, PHP) infer the type at runtime. The programmer writes less, and the language is more flexible, but the compiler cannot catch type errors before the program runs. The cost is a much higher rate of bugs, particularly in large codebases. The benefit is faster development of small programs.[6, 7, 18]

Since 2010, the two camps have moved closer together. Statically typed languages have added type inference (C#, Rust, Kotlin, Swift, Scala), reducing the verbosity without sacrificing safety. Dynamically typed languages have added optional type annotations (TypeScript for JavaScript, type hints for Python, type annotations for Ruby), allowing the programmer to add static type checking where it is most useful. The result is that the choice between static and dynamic typing is no longer a choice between two camps but a spectrum, and most modern software is written in languages that allow both styles. The trade-off remains: a more expressive language is easier to write, but a more strictly typed language is easier to maintain.[6, 7, 19]

Modern compilers: optimisations, SSA, LLVM

The architecture of a modern compiler is a multi-stage pipeline. The front-end reads the source code, performs lexical analysis (grouping the characters into tokens), syntactic analysis (parsing the tokens into an abstract syntax tree), and semantic analysis (resolving names, checking types, inferring types where possible). The output of the front-end is a language-independent intermediate representation, typically in static single-assignment form (SSA), in which every variable is assigned exactly once. The middle-end applies a series of optimisations to the SSA: constant folding, dead-code elimination, inlining, loop transformations, vectorisation, parallelisation, register allocation. The back-end lowers the optimised SSA to machine code for a specific target (x86, ARM, RISC-V, GPU), handling instruction selection, instruction scheduling, and the layout of data in memory.[7, 8, 20]

The separation of the front-end, middle-end, and back-end is the most important architectural idea in modern compiler design. It allows a single intermediate representation and a single set of optimisations to be reused across many source languages and many target architectures. The most successful realisation of this idea is LLVM, a compiler infrastructure originally developed at the University of Illinois and now maintained by a large open-source community. The LLVM middle-end and the LLVM back-end (which supports x86, ARM, RISC-V, GPU, and many other targets) are reused by the front-ends for C, C++, Objective-C, Rust, Swift, Julia, and many other languages. The result is that a new language can be brought to market with high-quality code generation in a small fraction of the time it would have taken a few decades ago.[7, 8, 21]

The GNU Compiler Collection (GCC), originally written by Richard Stallman in 1987 as part of the GNU project, is the other major open-source compiler infrastructure. GCC is the default compiler on most Linux systems and supports a similar set of front-ends and back-ends. The two projects — LLVM and GCC — compete in the open-source world, and the performance gap between them is typically small (within 5–10 percent on most benchmarks). Both are remarkably complex systems: GCC has more than 15 million lines of code, and LLVM has more than 5 million.[8, 9, 22]

The dominance of a handful of languages

Despite the diversity of programming languages (the Wikipedia list of programming languages contains more than 700 entries), the vast majority of software is written in perhaps a dozen languages. The 2024 TIOBE index puts Python, C, C++, Java, C#, and JavaScript at the top; the IEEE Spectrum ranking puts Python first, followed by C, C++, Java, and C#. The RedMonk ranking, which correlates GitHub code presence with Stack Overflow discussion, puts JavaScript, Python, Java, and TypeScript at the top. Despite differences in methodology, the rankings agree on the dominance of a small set of languages: Python for data science, scripting, and general-purpose use; C and C++ for systems programming and performance-critical code; Java for enterprise backends; JavaScript and TypeScript for web frontends.[9, 10, 23]

The dominance of a few languages is driven by network effects. More programmers know Python, so more new projects are written in Python. More libraries are written for Python, so Python is more productive for new tasks. More tools support Python, so Python is easier to learn and use. The result is a self-reinforcing cycle, in which the dominant languages become more dominant, and the long tail of less popular languages struggles to find programmers, libraries, and tools. The switching costs are high: rewriting a large codebase in a new language is expensive, and the benefits of the new language have to be substantial to justify it.[10, 11, 23]

There are signs that the dominance of the established languages is beginning to erode. Rust, developed at Mozilla and first released in 2015, has been adopted by Microsoft, Google, Amazon, and Meta for systems-level code, and is increasingly used in the Linux kernel itself. Kotlin, developed by JetBrains and now the preferred language for Android development, has largely replaced Java for new mobile apps. Go, developed at Google, has become the dominant language for cloud-native infrastructure (Docker, Kubernetes, Terraform, Prometheus are all written in Go). Swift, developed by Apple, has largely replaced Objective-C for iOS and macOS development. The trend is towards more specialised languages, each dominant in a particular domain.[11, 12, 24]

The future: safety, parallelism, AI

The frontier of programming language design is in three directions. The first is safety. Memory-safety bugs — buffer overflows, use-after-free, double-free, uninitialised reads — are responsible for an estimated 70 percent of all security vulnerabilities in large C and C++ codebases. The cost of these bugs, in both economic and human terms, is enormous. The response has been a new generation of memory-safe languages: Rust, Swift, Go, Kotlin, Java (after a fashion), C# (with nullable reference types), and even the gradual migration of C and C++ codebases to safer alternatives. The US National Security Agency, in 2022, published a guidance document recommending the use of memory-safe languages. The trend is now irreversible.[12, 13, 25]

The second direction is parallelism. The free lunch from Moore’s law is over; the only way to get more performance is to use more cores, and to use specialised hardware (GPUs, TPUs, FPGAs). The challenge is that writing parallel code is hard: race conditions, deadlocks, and non-determinism are difficult to debug, and the existing languages were not designed with parallelism in mind. The response has been a wave of new language features: async/await for asynchrony, channels and goroutines for message passing, ownership types for memory safety without garbage collection, automatic parallelisation in compilers. The languages that do this best — Rust, Julia, modern C++ with TBB, Go — are gaining ground in performance-critical applications.[13, 24, 26]

The third direction is AI assistance. The release of GitHub Copilot in 2021, followed by ChatGPT in 2022, and a wave of competitors in 2023 and 2024, has begun to change the economics of writing code. A modern AI assistant can write a small function, debug a routine error, or translate between languages, with high accuracy and in a fraction of the time it would take a human. The implication for the future of programming is unclear: some predict the end of programming as a profession, others a productivity boom in which programmers write 5–10 times as much code per day. The empirical evidence, in 2024, suggests that the truth is closer to the second scenario: programmers are more productive, but the bottleneck has moved from writing code to designing systems, reviewing code, and testing. The art of programming is changing, not disappearing.[13, 24, 27]

Comparison: leading programming languages (2024)

Language	First appeared	Designer(s)	Type system	Memory management	Primary use
C	1972	Dennis Ritchie (Bell Labs)	Static, weak	Manual	Systems programming, embedded
C++	1985	Bjarne Stroustrup (Bell Labs)	Static, strong	Manual (with smart pointers)	Systems, games, finance
Java	1995	James Gosling (Sun Microsystems)	Static, strong (with type erasure)	Garbage collected	Enterprise backends, Android (legacy)
Python	1991	Guido van Rossum	Dynamic, strong (optional hints)	Garbage collected	Data science, scripting, ML
JavaScript	1995	Brendan Eich (Netscape)	Dynamic, weak	Garbage collected	Web frontend, Node.js backend
Go	2009	Robert Griesemer, Rob Pike, Ken Thompson (Google)	Static, strong, inferred	Garbage collected (concurrent)	Cloud infrastructure, DevOps
Rust	2015	Graydon Hoare (Mozilla)	Static, strong, inferred	Ownership (no GC)	Systems programming, security-critical
Kotlin	2011	JetBrains	Static, strong, inferred	Garbage collected	Android, server-side
Swift	2014	Apple	Static, strong, inferred	Automatic Reference Counting	iOS, macOS, server-side

Key numbers

• ~700: number of programming languages in Wikipedia’s list (2024).
• ~12: number of programming languages in which the vast majority of software is written.
• ~70 percent: estimated share of security vulnerabilities in large C/C++ codebases caused by memory-safety bugs.
• ~15 million: lines of code in GCC (2024).
• ~5 million: lines of code in LLVM (2024).
• ~30 percent: estimated boost in programmer productivity from AI coding assistants (early 2024 evidence).
• 1957: year of the first Fortran compiler.
• 1958: year of the first Lisp system.

References

1. Wikipedia, “Fortran.” (Standard reference on Fortran, with history of the compiler.) https://en.wikipedia.org/wiki/Fortran
2. Wikipedia, “History of programming languages.” (Standard reference on the history of programming languages.) https://en.wikipedia.org/wiki/History_of_programming_languages
3. Wikipedia, “Lisp (programming language).” (Standard reference on Lisp, with history of the bootstrap problem.) https://en.wikipedia.org/wiki/Lisp_(programming_language)
4. Wikipedia, “ALGOL.” (Standard reference on ALGOL, the most influential language design of the 1960s.) https://en.wikipedia.org/wiki/ALGOL
5. Wikipedia, “Type system.” (Standard reference on type systems in programming languages.) https://en.wikipedia.org/wiki/Type_system
6. Wikipedia, “Static typing.” (Standard reference on static type checking.) https://en.wikipedia.org/wiki/Type_system#Static_typing
7. Wikipedia, “LLVM.” (Standard reference on the LLVM compiler infrastructure.) https://en.wikipedia.org/wiki/LLVM
8. Aho, A. V., M. S. Lam, R. Sethi, and J. D. Ullman. Compilers: Principles, Techniques, and Tools. 2nd ed. (The “Dragon Book”). Upper Saddle River, NJ: Pearson, 2006. (The standard reference on compiler construction.)
9. Wikipedia, “GNU Compiler Collection.” (Standard reference on GCC.) https://en.wikipedia.org/wiki/GNU_Compiler_Collection
10. Wikipedia, “TIOBE.” (Standard reference on the TIOBE programming language index.) https://en.wikipedia.org/wiki/TIOBE
11. Wikipedia, “Programming language.” (Standard reference on programming languages.) https://en.wikipedia.org/wiki/Programming_language
12. Wikipedia, “Rust (programming language).” (Standard reference on Rust.) https://en.wikipedia.org/wiki/Rust_(programming_language)
13. National Security Agency. “Memory safe languages.” Cybersecurity advisory, 2022. https://media.defense.gov/2022/Nov/10/2003132746/-1/-1/0/CSI_MEMORY_SAFE_LANGUAGES.PDF
14. Backus, J. “The history of Fortran I, II, and III.” In History of Programming Languages. New York: ACM, 1978. (A first-person history of the Fortran project by its lead designer.)
15. Wikipedia, “John Backus.” (Standard reference on John Backus, with his contributions to Fortran and to functional programming.) https://en.wikipedia.org/wiki/John_Backus
16. McCarthy, J. “Recursive functions of symbolic expressions and their computation by machine, Part I.” Communications of the ACM 3(4) (1960), 184 - 195. (The original Lisp paper.)
17. Wikipedia, “Comparison of programming languages.” (A reference on the major programming language families.) https://en.wikipedia.org/wiki/Comparison_of_programming_languages
18. Wikipedia, “Type safety.” (Standard reference on type safety in programming languages.) https://en.wikipedia.org/wiki/Type_safety
19. Wikipedia, “Type inference.” (Standard reference on type inference, with examples from ML, Haskell, Rust, and others.) https://en.wikipedia.org/wiki/Type_inference
20. Wikipedia, “Static single-assignment form.” (Standard reference on SSA form, the standard intermediate representation in modern compilers.) https://en.wikipedia.org/wiki/Static_single-assignment_form
21. Lattner, C. “LLVM: an infrastructure for multi-stage optimization.” PhD thesis, University of Illinois at Urbana-Champaign, 2002. (The original LLVM dissertation.)
22. Wikipedia, “Richard Stallman.” (Standard reference on Richard Stallman and his role in the GNU project.) https://en.wikipedia.org/wiki/Richard_Stallman
23. RedMonk. “Programming language rankings.” Industry analysis, 2024. https://redmonk.com/sogrady/2024/03/12/language-rankings-6-24/
24. IEEE Spectrum. “The top programming languages.” Annual ranking, 2024. https://spectrum.ieee.org/top-programming-languages
25. Wikipedia, “Memory safety.” (Standard reference on memory safety, with a section on memory-safe languages.) https://en.wikipedia.org/wiki/Memory_safety
26. Wikipedia, “Parallel computing.” (Standard reference on parallel computing, with sections on languages and tools.) https://en.wikipedia.org/wiki/Parallel_computing
27. Wikipedia, “GitHub Copilot.” (Standard reference on GitHub Copilot and AI-assisted programming.) https://en.wikipedia.org/wiki/GitHub_Copilot