A Brief History… and a Future AI Event Horizon for Software Engineers

My fully completed Advent of Code stars for 2024. The first time I’ve ever gotten all 50 stars.

Insanity Update 3 — August 29, 2026

Apple has introduced their new M5 and M6 chips. The M6 still has a 512 GB unified RAM configuration, but what makes the M5 Ultra special is a staggering 1.2 TB/s memory bandwidth, which means you could theoretically sweep model weights—even models up to 500 GB or so—multiple times a second, for every single token.

Insanity Update 2 — July 23, 2025

Well, Claude 4 has long since been out (with GPT-5 rumoured within the next few weeks?). Anyway, I wanted to add that GitHub Copilot agent mode was officially added (out of beta) in May 2025, and just last week (July), agent mode with MCP tool usage was officially out of beta.

I use GitHub Copilot as sort of the standard accepted quality measure when it comes to all these MCP tools, since it comes from Microsoft—the rest, after all, are all startups.

Insanity Update — March 5, 2025

Welp, my timeline is already too slow! A few days ago, Anthropic announced Claude 3.7, which, okay, hasn’t totally broken anything on my timeline here, but just today Apple did by releasing its M3 Ultra, which exceeded my guess of 256 GB of RAM for 2025 and can be configured with 512 GB of RAM, enabling models of up to 600 billion parameters to run… you can run almost the full DeepSeek model locally!!!!

The Truth…

I have a confession. In December 2024, I fully completed “Advent of Code,” or “AoC,” in Rust, with all 50 challenge stars. How did I do it? Nearly entirely using careful prompting with Claude 3.5 Sonnet! To be honest, I was extremely impressed by Claude’s ability to consistently write code that produced fully working solutions. I’d estimate that this was the case approximately 75% of the time—with a single shot! Furthermore, this was with a single shot using only the problem prompt itself (which, in AoC format, always includes step-by-step smaller examples).

As the 25 days of Christmas went on, many of the AoC problems became increasingly nontrivial. Solutions typically require usage of advanced algorithms, such as breadth-first search (or BFS), as well as Dijkstra’s algorithm (i.e., finding the shortest path in a maze when the maze locations, or routes, have costs associated with them).

With the most challenging of problems, I had to sometimes scaffold some Rust code myself, as well as prompt Claude to think with a “puzzle”-like mentality, because usually the challenges are set up in a way to include an “aha” moment that leads you to the most efficient style of solution.

With this sort of workflow, I ended up spending about 55 minutes per problem, or 28 minutes per part, for the whole thing:

23 hrs and 9 mins, logged by our friends over at WakaTime.

Not bad, and definitely something I couldn’t have achieved at any earlier point in my software career. I’d like to chalk it up to 2024 being my “most experienced and expert year as a software engineer”—after all, our experience should only grow throughout the years, right? But of course, I can’t take full credit; it has much more to do with the LLM generation and discourse workflow I described above.

How Did We Get Here?

To put this all into context—and I know, it sometimes feels like it’s just getting faster and faster every day—let’s think of everything that’s happened with AI models and hardware in the past few years:

• June 2022: Microsoft releases GitHub Copilot to individual developers.
• November 2022: OpenAI releases ChatGPT (GPT-3.5), a watershed moment when transformers actually became recognized as somewhat useful for some tasks—or, at the very least, very interesting with creative work.
• November 2023: A year after the fated ChatGPT release, Apple releases its M3 chip series, which includes a chip with up to 128 GB of memory, enough to run even the most advanced LLM models locally. Previously, the M2 Ultra had offered up to 192 GB.
• June 2024: Anthropic releases its Claude 3.5 Sonnet model, which was, at that date, the most advanced model for coding. (Even in early 2025, it still is the top model on HumanEval—the benchmark for code generation—though it would be nice to have the OpenAI o3 benchmark not just for “AGI” but for HumanEval as well to compare against Claude!)
• December 2024: On Christmas Day, DeepSeek V3 is released. This model is a strong Mixture-of-Experts (MoE) language model with 671B total parameters, fully open-source, and rivals leaders Claude Sonnet and o1.

In short, we saw another 128 GB chip option in the M3 generation after the M2 Ultra’s 192 GB maximum, and went from a “sort of useful but funny” ChatGPT product to top-tier open-source models in two years. In hardware terms, is it so hard to think that we can get to 256 GB within another year? And, in model terms, that we can get to an open-source model beating one of the “behemoth” models in at least one benchmark by the end of 2025? If we can imagine an improved model like a Sonnet 4.0 that could do even the hardest coding challenges flawlessly (i.e., a 99.9 or even perfect score on HumanEval) with a single shot, well, then I think we have the recipe for a near-autonomous coder sitting right on your hardware that can solve any application-level problem even across a large codebase.

And… silly me, while already doing research for this historical timeline, I found this on Apple’s website:

M4 Max supports up to 128GB of fast unified memory and up to 546GB/s of memory bandwidth, which is 4x the bandwidth of the latest AI PC chip. This allows developers to easily interact with large language models that have nearly 200 billion parameters.

Then, of course, there’s the latest GB10 “superchip” from NVIDIA:

The GB10 Superchip enables Project DIGITS to deliver powerful performance using only a standard electrical outlet. Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage. With the supercomputer, developers can run up to 200-billion-parameter large language models to supercharge AI innovation. In addition, using NVIDIA ConnectX® networking, two Project DIGITS AI supercomputers can be linked to run up to 405-billion-parameter models.

So… NVIDIA claims it’s a “personal supercomputer,” but let’s be honest—it’s the same exact spec as the M4… nice try, NVIDIA!

So, even as I write this historical timeline, some of the most impressive achievements have occurred in the past month!

A Brief Sketch of Predictions / a Possible 2025 Timeline

• Q1 2025: ChatGPT releases Orion (GPT-5), which is a 10-trillion-parameter model. Due to the limits of the GPT architecture, which nobody seems to believe except me, it barely outperforms o3 on the ARC-AGI-PUB, leaving many, particularly those on the ‘big model cutting edge’ side of things, scratching their heads on what to do next.
• Q2 2025: Anthropic releases Claude 4 Sonnet, which earns a perfect score on HumanEval.
• Q3 2025: Apple releases an M5 chip option with 164 GB of RAM.
• Q4 2025: DeepSeek V3.5 or a similar open-source, out-of-the-blue actor appears, beating Google, OpenAI, or Anthropic models in at least one benchmark, but likely more. Open-sourcers rejoice!

…And Beyond?

Let’s assume—and this is a big assumption—that Apple (or any chip maker, for that matter) can double the amount of RAM per chip every year going forward for at least the next five years. That would mean this year, we could already see a theoretical 256 GB of unified RAM, where you could run a 400-billion-parameter model locally. That means with 512 GB, you can run 800 billion, and so on.

As a chart, that looks like this:

Year	Unified RAM Size	Model Parameters
2023	64 GB	100 billion (already in the history books)
2024	128 GB	200 billion (already achieved)
2025	256 GB	400 billion (256 GB unified RAM THIS year?!)
2026	512 GB	800 billion
2027	1 TB	1.6 trillion (for scale: rumoured GPT-4 size)
2028	2 TB	3.2 trillion
2029	4 TB	6.4 trillion
2030	8 TB	12.8 trillion (rumoured smaller GPT-5 size)
2031	16 TB	25.6 trillion
2032	32 TB	51.2 trillion (rumoured larger GPT-5 size)
2033	64 TB	102.4 trillion
2034	128 TB	204.8 trillion
2035	256 TB	409.6 trillion (human level “size” achieved)

This means by 2027 you could theoretically run a GPT-4-sized model on your laptop hardware (if there is an open-source group up for the task of training and publishing such a model). By 2030 you could run a rumoured GPT-5 (which doesn’t even exist yet! 😂)—a rumoured true human-level intelligence model, again, locally. It’s simply insane that such a mind-boggling sentence is theoretically possible in five years.

This neat post I found by Dave Shapiro fell along the same line of thinking, estimating that human-level LLMs could be around as early as 2030. (I encourage you to go read that post, it’s fantastic!)

Dave wrote that post back in 2021, and even by his “conservative” estimate of a 2x increase in size per year (like mine), we should be at a 25.6-trillion-parameter model by now in 2025! Currently, the largest model out there is believed to be GPT-4 at 1.8 trillion parameters, which is a 14x reduction from that 25.6-trillion-parameter mark. And even if GPT-5 actually comes out this year, we’ll still be lagging behind at a rumoured 10 trillion parameters (but hey, the highest rumoured size sits up at 80 trillion—so in this case, he would be right!)

The Bad News (My Projection)

At some point, however, this “basic” LLM model configuration will have run its course. I say basic because the architecture is still based on transformers after all this time, and perhaps the only real “step function” change is that of chain-of-thought or reasoning models, and even then, it’s not a clear architectural shift but rather a sort of way of reworking how the model itself is used—even GPT-4, one of the best in its class, is believed to be just a series of eight smaller models. So, a lot of what we’ve seen so far consists of iterations and clever ways of using the transformer architecture, unless o3 is really doing something different.

These foundational model folks (and the market in general) are starting to realize that chain-of-thought models are actually the most important anyway. It’s quite logical too: if you have an agentic model with a high level of autonomous reasoning that can accurately run on a device, e.g., use Google, a calculator program, Excel, or write code, and so on, why would you feed it training data like the entirety of Wikipedia, or multiplication tables, or pictures and videos and all that nonsense? That stuff just bloats the model, makes it incredibly expensive to train, and likely makes it too difficult to hammer into a consumer product with all the benchmarks these companies are trying to be at the top of.

So… What Does this Really All Mean?

Even with these relatively alarming projections, I still don't think any jobs are going anywhere soon. For software engineering, at some point, someone has to do the prompting, the product design, execution, and marketing. Everything in person is still dominated by good ol’ humans.

So, once again, don’t believe everything you read on Reddit.

—Chris