You can’t decouple technical approaches from the economic forces that shape them. When systems are built to be profitable over the long term, architecture matters. During the ZIRP era speed was everything, but in this new epoch, those of us building real systems will need to design for profitability and sustainability from the start.
We’ll look at these problems in three layers, system-level shifts, organizational and cultural breakdowns, and the emerging class of external and safety risks that AI and automation are already accelerating.
System-Level Challenges
In the 2010s, we focused on scaling infrastructure, deploying faster, and adopting cloud-native tools. Much of this was reactive in nature, respond to failures, absorb load, keep services up. That foundation has matured. Now the challenge is comprehending what’s been built.
| Challenge | 2015: Microservices Era | 2025: Observable Reflective Era |
|---|---|---|
| Scale & Complexity | We scaled horizontally to absorb load. Systems had to stay up no matter what. | We’re drowning in APIs and internal tools. The challenge is knowing how it all fits together. |
| System Behaviour | Intelligence was transactional. Systems scoped context to a single request, response, or session. | Intelligence spans spacetime. We need causal scoping, the ability to trace what happened, who made the decision, and why. |
| Deployment & Delivery | Shipping became automated. CI/CD pipelines moved code, but not understanding. Auditability was an afterthought. | Everyone can ship code. Now we need to understand what shipped, why, and how it was tested and approved. Auditability is becoming critical. |
| Infrastructure Models | Scheduling was the frontier. Mesos, Kubernetes, and Spark fought over where and how code should be scheduled and run. | We largely solved scheduling and I/O. Now we fumble with telemetry and auditability. The real challenge is visibility into decision points across fluid infrastructure. |
| Data Models & Access | Big data became fast data. Everyone chased raw throughput, optimizing for problems that only a handful of companies actually had. | Data is fast and increasingly accountable. We need reflective data systems that expose lineage, decisions, and downstream impact. |
The bottom line is that systems complexity along with aloof design methodologies have led to incomprehensibility. Our systems are technically capable, but only when someone understands why they behave the way they do.
Organizational and Cultural Challenges
The deeper truth is that most of our systems reflect our organizations. Team structures, incentives, delivery practices, and architectural habits all leave a trace in the code. And many of those traces don’t age well.
| Challenge | 2015: Microservices Era | 2025: Observable Reflective Era |
|---|---|---|
| Change & Adaptability | Heroic chaos. Fail fast and patch. It didn’t matter, because the competition struggled too. Some teams were competent, but nearly all were losing money. | Measured evolution. Change must be logical (observe, orient, decide, act). Assume the competition is mature, capable, and profitable. |
| Team Velocity | We optimized for velocity metrics, shipped complexity behind microservice fences, and rarely questioned whether speed was producing understanding. | As systems gain machine intelligence, moving fast without reflection is like accelerating into a wall. Correctness once again matters. |
In 2025, clarity is the bottleneck. Architecture connects decisions, intent, and outcomes. Teams that observe, orient, decide, and act with purpose will outpace everyone else.
External and Safety Risks
Even if your systems are technically solid and your team is well-aligned, the world is changing around you. AI has moved from toy to threat surface. Automation will increasingly act without oversight, making decisions as we sleep. And opaque systems are increasingly dangerous and unacceptable.
| Challenge | 2015: Microservices Era | 2025: Observable Reflective Era |
|---|---|---|
| AI and Autonomy | AI was a research project. Most systems were built for human actors. | AI is in the loop. Agents now operate autonomously, triggering flows, writing code, and taking action. |
| Safety and Compliance | Compliance was checklist-driven. Logs were good enough. | Auditability is table stakes. Systems must preserve intent, justify actions, and disclose outcomes. |
| Security and Misuse | Threats were external. Attackers were mostly human, and surfaces were fairly predictable. | AI-driven attacks operate at scale. Autonomous misuse is now possible from internal and external actors. Intelligent systems will be stress tested. |
| Decision Liability | Decisions were traceable through people and tribal knowledge. Management pushed accountability down to developers. | The developers who used to absorb blame are gone, replaced by AI that can’t be held accountable. Liability has nowhere to go but back up. |
We are already seeing AI-generated code pushed to production, LLMs used in critical domains, and autonomous agents making irreversible decisions. If your system cannot expose what it did and why, you are operating on borrowed time.
These three layers don’t exist in isolation. They compound. And the compounding didn’t start with AI.
Slop Isn’t New
The Agile Manifesto was a good document written by practitioners who were fed up with heavyweight process. Its four values and twelve principles describe a way of working that prioritises responsiveness, collaboration, and working software. There is nothing wrong with it. The problem is what happened to it afterward.
As Imagined
A set of values written by practitioners who lived under the weight of heavyweight process and wanted something better.
- Low ceremony, fluid priorities, direct collaboration
- Working software as the primary measure of progress
- Responding to change over following a plan
- A rejection of process for the sake of process
As Designed
Good-faith interpretation by teams trying to apply the values under real constraints.
- Priorities shift because the work demands it, not because a timebox expired
- Specs and design emerge naturally when people care about the outcome
- Ceremony is minimal and serves the team, not the other way around
- Trade-offs are revisited often
As Corporatised
What vendors, certifications, and corporate adoption turned agile into.
- Planning poker as compliance theatre, velocity as a vanity metric
- Time-boxing as a pressure tool, not a focusing mechanism
- “Working software over comprehensive documentation” weaponised against planning and designing
- Values becoming products themselves, and all of the vapid trappings: certified coaches, licensed frameworks, enterprise tooling
As Practiced in FOSS
How free and open source communities actually work, and the closest thing to the Manifesto practiced honestly.
- Maintainers triage, prioritise, and ship based on what matters, no sprint reviews, no story points
- Contributors write RFCs, design docs, and specs that are often more rigorous than anything produced inside a corporate sprint
- Collaboration is direct and asynchronous, driven by the work, not by a scheduled ceremony
- When you care about what you’re building, highly correct specifications emerge naturally with low ceremony
When practitioners work on something they genuinely care about, free from vendor capture and corporate incentive structures, they converge naturally on exactly what the Manifesto described. Fluid prioritisation. Direct collaboration. Working software. Responding to change. FOSS communities don’t need a Scrum Master to tell them to collaborate, or a planning poker deck to estimate work. They do it because the work demands it and because they give a damn about the outcome. The Manifesto didn’t fail, but in many corporate environments it was captured, productised, and hollowed out by the same forces that hollowed out architecture.
The industry is acting like AI invented slop. It didn’t. Many of us have spent entire careers trying to do excellent work, designing carefully, advocating for tests, pushing for operational discipline. But the incentive structures rarely rewarded that work.
The reality most developers live within is one of pressure. Companies acquired by private equity, performance reviews tied to velocity metrics, the ever-present threat of a PIP, and now the looming spectre of being replaced by AI entirely. A lucky few landed at the right company at the right time and exited with significant equity, but globally speaking that outcome is rare, and not something most developers outside the United States have ever experienced. Older developers who lived through the dotcom crash fared even worse. Many exercised stock options at peak valuations, only to watch the shares become worthless while still owing taxes on gains they never received.1 The promised reward for years of below-market salaries evaporated overnight.
This leads to a massive gap between the lived experience of working developers and the thought leadership that claims to speak for them. The missing conversation is telling. Most developers aren’t sitting around strategising about how to produce more shareholder value with AI. They’re trying to hang onto their jobs and decide if they’ll even have a place in the industry a year from now. That’s the honest reality, and pretending otherwise ignores the incentives that shaped everything we just described.
Slop isn’t a consequence of any single technology. It’s a consequence of incentives, and the fallout is all around us if we look hard enough. Some of it has cost safety and lives. Toyota’s unintended acceleration was traced to software with 11,000 global variables and untestable spaghetti code.2 Boeing’s 737 Max MCAS relied on a single sensor with no redundancy, and pilots weren’t told the system existed until after two crashes killed 346 people.3 These weren’t AI failures. They were caused by misaligned incentives up and down the entire industry.
Human-generated slop was slow, and that slowness gave teams time to pull the handbrake. AI-generated slop is fast, and the same processes that barely contained slow slop are now straining under the load.
Autonomous agents are now inheriting these systems and the processes that produced them. If they can’t somehow compensate for decades of accumulated shortcuts, they’ll be the next to take the blame. The pattern is familiar. Leadership blamed developers for slow delivery, then replaced them with AI. When agents fail, they’ll be blamed too. Accountability always flows downward, never back to the incentives that shaped the outcome.
The next chapter explores what happens when agents start making decisions inside these systems, and why the tradecraft that was once dismissed as overhead may be the only thing that prevents the cycle from repeating.
Secfi, “The history of employee stock options”, 2023. https://secfi.com/learn/history-of-employee-stock-options ↩︎
EDN, “Toyota’s killer firmware: Bad design and its consequences”, 2014. https://www.edn.com/toyotas-killer-firmware-bad-design-and-its-consequences/ ↩︎
IEEE Spectrum, “How the Boeing 737 Max Disaster Looks to a Software Developer”, 2019. https://spectrum.ieee.org/how-the-boeing-737-max-disaster-looks-to-a-software-developer ↩︎