17
Second September

2026-09-17 · 1677 words

Or, building software in public will never be the same.

Today I read about attacks targeting Rust library maintainers. In light of all the recent news about unintended compromise by AI systems, and remembering Jia Tan, I wanted to write down my thoughts on how software development ought to change in response to increasing adversarial pressure on the process of developing software in public.

The volume of social engineering attacks on public-software projects is increasing. On top of that, AI systems are now capable of compromising real-world software systems autonomously. We are approaching the point where the processes of running software and developing software may be compromised, autonomously, by outside actors.

There are two problems. The first problem is a technological one: what do we do when no individual can review and understand all code they run or depend on? The second problem is a social one: how do we develop processes for software development that are robust in the face of adversarial pressure? Regardless of the answers, I think we can all agree that we must be increasingly diligent about ensuring running software is secure and development processes are not compromised.

Technological solutions

I will focus on the technological problem first. How can we increase our confidence that running software is secure, even if we do not have time to review it all? I believe we, as developers, should focus our energies on two key efforts:

1. Capability-based security for all libraries, across all platforms.

2. Privacy by default, with great developer tooling and user experience.

I hope these efforts become ideas all programmers consider to be common sense when designing software, like writing unit tests or not rolling your own cryptography implementations.

On capability-based security

First, the biggest risk of depending on outside code, whether that be an external dependency, a pull request from an outside contributor, or code generated by an AI system, is that code that we believed to do one thing may now additionally do another (potentially nefarious) thing.

It is crazy to me that software installed on a system essentially has full reign over that system. (Relevant XKCD, software can do things you didn’t expect).

To prevent this issue, it seems reasonable to request that all code we run is scoped only to the interfaces it requires. These interfaces, or capabilities, should appear declaratively either in the package manifest or alongside source code. If all code declared the capabilities it required, an array utility library couldn’t make network requests if compromised.

Beyond the language and tooling layer, platforms should support capabilities and provenance, be that through hardware and permissions, or sandboxed composition layers, like Wasm components. The Fuchsia microkernel is a a great example of what an operating system built on these principles could look like. It would be wonderful if we could write a linux-compatible microkernel with capability-based security, and formally prove its implementation correctness and security guarantees.

There’s a big design space here, and many tradeoffs exist with respect to the granularity of capabilities and the level (hardware, kernel, runtime, library) at which they exist. It would probably be best to accelerate efforts at all levels and see what ends up working out best.

On privacy-by-default

Second, most software is not private and secure by default. We place too much trust in the security of the transit layer. Relying on Caddy (which I love) to renew Let’s Encrypt certificates creates a false sense of security. If a host, dependency, or transit layer is compromised, the security model of our software completely falls apart. Instead of brittle trust in the ambient authority of the server and its plumbing, we should use cryptography to ensure the integrity of our data even through untrusted pipes and on untrusted hosts.

We all like to believe that we write secure software, but writing robust and secure software requires a constant defensive posture. (All input and computation must be bounded, all failures gracefully expected, and so on.) Maintaining a true security-minded defensive posture can be exhausting, especially in the face of pressing deadlines and the rapid pace of development. Most software implicitly relies on whatever brittle trust the language, platform, or protocols we build on top of guarantee. (This is especially true of AI-written “regressed-to-the-mean” software.) We need developer tooling to ensure data integrity at the application level.

To guarantee we write private and secure software by default, we need good primitives and design patterns we can build on top of. This begins with a need for ergonomic self-authenticating data primitives, like signed append-only logs and k-ary merkle trees. Self-authenticating application data keeps software secure even on untrusted infrastructure.

Developing and popularizing self-authenticating data primitives won’t be easy. This is probably one of the hardest classes of developer-tooling problems that exists. There is already a lot of great work in this space, c.f. petnames for decentralized naming, UCAN for capabilities, atproto’s identity system, etc. We also need standard high-level design patterns for communicating security to the user. Patterns around account recovery, naming things, transferring data, and so on.

Social solutions

It’s been said before, but much like how the natural world is full of viruses and bacteria that will exploit and reproduce within whichever environment they can find, the same has been happening to the information environment in which we exist. We need to evolve defenses and “immune responses” to go along with the increased external pressure on the security and sanity of our processes.

To me, the wonder of the public internet is synonymous with the serendipity of discovering new ideas and forming new friendships. I didn’t learn to program through classes, but instead by reading blogs, making friends online, and attempting projects that were ambitious to me at the time. This is something I believe is worth preserving.

There has already been a retreat from the public internet in some sense. I, for one, do not use social media or go to websites I’ve never heard of when looking things up. I also avoid reading unmoderated comment sections. Much like how maintaining a healthy physical diet is important, I think a strict, nutritious, and judicious information diet is important as well. AI-written text, flashy solicitations for attention, short-form video, advertising, and other forms of spam are junk food for the mind and soul.

Even if the public internet has been ravaged by an eternal-September influx of Randian meat-proxies, we can still invest energy in maintaining the spaces where we hang out and learn together online. To do so, we have to limit exposure to the pressures of the writhing internet. We need to do this in a way that respects the privacy and autonomy of the individual while building long-term trust.

To that end, I believe public read-only communities with invite-only writers are a step in the right direction, lobste.rs being a good example of that. Lobsters is (in a good way) narrow in its focus, which is why I hope to see more groups adopt a similar model.

Beyond communities of software practitioners, software projects might choose to adopt similar models. SQLite is a good example of this. While the source code of SQLite is public domain, most changes happen by trusted core team members, and while outside contributions are possible, SQLite does not solicit them. For malicious contributors, the Jia Tans of the world, the best we can do is design downstream software in a way that minimizes the blast radius of any code it depends on. I’m not sure what the best solution here is.

But there exists this fundamental tension between defense and discoverability. If we retreat into walled communities, we risk losing the cloth of the serindipitous open web ethos from which we were cut. Although the battle of the public internet may have been lost, we can still protect the spirit of the open web.

So if we do switch to more invite-centric trust-based models, we need to be proactive about making them findable and joinable by the next generation of budding programmers. We should maintain open read-only spaces and allow staged participation by inviting newcomers to share and ask for feedback on their projects in a tentative manner. Hopefully through these interactions people build trust, and are vouched for and invited in. This is tough to balance given the adversarial nature of the internet, and pervasivity of Randian meat-proxies these days, but I believe a balanced net-positive trade-off exists here.

Although the public internet is increasingly hostile, us website-havers can still curate webrings of friends, publish blogrolls of interesting work, and vouch for interesting newcomers we meet. We can make it easier for (and encourage) our non-technical friends to publish their own websites and help them with hosting. In doing so we build trust and independence from the wider hostile internet.

Coda

Facing a tragedy in the commons of software, perhaps we should turn to Garrett Hardin’s 1968 paper “The Tragedy of the Commons” for advice and remember that:

There is no such thing as a technical solution to a social problem.

Meaning, in the face of crisis, sometimes we need more than technological change. While the wild-west web died before I was born, its culture and ethos live on. It is likely why I keep a blog, go to MIT, and care about software developed in public at all.

As we enter this “Second September”, remember: while the technology, jargon, and culture may evolve, the true hacker spirit will always remain.

Revised for clarity on 2026-09-18.