Thread by Geoffrey Litt
- Tweet
- Mar 19, 2023
- #UserExperience #ComputerProgramming
Thread
What if -- despite all the hype -- we are in fact underestimating the effect LLMs will have on the nature of software distribution and end-user programming? some early, v tentative thoughts: 1/
it seems likely to me that all computer users will soon have ability to 1) develop small software tools from scratch, 2) describe *changes* they'd like made to existing software they're already using. what will this mean for software ecosystems?? 2/
you, skeptic: "nooo but the LLM-generated software will be lower-quality than handcoded software by pro teams: filled with bugs, uglier, bad... 😡 "
Yeah totally true... also all qualities which also apply to spreadsheets vs "real apps"! 3/
Yeah totally true... also all qualities which also apply to spreadsheets vs "real apps"! 3/
you again: "most computer users aren't nearly as good as programmers / designers at turning fuzzy ideas into real tools!!"
yeah also fine, the LLM will help them develop the spec, people will improve, and yeah, maybe most people won't get great but doesn't matter 🙂 4/
yeah also fine, the LLM will help them develop the spec, people will improve, and yeah, maybe most people won't get great but doesn't matter 🙂 4/
ok now that we've handwaved past those basic objections we can get to the interesting stuff 😉
until now, programming was by far primary bottleneck to customizing / creating software.
if that bottleneck is diminished, what pressure does that apply in the ecosystem? 5/
until now, programming was by far primary bottleneck to customizing / creating software.
if that bottleneck is diminished, what pressure does that apply in the ecosystem? 5/
The status quo context: software is mostly hostile to customization today at the *architectural* level. Except for small plugin + extension surfaces, general assumption is that software is centrally developed and one-size-fits-all. 6/
Notably, even an accomplishment as radical as removing the programming bottleneck doesn't automatically solve the issue!!
Eg: a perfect AGI-powered browser-extension-builder bot can't edit the code or data hidden away on the server. 7/
Eg: a perfect AGI-powered browser-extension-builder bot can't edit the code or data hidden away on the server. 7/
In other words: today, even if you could hire a team of personal programmers to mod all your software for you, they'd face tons of structural barriers.
this thought experiment hasn't been very relevant until now because coding is too hard, but maybe now it will be moreso? 8/
this thought experiment hasn't been very relevant until now because coding is too hard, but maybe now it will be moreso? 8/
So if this hypothesis were to play out, what pressures might be applied happening to the system..?
First: maybe consumer demand for malleable software would increase. People will prefer OS / apps that play nice w/ their personal software-building assistant. 9/
First: maybe consumer demand for malleable software would increase. People will prefer OS / apps that play nice w/ their personal software-building assistant. 9/
Perhaps could look something like more local-first software with smalltalk-style runtime modification and extension abilities, and less like cloud saas apps prebaked and sold to the masses. 10/
Could also imagine biz teams building more software in-house and buying less software off the shelf. Or at least doing a lot more last-mile customization. A return to the origins of software dev, before standardized software! 11/
I think the fun and interesting part here is that even if LLMs do, incredibly, deliver on the promise of democratizing programming, there are many other foundations that'll need to fall into place to fully realize the vision of personal, malleable software...
12/
12/
Like everyone I have no idea where this is all headed, but trying to keep an open mind about the possibilities for empowering regular people to use computers, and avoid gatekeeping even if that's a bit scary for future programmer job prospects...
13/
13/
A relevant historical note:
Software started out custom-developed within businesses; it only became a mass-produced commodity when demand grew past avail supply.
(excerpts from @MidasNouwens's excellent dissertation on negotiable software)
14/
Software started out custom-developed within businesses; it only became a mass-produced commodity when demand grew past avail supply.
(excerpts from @MidasNouwens's excellent dissertation on negotiable software)
14/
@MidasNouwens Another fun puzzle to think about: seems like AI will lower the cost of rough interactive prototyping -- "sketching in code" -- dramatically faster than cost of making a fully airtight prod implementation. What will be implications of that? 15/
@MidasNouwens The line btwn prototype / production is already blurry today (startup founder shipping crappy v0 before hiring engineers) but maybe LLM could push this further?
I'm very skeptical of "your backend is just an LLM" but it does make sense as a way to quickly try an idea out
16/
I'm very skeptical of "your backend is just an LLM" but it does make sense as a way to quickly try an idea out
16/
@MidasNouwens And so maybe you could imagine a teacher spends a lunch break actually making a little app to help them in their classroom, instead of just dreaming of someday finding someone to build it for them... and then the prototype iteratively becomes real?
17/
17/
@MidasNouwens So many tricky challenges when novices are using this tech instead of pro devs.
I think this 2011 paper from @amyjko et al beautifully summarizes the challenges... how to approach requirements specs, validation, etc?
web.media.mit.edu/~lieber/Publications/End-User-Software-Engineering.pdf
18/
I think this 2011 paper from @amyjko et al beautifully summarizes the challenges... how to approach requirements specs, validation, etc?
web.media.mit.edu/~lieber/Publications/End-User-Software-Engineering.pdf
18/
@MidasNouwens @amyjko And I guess a meta-question for myself and anyone who wants this vision:
Given that there will be *tremendous* energy poured into making professional devs more efficient using AI...
what other work that can be done to nudge things towards end-user-malleable software?
19/
Given that there will be *tremendous* energy poured into making professional devs more efficient using AI...
what other work that can be done to nudge things towards end-user-malleable software?
19/
@MidasNouwens @amyjko A few that are top of mind for me:
- runtimes that support live modification / branching of running software
- helping novices iterate on their requirements
- extension architectures to make mods integrate as smoothly as possible into existing experience
20/
- runtimes that support live modification / branching of running software
- helping novices iterate on their requirements
- extension architectures to make mods integrate as smoothly as possible into existing experience
20/
@MidasNouwens @amyjko These are age-old problems of malleable software that many people have been exploring for decades. They just feel more promising to work on now because the programming bottleneck may at last be opening up! 🤓
21/
21/
@MidasNouwens @amyjko OK phew, that's enough for one Twitter thread -- I've been writing about LLMs for end-user programming and software customization on my email newsletter / blog, sign up there to follow along:
buttondown.email/geoffreylitt
22/22
buttondown.email/geoffreylitt
22/22
