LLMs Are the Ultimate Demoware
Disclaimers
- This is an article about AI. If you’re tired of talking about AI please close this tab and don’t yell at me!
- I’m not an economist, or an AI expert, just a chart enthusiast and programmer.
Definitions
“Software” is a program that runs on your computer that allows you to accomplish a task. Software is “good” if it frequently allows you to accomplish said task without too much trouble.
“Demoware” is a type of software that looks “good” during a demonstration. That is, it can help accomplish a simulated task during a demo, but then fails to consistently help in completing tasks when deployed for daily use.
“LLMs” are Large Language Models, like GPT-5, Claude, etc. People typically interact with LLMs through chatbots, and in specialized fields like programming, through dedicated software such as the Cursor development environment.
How does a software demo work
When developing demoware, which I have done in the past, you develop a basic prototype of software and find a particular scenario in which it performs quite well.
I’ve made plenty of dashboards that were largely demoware. In the demo, you can say “With this new dashboard, we’ll look at X salesperson. The dashboard shows they’ve been performing significantly above average, so maybe they deserve a raise!”. Managers in the room nod. “This will lead to great efficiencies! I am good at purchasing technology!”
Once the demoware has been purchased by management, an email goes out to staff that everyone should log in and use the dashboard. In day to day use, when looking at data that wasn’t cherry picked for the demo, the dashboard fails to deliver on the promised insights or time savings. In time, the dashboard is ignored.
Despite the lack of real value, the software was purchased and the software vendor has already made their money, at least in the short term.
LLMs are the ultimate demoware
Traditional demoware, like a useless dashboard, takes a fair amount of time and energy to engineer.
LLMs, on the other hand, can provide seemingly excellent results in a demo environment with minimal work. LLMs have a surface level understanding of essentially every domain, and with a bit of prompting, can create an excellent demo for many use cases.
- “Look at how our AI tutor explains derivatives!”
- “Look at how our AI support agent resolves a common issue!”
- “Look at how our AI coding assistant creates…… a dashboard!”
You get the idea. However, LLMs are shallow. Outside of the demo, there are very real downsides to consider.
What happens when the student doesn’t want to engage with an AI tutor? What if the AI support agent is faced with an uncommon issue and makes a mistake? What happens when the AI coding assistant can’t make something more complicated than a useless dashboard?
As mentioned before, demoware is easy to buy. After all, the demo looked great! The adoption of LLM demoware is aided by the constant bombardment of AI hype in the tech media and Linkedin. “If there’s so much hype, clearly this demo that I’m seeing must lead to real outcomes, right?”
A year ago, it might have been correct to give AI software vendors the benefit of the doubt. Models seemed to be improving rapidly, and a standard argument for why LLM-based software was more than demoware is that sometime after the demo, the models would improve and this would in turn transform the demoware into good software. However, models are no longer improving rapidly, and this argument is running out of steam.
We’re past the honeymoon, “look at that awesome demo!” phase of the AI adoption curve. If you want me to adopt your AI software, I need it to provide real value. Here’s a good test of whether software is demoware or something more: If you took the software away, could you do your job?
If I couldn’t use email, spreadsheets, or my invoicing software, it would be substantially harder to run my business. Can we say the same about the AI software being sold today? For me, the answer is no, and I think the tech industry is in a lot of trouble as more businesses come to that conclusion.
Modern software isn’t purchased once and used forever. Modern software businesses rely on recurring revenue, which is predicated on customers seeing real value in real scenarios, beyond the initial demo. What happens to the hundreds of billions of dollars of GPUs the industry has purchased when nobody chooses to renew?