New Blog: What Would A Serious AI Product Look Like?
-
New Blog: What Would A Serious AI Product Look Like?
Every “AI” tool is missing critical features that you would need if you wanted to do real work with them. What would those look like, if they existed?
@glyph wrt the natural-language interfaces... natural language is a *terrible* way to deal with a lot of things. Particularly in the sciences and engineering and math and computing, where we have, for the entire lifetimes of those as serious fields, gone *out of our way* to develop entire new notation systems and standards to try to deal with them without natural language...
Like, pretty often you can have one page of math notation, or you can have a 60-page natural language explanation. -
I got tired of being told that, as a skeptic, I don't know anything about how to actually use an LLM, that I just need to use it more in order to understand the workflows. I understand the workflows. They are either too dangerous or they don't exist.
@glyph I daily see the output extruded with LLMs by enthusiastic supporters who have been eagerly using the tools to generate code for a couple of years now and…using it more does not cause the output to be good.
-
That sounds a lot like: "You can't understand how to properly use drugs if you don't use the drugs enough." ( Please substitute the legal or otherwise performance-enhancing substance of your choice.)
I thought it was just a tool. Shouldn't tools include proper documentation or training. Maybe some descriptions of when they're the right tool for the job and when they're not?
Yes, tools have a specific utility and purpose. Yes, you may gain some knowledge on the intended utility and use case by reading about the tools. No, you don’t get an understanding (aka, knowledge + experience) just by reading.
For example, I think most people know what a circular saw, jigsaw, and router look like and they have some knowledge of how they function.
Most people don’t have the understanding of how to make a lap joint using said tools, or why.
-
This post will do exactly nothing to stop the flood of such replies. But a man can dream. And now at least I can just drop a link and move on.
@glyph totally agree about how bots shouldn't be allowed ot use first-person language. In the last Washington state legislative session, there was a bill prohibiting chatbots from using several manipulative techniques with minors, and I pointed out that using first-person language was even more manipulative than the techniques they were banning so they really should include that as well. For that matter I also suggested that only prohibiting applying these techniques to minors legitimized applying them to seniors, adults with mental health problems and other vulnerable groups, so the extensions should apply to everybody -- and several researches who had done studies on the harms of AI chatbots also suggested that. But no. It's almost as if they were more concerned about looking like they were doing something to reduce harms than about actually doing something to reduce harms.
-
@glyph totally agree about how bots shouldn't be allowed ot use first-person language. In the last Washington state legislative session, there was a bill prohibiting chatbots from using several manipulative techniques with minors, and I pointed out that using first-person language was even more manipulative than the techniques they were banning so they really should include that as well. For that matter I also suggested that only prohibiting applying these techniques to minors legitimized applying them to seniors, adults with mental health problems and other vulnerable groups, so the extensions should apply to everybody -- and several researches who had done studies on the harms of AI chatbots also suggested that. But no. It's almost as if they were more concerned about looking like they were doing something to reduce harms than about actually doing something to reduce harms.
@jdp23 man I have so much to say about this but I have already written so many words today. suffice it to say I couldn’t agree more
-
Yes, tools have a specific utility and purpose. Yes, you may gain some knowledge on the intended utility and use case by reading about the tools. No, you don’t get an understanding (aka, knowledge + experience) just by reading.
For example, I think most people know what a circular saw, jigsaw, and router look like and they have some knowledge of how they function.
Most people don’t have the understanding of how to make a lap joint using said tools, or why.
@Kehvarl @glyph Now, there is no doubt in my mind that Glyph has both the knowledge, experience and understanding he claims to have.
But just because you’ve read about LLMs and agentic workflows doesn’t mean you understand how appealing and debilitating—in one word, how corrupting—they are.
It’s almost like all those jokes and stories about genies that kind of understood what you asked for and then took wild liberties with one specific clause.
Today you get 3 wishes. Tomorrow 10k tokens.
-
Yes, tools have a specific utility and purpose. Yes, you may gain some knowledge on the intended utility and use case by reading about the tools. No, you don’t get an understanding (aka, knowledge + experience) just by reading.
For example, I think most people know what a circular saw, jigsaw, and router look like and they have some knowledge of how they function.
Most people don’t have the understanding of how to make a lap joint using said tools, or why.
-
@glyph totally agree about how bots shouldn't be allowed ot use first-person language. In the last Washington state legislative session, there was a bill prohibiting chatbots from using several manipulative techniques with minors, and I pointed out that using first-person language was even more manipulative than the techniques they were banning so they really should include that as well. For that matter I also suggested that only prohibiting applying these techniques to minors legitimized applying them to seniors, adults with mental health problems and other vulnerable groups, so the extensions should apply to everybody -- and several researches who had done studies on the harms of AI chatbots also suggested that. But no. It's almost as if they were more concerned about looking like they were doing something to reduce harms than about actually doing something to reduce harms.
-
New Blog: What Would A Serious AI Product Look Like?
Every “AI” tool is missing critical features that you would need if you wanted to do real work with them. What would those look like, if they existed?
@glyph all of this so much. Thank you for putting the words to the page!
-
New Blog: What Would A Serious AI Product Look Like?
Every “AI” tool is missing critical features that you would need if you wanted to do real work with them. What would those look like, if they existed?
@glyph gratitude for this post!
// I've made a few tools for myself using LLMs - scripts or programs. *If* I use an LLM at runtime, it's invoked through a standard UI action - never chat, and the underlying context is configurable. Had not realized until reading your words that this isn't standard practice.
The harnesses from "frontier" labs are like old science kits for kids with dangerous materials. One must infer that people involved in building them are unserious and/or delusional.
-
New Blog: What Would A Serious AI Product Look Like?
Every “AI” tool is missing critical features that you would need if you wanted to do real work with them. What would those look like, if they existed?
@glyph I do not disagree but I am not sure LLMs can actually still "work" with these. Or that the current architecture can support these measures.
Which... Well reach the same point as your conclusion isn't it....
-
@glyph I do not disagree but I am not sure LLMs can actually still "work" with these. Or that the current architecture can support these measures.
Which... Well reach the same point as your conclusion isn't it....
@Di4na yeah, that was kinda the point. I guess it was well-written if I brought you along with me to the end before you got there:)
-
I got tired of being told that, as a skeptic, I don't know anything about how to actually use an LLM, that I just need to use it more in order to understand the workflows. I understand the workflows. They are either too dangerous or they don't exist.
@glyph Did they just "skill issue lol" your scepticism? Yeah, seems right.
-
@Di4na yeah, that was kinda the point. I guess it was well-written if I brought you along with me to the end before you got there:)
@glyph Oh I was there at the start, and I felt that point that LLM are *fundamentally* not able to do these things that are needed is something I did not feel in there, but yes. Well written
and thank you for putting it out! -
New Blog: What Would A Serious AI Product Look Like?
Every “AI” tool is missing critical features that you would need if you wanted to do real work with them. What would those look like, if they existed?
@glyph I think there should be more applications of an LLM's outputs and internal state besides just answering prompts. Such as spellchecking, next word prediction (for when you're out of ideas on how to finish a sentence) or detecting unexpected phrasings. -
New Blog: What Would A Serious AI Product Look Like?
Every “AI” tool is missing critical features that you would need if you wanted to do real work with them. What would those look like, if they existed?
@glyph I like this post a lot. I have one minor piece of feedback if you'd like to hear it, if not feel free to ignore.
One mitigation you suggest is an in house therapist at companies to check for AI-usage-related health issues. As someone who has been in therapy outside of work I'm skeptical of this as the solution. I suspect you're looking for an analogy to an in house doctor for diagnosis of physical injuries. If that's what you mean (a) you probably mean psychologist rather than therapist but (b) I don't trust my employer with that knowledge of myself anyway, as the incentives are misaligned, and maybe something like worker's comp is the model to go for.
-
New Blog: What Would A Serious AI Product Look Like?
Every “AI” tool is missing critical features that you would need if you wanted to do real work with them. What would those look like, if they existed?
@glyph Sometimes, in the middle of one of your delicious threads, I recall that I started following you after "I Think I'm Done Thinking About genAI For Now" and how I'm glad that turned out to be a lie.
-
@glyph totally agree about how bots shouldn't be allowed ot use first-person language. In the last Washington state legislative session, there was a bill prohibiting chatbots from using several manipulative techniques with minors, and I pointed out that using first-person language was even more manipulative than the techniques they were banning so they really should include that as well. For that matter I also suggested that only prohibiting applying these techniques to minors legitimized applying them to seniors, adults with mental health problems and other vulnerable groups, so the extensions should apply to everybody -- and several researches who had done studies on the harms of AI chatbots also suggested that. But no. It's almost as if they were more concerned about looking like they were doing something to reduce harms than about actually doing something to reduce harms.
-
@glyph I like this post a lot. I have one minor piece of feedback if you'd like to hear it, if not feel free to ignore.
One mitigation you suggest is an in house therapist at companies to check for AI-usage-related health issues. As someone who has been in therapy outside of work I'm skeptical of this as the solution. I suspect you're looking for an analogy to an in house doctor for diagnosis of physical injuries. If that's what you mean (a) you probably mean psychologist rather than therapist but (b) I don't trust my employer with that knowledge of myself anyway, as the incentives are misaligned, and maybe something like worker's comp is the model to go for.
@anne I did not so much mean that this is the best way to do it, as I meant that doing SOMETHING like this would indicate a level of seriousness about the problem. There are plenty of other mental health supports they could attempt instead. But instead of that, we have … nothing. I have yet to hear of a single org offering specific mental health resources along with an AI mandate.
-
New Blog: What Would A Serious AI Product Look Like?
Every “AI” tool is missing critical features that you would need if you wanted to do real work with them. What would those look like, if they existed?
@glyph excellent post. I love how your description of how results should be presented for research is almost exactly what Google search looked like 10 years ago: a list of links with relevant excerpts, no faulty "summaries".
One problem that persists in this use case is bias, but regular old search had the same problem (it didn't have other problems like energy use to the same degree).
