Machine translations are often brought up as a gotcha whenever I criticize LLMs.
-
@Sonikku there is absolutely no way this is real
@AVincentInSpace it literally is haha. Fellowship was just as bad.
I was still dialup back in those days so I’d order my bootleg DVDs from a dude in Hong Kong and I just about died laughing when I turned on subtitles randomly
-
Machine translations are often brought up as a gotcha whenever I criticize LLMs. It's worth pointing out two things: Machine translations existed decades before LLMs, and yes, machine translations are useful. However: I would never in my life read a machine translated book. Understanding what a social media post is talking about in rough terms? Sure. Literature? Absolutely not. Hell, have you ever seen machine translated subtitles? It's absolute garbage.
@Gargron I worked with subtitle translations for years... I need to comment on this!
The main issue people working with machine translated subtitles is that people take models for translating things in a single modal – text – and applying to a multimodal media – video. Of course the results are horrible!
There are research on improving that, sure, I did some, even, but even we are FAAAR from getting them any good. Translating "The nurse aided the doctor take care of the patient." to many languages require guessing the gender of three people! LLMs will often default to male, female and male, due to bias.
But, the sad thing we have to admit: many works of art are so unpopular the only translations people will have are machine ones, from weird anime like Sazae-san, to Mastodon toots.
-
@Gargron Machine translated UIs are even worse a crime. LLMs don't have the slightest idea of the context of some random button, and (looking at Microsoft's German UI translations recently) seem to choose the worst possible word to drop into that.
-
All your bases are belong to Us
-
@Gargron I minored in linguistics in college, and a lot of exciting work was being done at the time around developing syntax models of how languages worked (and different ways humans use syntax), in part to inform machine translation models. This was more than 25 years ago. No LLMs involved.
I have not kept up with current developments in machine translation but I strongly suspect that it's built on the foundation of those decades of work actually understanding how languages function, and what maps or doesn't map. Which is completely different than expecting generative AI to create a model.@lauerhahn @Gargron Alas no. Most machine translation engines now are purely statistical. They don't bother with semantic analysis, they just brute forced a mathematical model with tons of data.
-
@Gargron Many years ago, while on holiday in Amsterdam, I bought a Dutch translation of a book by one of my favourite authors, Terry Pratchett.
In it, there was an essay, in English, by Terry, about his struggles to find a translator for the book, which was only accomplished when he realised that it wasn't just a case of taking the text and replacing it with Dutch.
No, large sections would have to be entirely re-written by the translator, to use concepts that a Dutch audience would find familiar.
And not just in Dutch, but every language.
The example he gave was one character who was experiencing the feeling of being stuck in traffic on a busy road on a Sunday afternoon, and after miles of driving, finding that the cause of the tailback was a little old lady out for her weekly drive to church in her trusty old Morris Marina, never getting above 20 MPH becuase it felt too fast.
This is something that British people are well acquanted with, but the Dutch translator had to come up with a completely different way of explaining this, because it's not something particularly prevalant over there.
It's not just about translating the words, its translating the feelings, the emotions, to something readers in another place will understand.
And LLM's fail spectacularly at that.Morris Marina wasn't completely uncommon in the Netherlands - although only a handful made it over there and interestingly it seems nearly all have been preserved as oldtimers!
But the old lady in NL would more likely ride her bicycle to church, probably at much less than 30 km/h and quite likely so would all the rest of the congregation - the translator would have definitely needed to find another concept to match this..
I got a DIY maintenance manual for my car which is in German (there isn't an equivalent one of same quality in English) and I definitely won't wholly trust an LLM to translate that (instead I print the relevant pages, go through it by hand and make notes of points that don't immediately come to mind as my German is only as good as a teenager)
-
@Gargron Many years ago, while on holiday in Amsterdam, I bought a Dutch translation of a book by one of my favourite authors, Terry Pratchett.
In it, there was an essay, in English, by Terry, about his struggles to find a translator for the book, which was only accomplished when he realised that it wasn't just a case of taking the text and replacing it with Dutch.
No, large sections would have to be entirely re-written by the translator, to use concepts that a Dutch audience would find familiar.
And not just in Dutch, but every language.
The example he gave was one character who was experiencing the feeling of being stuck in traffic on a busy road on a Sunday afternoon, and after miles of driving, finding that the cause of the tailback was a little old lady out for her weekly drive to church in her trusty old Morris Marina, never getting above 20 MPH becuase it felt too fast.
This is something that British people are well acquanted with, but the Dutch translator had to come up with a completely different way of explaining this, because it's not something particularly prevalant over there.
It's not just about translating the words, its translating the feelings, the emotions, to something readers in another place will understand.
And LLM's fail spectacularly at that.@stuartb @Gargron@mastodon.social
Exactly!
I did some work for a technical translation company after I retired (not translating I hasten to add) and the skill was making the language relevant to the target audience. To do that the translator had to have both a knowledge of the subject matter, the language and what sort of person would be using the translation.
And they wondered why Google wasn't good enough, and why we charged what we did. -
@grishka @Gargron Google Translate switched to using the same tech LLM uses. Actually, it's the opposite: the transformer model that LLM uses was created for translation first.
If you are going to compare both, since the tech is pretty much the same, the main change between then is how they are trained: people often use LLMs that are trained to behave chatbots for translation, it create biases that are not present in models that are only trained for translation, mainly, LLMs are prone to "ignore all instructions".
But the tech is pretty much the same: transformer models deal way better with context in comparison with older models.
-
I have the impression that primarily anglophone people don't read as much translated literature, because so much good literature already exists in their language, so this issue may not be as familiar within that demographic. As someone who did not grow up anglophone, I can tell you there is a world of difference between a good and a bad translation even when done by humans. Machine translations are not even on the scale.
@Gargron Preach!
-
Machine translations are often brought up as a gotcha whenever I criticize LLMs. It's worth pointing out two things: Machine translations existed decades before LLMs, and yes, machine translations are useful. However: I would never in my life read a machine translated book. Understanding what a social media post is talking about in rough terms? Sure. Literature? Absolutely not. Hell, have you ever seen machine translated subtitles? It's absolute garbage.
@Gargron deepl isnt that awful tbh
-
@Gargron Machine translated UIs are even worse a crime. LLMs don't have the slightest idea of the context of some random button, and (looking at Microsoft's German UI translations recently) seem to choose the worst possible word to drop into that.
@galaxis @Gargron To be fair, it happens with human translations too: I fixed some translations in open-source projects in which the translation interface only showed the text, but not the context, and the previous translator translated it wrongly. Example: Wikipedia had "Large (width)" (Largo) translated to "Huge" (Grande). If you check the edit history for this entry in Wikimedia, that's my name fixing this issue. But, sure, it's mostly common in machine translations, as I commented in some other toot in this thread.
-
I have the impression that primarily anglophone people don't read as much translated literature, because so much good literature already exists in their language, so this issue may not be as familiar within that demographic. As someone who did not grow up anglophone, I can tell you there is a world of difference between a good and a bad translation even when done by humans. Machine translations are not even on the scale.
@Gargron Douglas Hofstadter's 2018 assessment of the state of machine translation holds up remarkably well (he agreed with you):
-
Machine translations are often brought up as a gotcha whenever I criticize LLMs. It's worth pointing out two things: Machine translations existed decades before LLMs, and yes, machine translations are useful. However: I would never in my life read a machine translated book. Understanding what a social media post is talking about in rough terms? Sure. Literature? Absolutely not. Hell, have you ever seen machine translated subtitles? It's absolute garbage.
@Gargron
For many years, YouTube has been filled with tragic machine-translated captions, even before the new AI trend of the past few years. Not only machine translated ones, but also machine generated ones converted from spoken words.The problem with any machine translations or generated captions is that people aren't proofreading them. Viewers get garbage and the machines didn't get proper feedback.
-
@Gargron Many years ago, while on holiday in Amsterdam, I bought a Dutch translation of a book by one of my favourite authors, Terry Pratchett.
In it, there was an essay, in English, by Terry, about his struggles to find a translator for the book, which was only accomplished when he realised that it wasn't just a case of taking the text and replacing it with Dutch.
No, large sections would have to be entirely re-written by the translator, to use concepts that a Dutch audience would find familiar.
And not just in Dutch, but every language.
The example he gave was one character who was experiencing the feeling of being stuck in traffic on a busy road on a Sunday afternoon, and after miles of driving, finding that the cause of the tailback was a little old lady out for her weekly drive to church in her trusty old Morris Marina, never getting above 20 MPH becuase it felt too fast.
This is something that British people are well acquanted with, but the Dutch translator had to come up with a completely different way of explaining this, because it's not something particularly prevalant over there.
It's not just about translating the words, its translating the feelings, the emotions, to something readers in another place will understand.
And LLM's fail spectacularly at that. -
I have the impression that primarily anglophone people don't read as much translated literature, because so much good literature already exists in their language, so this issue may not be as familiar within that demographic. As someone who did not grow up anglophone, I can tell you there is a world of difference between a good and a bad translation even when done by humans. Machine translations are not even on the scale.
@Gargron I've translated two commercial games in my native language (French), and I've grown to appreciate games that have French translations made by actual humans (especially those with dialogues), because there's always a bunch of stuff that any machine translation algorithm is never going to pick up and, when done right, really makes it worth playing
-
Machine translations are often brought up as a gotcha whenever I criticize LLMs. It's worth pointing out two things: Machine translations existed decades before LLMs, and yes, machine translations are useful. However: I would never in my life read a machine translated book. Understanding what a social media post is talking about in rough terms? Sure. Literature? Absolutely not. Hell, have you ever seen machine translated subtitles? It's absolute garbage.
@Gargron This.
-
@virgilpierce @Gargron
There's an old joke from the 1960s about machine translation of the saying "the spirit is willing, but the flesh is weak" from English to Russian and then back again.
The result was "the vodka is good but the meat is rotten."I've heard that one in German with some equivalent of "but the steak is not quite done".
-
Machine translations are often brought up as a gotcha whenever I criticize LLMs. It's worth pointing out two things: Machine translations existed decades before LLMs, and yes, machine translations are useful. However: I would never in my life read a machine translated book. Understanding what a social media post is talking about in rough terms? Sure. Literature? Absolutely not. Hell, have you ever seen machine translated subtitles? It's absolute garbage.
@Gargron@mastodon.social I think it's actually perfectly fine to just accept they fill their niche of roughly guessing what something means despite the fact modern translation models work pretty much the same as actual large language models? People trying to gotcha over this are usually not acting in good faith anyway (or dangerously reducing/simplifying things)
-
@Gargron I think anglophones experience start difference between good and bad translations more often through video games
-
All your bases are belong to Us
@gabboman@gabboman.xyz @aeva@mastodon.gamedev.place @Gargron@mastodon.social tbf that's not translation, that's japanese speakers writing english
And IMO broken english in an old videogame is so much better than soulless LLM translation. Like yeah, it may be jibberish, but it's a part of the charm
