Amazon just changed an actor's mouth. In more than 240 countries.
The piece
Amazon just changed an actor's mouth. In two hundred and forty countries. Last week Prime Video turned on AI lip-sync for Maxton Hall. It reads the actor's mouth movements on screen, and then alters them to match the English dub. Last week Prime Video turned on AI lip-sync for Maxton Hall. It reads the actor's mouth movements on screen, and then alters them to match the English dub.
Read Amazon's own wording carefully, because they were careful. It matches the mouth to “human-recorded” dialogue. The dub actors are still people. The voice kept its job. The mouth is what changed. The dub actors are still people. The voice kept its job. The mouth is what changed.
This is their most-watched international series. Seasons one and two are live globally right now, and season three lands in more than two hundred and forty countries on December ninth. This is their most-watched international series. Seasons one and two are live globally right now, and season three lands in more than two hundred and forty countries on December ninth. This is their most-watched international series. Seasons one and two are live globally right now, and season three lands in more than two hundred and forty countries on December ninth.
This is their most-watched international series. Seasons one and two are live globally right now, and season three lands in more than two hundred and forty countries on December ninth. This is their most-watched international series. Seasons one and two are live globally right now, and season three lands in more than two hundred and forty countries on December ninth. Their VP of Technology says they're creating a more seamless, immersive way to enjoy global content,
all done, quote, “under creative oversight to ensure the integrity of their artistic vision is preserved.” And then somebody replying to Variety said the thing nobody at Amazon said. Quote: “It's the job of translators to provide a translation that matches the actors' mouth movements as much as possible without losing meaning. It's a whole thing. It's been done in non-English-speaking countries for decades.”
Quote: “It's the job of translators to provide a translation that matches the actors' mouth movements as much as possible without losing meaning. It's a whole thing. It's been done in non-English-speaking countries for decades.” That's the whole story right there. This problem was already solved. Not by a machine, by a craft.
Writing a dub line that lands on the mouth shape and still means what it meant. People have spent entire careers getting good at that. So AI didn't solve something nobody could do. It routed around a skill. It made the constraint stop mattering. So AI didn't solve something nobody could do. It routed around a skill. It made the constraint stop mattering.
Instead of writing a line that fits the mouth, now you change the mouth to fit any line. I've spent thirty years learning to solve problems inside a constraint. That's most of what craft is. And what keeps happening now isn't that somebody does it better than me. It's that the constraint quietly gets deleted, and the skill I built to beat it goes with it.
I don't have a clean answer for that one yet. I'm just naming it. To be straight with you: nobody lost a job here that I can point to. The dub actors are human and Amazon says so twice. Their earlier pilot paired AI with localization pros doing quality control.
The dub actors are human and Amazon says so twice. Their earlier pilot paired AI with localization pros doing quality control. And honestly, it fixes something audiences have complained about for fifty years. This isn't a layoff story. It's a harder one. Go find one constraint you're genuinely expert at working around.
Whatever's on the other side of it (that's the thing to learn. Here, it isn't mouth-matching. It's what the line should actually mean. That part, they still can't do. Whatever's on the other side of it) that's the thing to learn. Here, it isn't mouth-matching. It's what the line should actually mean. That part, they still can't do. Comment the one constraint you're expert at working around, I go through every one of them.
Source: Amazon's newsroom (9/9), Slator (9/9). Clip: official Maxton Hall S2 trailer, Prime Video.