AI Resurrection (and other AI thoughts)

[ThaOGDreamWeaver]

Not strictly a comic book/nerd movie topic, but...

 

California has recently passed two AI bills, AB2602 - preventing studios from using a performer's likeness or even movements without consent for AI recreations - so covering MoCap as well - and AB1836, which covers dead performers in the same way, passing the rights to their estate. 

(FYI: that uncanny-valley horror in Rogue One may have been an insult to Peter Cushing's memory, but LucasFilm did seek permission from his heirs first. Not everybody does.)

 

TBH, I'm not sure studios should be using generative AI in that way at all. I'm sure the quality will improve exponentially over time with learning, but it's still much more expensive than actually hiring an actor. And if the original actor has passed away, giving somebody new a crack at it can add to that performance rather than trying to teach a machine how to be a bad copy.

 

What do you lot think about deepcloning in movies? Do you want to see a digital Hugh Jackman being Wolverine until he's ninety (or more)?

Edited September 1 by ThaOGDreamWeaver

[Techwright]

I think it really depends on the project.  I'm actually surprised this is coming up now.  It's been, what, 15 years ago or so when a beer commercial used John Wayne's likeness from one of his war films to pitch a product?  I was of the understanding even back then it was considered extremely bad form, and a potential lawsuit, to use a deceased actor's likeness without the estate's blessing. 

 

It made incredible sense to use Peter Cushing's likeness in Rogue One (and I happened to like the results.  It was the Princess Leia one I found cheap.).  It made less sense for Laurence Olivier to be cast posthumously as the big bad in Sky Captain and the World of Tomorrow.  Boris Karloff or Vincent Price might have been a better fit, but I think it was intended as an homage to Sir Larry.  

 

I  think using an AI version of an actor has to make sense in context.  While I could see AI Jackman as Wolverine (and how tall would you make him, hmm?), I'd probably opt for another actor, but then we've largely been conditioned for that with comic book roles.  How many Batmen have we had?  How many Supermen?  I could see an AI Marlon Brando as Jor-El on a future Superman project.  It would be an homage, and I could respect it as such.  (Plus the parody of him in Megamind was hilarious.)  

 

The question, I suppose is how to define "make sense in context".   I, for one, would love to see a superb new Sherlock Holmes movie with the energy of the modern interpretations, but starring "Basil Rathbone" and "Nigel Bruce" (and where "Nigel" plays Watson as whip smart, and not a kindly bumbling yes-man.)  To me, that makes sense in context, assuming all parties are in agreement to do it.  The real Basil WAS Sherlock Holmes, despite other good performers, until Jeremy Brett came along.  Similarly, I'd like to finally see a quality Lone Ranger film with "Clayton Moore" and "Jay Silverheels".  Oh, and I do mean "superb" and "quality".  If you're going to necro the performers, have the decency to make certain the script is outstanding first.  Jason Isaacs just performed Cary Grant in Archie.  Geoffrey Rush played Peter Sellers back in 2004.  Might these performances have hit home more if the actors performed in AI-connected mo-cap with a Respeech to match the voice?  I'd at least find it intriguing. 

 

I think the possibility of AI's use goes up in correlation to how brief the appearance is.  Back a decade ago, Matt Smith had a whimsical moment as The Doctor when they placed his character into a clip of Laurel and Hardy from Way Out West.  It was a fun moment, but how much more impactful if for a dozen extra seconds he wasn't just in front of a green screen, but actually interacting with Laurel and Hardy?  Then the moment is gone and the Doctor moves on.   It moves from "That's cute" to "Oh, wow! Imagine the potential!"  The estates would have to sign off, be paid royalties, and personally, I'd want to see the performers credited if mo-cap was involved.  But handled respectfully, I don't see such moments as a problem. 

[ZacKing]

On 9/1/2024 at 4:40 PM, ThaOGDreamWeaver said:

TBH, I'm not sure studios should be using generative AI in that way at all. I'm sure the quality will improve exponentially over time with learning, but it's still much more expensive than actually hiring an actor. 

 

That depends on who the actor is.  Some of them make insane amounts of money -  see Tom Cruise, Adam Sandler, Dwayne Johnson etc.  You could make an entire movie with what some of these individuals get paid.  I read somewhere that Jennifer Connelly got something like $114K per minute for her role in Top Gun 2. 

 

On 9/1/2024 at 4:40 PM, ThaOGDreamWeaver said:

And if the original actor has passed away, giving somebody new a crack at it can add to that performance rather than trying to teach a machine how to be a bad copy.

 

This I completely agree with. 

[ThaOGDreamWeaver]

You can also recast key roles - and get different performances- if someone’s too expensive.
 

Or, for that matter, just too difficult to work with. (eg: the “no-a**hole” rule instituted by Feige…)

 

Quote

 

We have made the decision to not bring Ed Norton back to portray the title role of Bruce Banner in The Avengers.

Our decision is definitely not one based on monetary factors, but instead rooted in the need for an actor who embodies the creativity and collaborative spirit of our other talented cast members.

 

 

…ouch.

 

Adding an edit here for context - but spoilered for Alien:Romulus...

Spoiler

The movie brings back the late, great Sir Ian Holm. It's a much better job of work than Rogue One, not least because they borrowed a cast of Ian's head to scan off WETA, and he's voiced by fine Brit journeyman actor David Betts with a little revoxing help.

 

But it still begs the question. The role is Rook, another of the A120 series androids. It cost a lot to deepfake Ian, just so the audience gets a cue they don't really need, and gives away a plot point about A120s being nasty little b***ards loyal only to Weyland-Yutani...

 

...when you already have another actor playing a different series android in the movie. There are plenty of Highly Respected Brit Actors who can give you reasonably-priced gravitas and are prepared to play villain - why do you SPECIFICALLY need Ian?

 

Or even if you did - Rook is damaged in the film and gets plugged into a "friendly" android. Just the cast of Ian's head would have given you the clue, and then you face/heel the other actor. Or have Rook take control of the station and become an invisible enemy.

 

I'm with Sam Raimi on this one. Even if you've got the money and the tech, that doesn't mean you should spend it on something you can do practically. ("...where can I get an anti-grav disc?")

 

Edited September 3 by ThaOGDreamWeaver

[Techwright]

7 hours ago, ThaOGDreamWeaver said:

You can also recast key roles - and get different performances- if someone’s too expensive.
 

Or, for that matter, just too difficult to work with. (eg: the “no-a**hole” rule instituted by Feige…)

 

 

…ouch.

 

Adding an edit here for context - but spoilered for Alien:Romulus...

  Hide contents

The movie brings back the late, great Sir Ian Holm. It's a much better job of work than Rogue One, not least because they borrowed a cast of Ian's head to scan off WETA, and he's voiced by fine Brit journeyman actor David Betts with a little revoxing help.

 

But it still begs the question. The role is Rook, another of the A120 series androids. It cost a lot to deepfake Ian, just so the audience gets a cue they don't really need, and gives away a plot point about A120s being nasty little b***ards loyal only to Weyland-Yutani...

 

...when you already have another actor playing a different series android in the movie. There are plenty of Highly Respected Brit Actors who can give you reasonably-priced gravitas and are prepared to play villain - why do you SPECIFICALLY need Ian?

 

Or even if you did - Rook is damaged in the film and gets plugged into a "friendly" android. Just the cast of Ian's head would have given you the clue, and then you face/heel the other actor. Or have Rook take control of the station and become an invisible enemy.

 

I'm with Sam Raimi on this one. Even if you've got the money and the tech, that doesn't mean you should spend it on something you can do practically. ("...where can I get an anti-grav disc?")

 

There's a thought...pay an actor's estate to create great movies with his/her likeness and avoid his/her toxic meltdowns.  I look forward to the next Rex Harrison musical.  😁

 

Responding to what you've mentioned in the spoiler box:  Keep in mind, this is all in today's costs and standards.  Alien: Romulus is a 2024 film.  Rogue One is a 2016 film.  An 8 year difference. In CGI terms that's probably 2.5 generations, assuming something similar to Moore's Law.  Of course the 2024 movie CGI is going to look better.

 

Extending that thought, we should look beyond today.  CGI has greatly advanced in the last decade.  A.I. may speed this.  Just like computer technology, this increase is going to always cost more on the cutting edge, but less on the well-established end.  Sharknado 2030 is going to be a lot more realistic and scarier than the previous version, while still being budget friendly for SyFy. (I just made up an example.  No need to get worried.)   At some point, possibly sooner rather than later, we're going to cross that point where a CGI character is indistinguishable from the real thing, and then...we'll have Running Man. (j/k...I hope.)  ...we'll have a situation where "perfection", if I can call it that, is very costly, but in a few years, it becomes more affordable since CGI will find new things to push the horizon.  Eventually, it will be affordable to have "perfection".  That's the point everyone needs to look towards right now and come up with the best balance of rules, guidelines, permissions, and artistic freedom. 

 

 

Edited September 3 by Techwright

[ZacKing]

On 9/3/2024 at 12:48 PM, Techwright said:

There's a thought...pay an actor's estate to create great movies with his/her likeness and avoid his/her toxic meltdowns.

 

How about giving new people a shot at a role instead of reviving dead actors via CGI?  That's giving studios even more reason to not hire real people.

[battlewraith]

The studios see this as a way to maximize their profits and reducing labor costs (ie actors). The unnecessary appearance of notable actors in films right now via AI is a way of advancing the technology and normalizing it with audiences--as if there were some compelling artistic reason to use the likeness of an actor who is deceased. The endgame is probably not to continually cast notable actors of old in new films. It will be to create new digital "stars" who are based on samples of great actors that are remixed into another persona.

 

[PeregrineFalcon]

22 minutes ago, ZacKing said:

How about giving new people a shot at a role instead of reviving dead actors via CGI?  That's giving studios even more reason to not hire real people.

Agreed.

 

I like Arnold Schwarzenegger, but if you're going to make another Terminator movie then have a different model so it doesn't have to look like him.

 

However, if you're doing a prequel it's probably better to use CGI to make Grand Moff Tarkin look right than to just cast someone else in his role.

[Krimson]

Once voice AI technology like Whisper has been perfected, why would you need actors at all? 

[biostem]

I mean, using someone's likeness without them or their estate's permission definitely seems wrong to me.  That being said, I could even see that extending to only the person themselves, as you may get situations where the current owner/holder of said estate may hold positions ideologically opposed to the deceased person, and could use their likeness to have them endorse things they wouldn't while they were alive...

[Techwright]

4 hours ago, Krimson said:

Once voice AI technology like Whisper has been perfected, why would you need actors at all? 

 

This is the first I've heard of Whisper.  Is there a demostrated comparison with Respeecher?

 

As to why, I'd have to know if Whisper is pure AI or an AI to modulate an actor's voice to sound like another actor.  If pure AI, the challenge is to get the AI to know both subtlety and creativity.  The best voice actors have these skills, and I suspect no AI for a while will be completely able match or exceed both.  I could be wrong, though, I admit.

 

[Krimson]

16 hours ago, Techwright said:

 

This is the first I've heard of Whisper.  Is there a demostrated comparison with Respeecher?

 

As to why, I'd have to know if Whisper is pure AI or an AI to modulate an actor's voice to sound like another actor.  If pure AI, the challenge is to get the AI to know both subtlety and creativity.  The best voice actors have these skills, and I suspect no AI for a while will be completely able match or exceed both.  I could be wrong, though, I admit.

 

Whisper

[ThaOGDreamWeaver]

I'm not sure AI will ever replace acting. But it might eventually replace language dubbing and ADR.

 

In my line of work, I'm starting to regularly get punted a number of re-voxing technologies. 

https://www.rask.ai/ is an auto-translator that can voiceclone you and then use your voice phonemes to speak other languages.

 

I think it would be fine for short clips at the moment, like news or viral clips.

However, one of the creative folks we work with laid into me for even taking the meeting, and then helped me dig why, by running a show through it into French.

 

Aside from being oddly choppy, there's enough misuse of accents, phonemes and pacing to give it away as AI. That can be learned.

What's trickier is that it's doing a very literal translation of each sentence, in a way that doesn't necessarily fit with the destination language, or the action it's covering. 

(My favourite example of this is that there aren't many French swearwords you can draw out, so a specific scene in Pulp Fiction uses three expletives to cover one very long one.)

 

https://www.respeecher.com/, which m'learned coll' mentions above, works differently. 

Let's say you were doing, I don't know, the French localisation for Star Wars: Outlaws, and you're doing Vader's voice.

(Which oddly enough, they didn't use JEJ's actual voice for - he's permanently licensed it to Lucasfilm. NO-ONE ELSE WILL EVER BE VADER ON SCREEN.)

What you do is you record a performance first, putting as much suitable menace into the pacing as you can muster.

Then some clever Ukrainian AI types take your phonemes apart, replace it with the equivalent ones from the destination actor, and give it a manual polish.

Presto: your words are being spoken by the Dark Lord Of The Sith himself.

 

You can also try out some of this stuff for free on https://elevenlabs.io/ and others.

 

All of which gives me a headache... now audio nerds like me are being asked to find ways of spotting faked audio, both to protect the original authors and prevent ad fraud.

Fun.

[Krimson]

21 minutes ago, ThaOGDreamWeaver said:

I'm not sure AI will ever replace acting. But it might eventually replace language dubbing and ADR.

 

In my line of work, I'm starting to regularly get punted a number of re-voxing technologies. 

https://www.rask.ai/ is an auto-translator that can voiceclone you and then use your voice phonemes to speak other languages.

 

I think it would be fine for short clips at the moment, like news or viral clips.

However, one of the creative folks we work with laid into me for even taking the meeting, and then helped me dig why, by running a show through it into French.

 

Aside from being oddly choppy, there's enough misuse of accents, phonemes and pacing to give it away as AI. That can be learned.

What's trickier is that it's doing a very literal translation of each sentence, in a way that doesn't necessarily fit with the destination language, or the action it's covering. 

(My favourite example of this is that there aren't many French swearwords you can draw out, so a specific scene in Pulp Fiction uses three expletives to cover one very long one.)

 

https://www.respeecher.com/, which m'learned coll' mentions above, works differently. 

Let's say you were doing, I don't know, the French localisation for Star Wars: Outlaws, and you're doing Vader's voice.

(Which oddly enough, they didn't use JEJ's actual voice for - he's permanently licensed it to Lucasfilm. NO-ONE ELSE WILL EVER BE VADER ON SCREEN.)

What you do is you record a performance first, putting as much suitable menace into the pacing as you can muster.

Then some clever Ukrainian AI types take your phonemes apart, replace it with the equivalent ones from the destination actor, and give it a manual polish.

Presto: your words are being spoken by the Dark Lord Of The Sith himself.

 

You can also try out some of this stuff for free on https://elevenlabs.io/ and others.

 

All of which gives me a headache... now audio nerds like me are being asked to find ways of spotting faked audio, both to protect the original authors and prevent ad fraud.

Fun.

Something of interest

[Techwright]

On 9/4/2024 at 6:23 PM, biostem said:

That being said, I could even see that extending to only the person themselves, as you may get situations where the current owner/holder of said estate may hold positions ideologically opposed to the deceased person, and could use their likeness to have them endorse things they wouldn't while they were alive...

While I see your point, that kind of falls back on the late person themselves when setting up their estate.  While admittedly someone who died before the advent of CGI and AI would have no idea how technology might bring them back so vividly,  they likely still would know/understand that their photo likeness and possibly voice recordings could be used for marketing purposes long after they're gone.  It would make sense to pick executors of the estate who will hold to that person's vision regardless of personal feelings.   The trickier part is to prepare the estate to carry on for multiple generations of executors making sure that none act in their own interests, but in the interests of the deceased. 

 

JRR Tolkien is such an example, I think, though not as an actor.  His son, Christopher, tightly guarded his father's estate, and was angered when New Line Cinema took the films in a path he felt did not adhere to his father's wishes.  He even stated that it would not happen again.  However, with Christopher's death, the new generation clearly has wealth on their minds and has opened his iconic writings up to all those wishing to slap an official "Middle Earth" label on anything, whether it measures up or not.  I've not read JRR's will and testament, but I suspect it didn't tightly regulate his intentions.

 

While it could be reasonably said that someone past could not have planned their estate for CGI and AI, they would have had the chance for stating how their image and voice were to be used. After all, those sorts of things were in use for printed and radio advertisements back to the 1920s.

 

 

Edited September 10 by Techwright
A minor grammatical adjustment

[biostem]

5 minutes ago, Techwright said:

the new generation clearly has wealth on their minds

Agreed - I'm not as concerned about the immediate inheritors of said estate, but rather 1 or 2 generations after, where those people may not have ever personally known the deceased and/or don't care to preserve/respect their intentions when it comes to how to make money off the estate or any IPs it possesses...