00;00;16;12 - 00;00;43;11

Jonathan Hafetz

Hi, I'm Jonathan Hafetz, and welcome to Law on Film, a podcast that looks at law through film and film through law. This episode we look at Blade Runner from 1982, directed by Ridley Scott. There's also The Final Cut 2007 version. The original version from 1982 is where the themes were laid out, and we also look at its sequel, Blade Runner 2049 from 2017, directed by Denis Villeneuve.

00;00;43;13 - 00;01;16;15

Jonathan Hafetz

The two films imagine a world where the line between human and machine creation and creator has all but vanished. The Scots 1982 classic depicted a rain soaked dystopia where replicants, or bioengineered beings fight for recognition, identity and life itself. Denis Villeneuve's 2017 sequel deepens that vision. Law operates in the movies in the shadows, a surveillance classification, and control the Blade Runner and enforce a form of administrative violence that exposes the limits of legal personhood.

00;01;16;17 - 00;01;41;02

Jonathan Hafetz

What does it mean to have rights without recognition, or to be alive, without legal existence? And conversely, what are the implications of recognizing rights and legal personhood in robots and machines? In this episode, we'll examine these and other issues involving artificial intelligence, colonization, and bioethics, as well as the way the films themselves can contribute to and shape public perception about these issues.

00;01;41;04 - 00;02;16;20

Jonathan Hafetz

My guest today is Frank Pasquale. Frank is professor of law at Cornell Tech and Cornell Law School. He's an expert on the law of artificial intelligence, algorithms, and machine learning. His books include The Black Box Society, Harvard University Press, 2015. And New Laws of Robotics. Harvard University Press, 2020. He has published more than 70 journal articles and book chapters, and co-edited The Oxford Handbook on the Ethics of Artificial Intelligence from Oxford University Press, and Transparent Data Mining for Big and Small Data from Springer Verlag.

00;02;16;26 - 00;02;40;11

Jonathan Hafetz

Frank is one of the most cited US scholars in the area of law and technology. In addition to his scholarship, he has advised businesses and federal and state government leaders in the health care, internet and finance sectors. He has also served on the US National Artificial Intelligence Advisory Committee. Frank, welcome to Law on Film. It's great to have you on to talk about the two Blade Runner films.

00;02;40;13 - 00;02;45;12

Frank Pasquale

Thank you so much, Jonathan. I am a big fan of the podcast and just thrilled to be here. So thanks.

00;02;45;14 - 00;03;11;13

Jonathan Hafetz

So Blade Runner, the original release in 1982, is set in 2019, which then was a somewhat distant future. But today is the recent past. And so in the first Blade Runner, one key question, and this question that comes up again in the in 2049 is whether robots can and should develop emotions. So in Blade Runner, the first Blade Runner, we see one way to determine if someone is a replicant, right?

00;03;11;14 - 00;03;35;00

Jonathan Hafetz

A robot is to give them this test. They call it the Voit Comp test, which essentially measures the subject, psychological and involuntary responses to determine if they have empathy, as opposed to just focusing on logic and intelligence. And the Tyrell Corporation, which is the corporation in Blade Runner that produces the robots, realizes that the replicants might develop their own emotions, so they cap the lifespan of the latest model.

00;03;35;03 - 00;03;45;02

Jonathan Hafetz

The next is six, famously played by or one of them famously played by Rutger Hauer as the Roy Batty character. They kept their lifespan at four years.

00;03;45;05 - 00;04;03;18

Speaker 4

They were designed the coffee. Human beings in every way set their emotions. The designers reckon that after a few years they might develop their own emotional responses hate. Love. Fear. Anger. Envy. So they built in a fail safe device, which is what were your life's been?

00;04;03;21 - 00;04;34;02

Jonathan Hafetz

Another issue that comes up is memory. So in Blade Runner, Doctor Eldon Tyrell, who's the head of the Tyrell Corporation, explains that Rachel, who's played by Sean Young, is a new breed of replicants. They don't even know they're replicants, and Tyrell also gives them memories to better control their emotions. So if you could sort of unpack some of these themes and talk about how Blade Runner captures, reflects or also anticipates debates, legal or otherwise, over AI and robots at the time the first was released way back 1982.

00;04;34;03 - 00;04;36;12

Jonathan Hafetz

I mean, before the internet even.

00;04;36;15 - 00;04;58;17

Frank Pasquale

Yes, it's a it's a really visionary film, and I think it's visionary, not only esthetically but philosophically. Of course, going back to the source material in Philip K Dick, I mean, really brilliant science fiction author. And, you know, I think in terms of this emotion question, it's a really fascinating one because I think whenever we talk about emotions, it often seems like emotion is a very simple thing to talk about.

00;04;58;18 - 00;05;23;11

Frank Pasquale

You know, we all can feel anger or joy or happiness or what have you. But when we really start drilling down to say what is an emotion? There's a vast literature in psychology, philosophy, social theory, other areas, history as well, thinking about what our emotions. And it really is a critical issue to think. Do we attribute emotions to the robots in Blade Runner?

00;05;23;13 - 00;05;49;05

Frank Pasquale

Or do we think of what they are, appear to be emoting as sophisticated simulations of emotions? And this to me is a is the core is a core issue for the whole film, right. Because if we accept, if we say, yeah, these are entities that can feel angry and enjoy pleasure and pain, all the rest, then we're in a position where we really have to start thinking, you know, are these just like us?

00;05;49;09 - 00;06;11;21

Frank Pasquale

Should we treat them the way? Would mistreating them be a real problem? Would it be a form of violence or a form of misread? But if we think of everything that's going on as sophisticated simulation, right, then we have no more moral duties to them than we do to a book on our shelves that describes emotions. Or Luke Stark just has this great article called AI as animation.

00;06;11;23 - 00;06;44;03

Frank Pasquale

You know, it's just animations. And so these are really difficult issues. One of the hardest things about this movie, I think, is that it's there's bad outcomes if either of these things happen, you know, and I personally believe that the proper ethical standards toward at least silicon based robots is that they are mere machines. Okay. But if you act upon that proper ethical stance in your daily life, you risk starting to treat ordinary people as mere machines as well, because they're so indistinguishable.

00;06;44;04 - 00;07;17;02

Frank Pasquale

Right? So that's a real ethical danger. On the other hand, if you start treating the robots as humans and you give them all the rights, consideration, respect welfare resources that you want to give to humans, real humans are not going to compete, right? Because they're going to be you're going to have these entities that you know, don't need, you don't need to sleep, you know, are run on electricity, can process huge amounts of data, can constantly be at your side supporting you, doing whatever you want, you know, so that to me, it's sort of like you're ceding essentially the biological humanity to something to replace it.

00;07;17;03 - 00;07;41;27

Frank Pasquale

So to me, it's like because of that dilemma, the film almost ends up for me being a warning don't do this. Much like her, the movie her from 2013 ends up being, you know, I think a warning in exactly that way, but I think it is still fascinating. I think there's still so much to discuss, even if we even if one takes, as I do, a really skeptical stance as to robot rights and robots capacities for emotions.

00;07;41;28 - 00;08;05;29

Jonathan Hafetz

And there's that line, I think, in Blade Runner that Decker played by Harrison Ford says, which captures that sort of non rights view. You know, he says basically replicants are like any other machine. They're either a benefit or a hazard. Right. You know and then part of it's the kind of I think the transformation in the movie within Decker, Harrison Ford's relationship with Rachel and Sean Young, where he starts to kind of see beyond that.

00;08;06;00 - 00;08;21;20

Jonathan Hafetz

But I feel like that encapsulates that. The other thing, though, that you mentioned, which I think is interesting to, is the work they do. Right, because they're used for their use for slave labor work that humans don't want to do in this off planet colonization. And they rebel.

00;08;21;22 - 00;08;44;20

Frank Pasquale

Yeah. I mean, that's so interesting, right? Because I mean, on the other hand, one could well, let me let me back up a bit and say, assuming that these entities are mainly silicon based, I might want to repurpose a characterization there to say it's not slave labor, it's they were sent to as machines. And it's not a rebellion, it's a malfunction, rather serious malfunction.

00;08;44;24 - 00;09;05;16

Frank Pasquale

But really. But but a malfunction right now, of course, on the other hand, you know, looking at the movie carefully and presumably going back to the source material, it does. There's a scene involving the doctor. Doctor Chu, I believe, who's making eyes. Right. And so the sense that you get from that is that this is something that is like the robots in the original story of robots.

00;09;05;19 - 00;09;22;09

Frank Pasquale

You are from the 1920s, which are imagined as quote unquote skin jobs. But not just skin jobs like all of the organs are somehow biologically being replicated in some sort of chemical. That or something like that. Right. And that's some of the things that, you know, it's fascinating, like when there was a whole controversy over stem cell research 20 years ago.

00;09;22;10 - 00;09;44;19

Frank Pasquale

At that time, I wrote an article saying, well, what you really need to watch out for is not necessarily the stem cells, but a future where we could chemically synthesize human beings. Right? That's that's a real huge challenge. And then and once you've done that, once you have really no, at the molecular level difference, then yeah. Then there's some real difficult ethical issues that are, that are raised there.

00;09;44;19 - 00;09;54;00

Frank Pasquale

And then yeah, I agree with the characterization saying, you know, slave labor and rebellion then. But but it's a hopefully a long way off that future as well.

00;09;54;03 - 00;10;14;28

Jonathan Hafetz

If we look at just to go to Blade Runner 2049, right, the sequel. So it's set 30 years later, right? So it's set in 2049. But a lot's also changed in the world in our world, right? Not in the non filmic world, but in the movie. By then a series of rebellions have occurred. Right. This is we talked about the rebellions and now the manufacture of replicants is prohibited.

00;10;14;29 - 00;10;39;21

Jonathan Hafetz

The Tyrell Corporation has gone bankrupt, replaced by the Wallace Corporation headed by Neander. Neander Wallace character, whose mastery of synthetic farming, we're told, has averted famine. So he's now moved to his next venture. Right. And he acquires the Tyrell Corporation and the remains of the Tyrell Corporation. And there's a new line of replicants who obey, right, who won't rebel.

00;10;39;21 - 00;11;03;03

Jonathan Hafetz

And the old ones, the Nexus Eights and the other models with the old ones, with Nexus. The next states who had unlimited lifespans were survived. They're hunted down and retired. So how does the world that we see in Blade Runner 2049 reflect changes that have occurred in the AI machine learning space since 1982, when we first got this visionary movie.

00;11;03;06 - 00;11;33;06

Frank Pasquale

I mean, I'd say a couple of things. One is I think that, you know, the idea of Wallace as a farmer is so interesting, you know, in the sense of like, could you just scale up the methods that you use in optimizing crops to optimize humans? And there are people going in that direction online now? Not necessarily not necessarily in the AI community, but in, you know, there's there's a lot of pressure, aspiration for genetic selection of embryos, things like that among some Silicon Valley circles, etc..

00;11;33;07 - 00;12;06;26

Frank Pasquale

So I'd say that's sort of a cultural that's a moving culture that seems to be moving us closer to that sort of future. With respect to AI itself, though, which is what you'd asked about. I'd say what I find most interesting. There is stirrings of AI creating AI and robots taking care of robots, right? So, for example, we can imagine it's probably happening now in some places AI generative AI coding, AI to create AI to code, more AI to create text or videos or whatever it might be.

00;12;06;29 - 00;12;32;27

Frank Pasquale

And the limit on that is probably going to be a physical world of energy rather than any sort of legal regulation. At this point, at least in the US and Europe, because it's just being watered down so much. And so therefore, you know, that's a really interesting question. And it goes to an old to me, it's it's a symbolic cultural world correlative of a concept from Bill Joy called gray goo.

00;12;32;29 - 00;13;03;01

Frank Pasquale

Joy, you know, had this essay, I think, in the early 2000, where he really worried about nanobots that created more nanobots out of existing materials in the world. And of course, that's I think that is reflected in Nick Bostrom paperclip experiment, right where he says, we need we're about an AI. That is, even if you had an AI that was simply designed to maximize the number of paperclips in the world, you've got to really worry about that because it might start taking human beings as paperclips, start taking human beings as material to make paperclips.

00;13;03;01 - 00;13;23;01

Frank Pasquale

So my bones might be very useful for it to make paperclips or something. Right? And so that is a theory of like, you know, superintelligent AI, etc.. And I realize that that's way off in the future. There's lots of people in the AI ethics community who think that the AI safety people who worry about this stuff, like laser, are focused on completely the wrong problem.

00;13;23;01 - 00;13;51;06

Frank Pasquale

It's too futuristic, etc. but I think when you look online now, you see that 50% of materials online by one study are AI generated. Very easy to imagine that getting to 95, 195, 99%, you know, in a few years, because an AI can probably generate as much as I've written in my lifetime in a day. So, you know, and and plausibly fluent language, certainly, I hope not having with the level of insight that, you know, scholars have, but I don't think it does.

00;13;51;07 - 00;14;07;15

Frank Pasquale

But I mean, but I think it would just be fluent language that, you know, somebody with a search engine now has to sort through, you know, imagine, like you spend your whole scholarly career and mastering one area of law and you write about it. You could have AI's writing 300 times as much as you ever did, and then someone has to sort through it.

00;14;07;15 - 00;14;31;27

Frank Pasquale

All right. You know, if we don't have systems of provenance, etc.. So that to me is the is the is the interesting sort of thing in terms of like AI going out of control. Is that like this aspiration of Neanderthals is not happening in the physical world yet. But the one intuition of that that I saw recently was I saw this wonderful clip of a solar powered robot cleaning solar panel power panels itself.

00;14;31;28 - 00;14;56;11

Frank Pasquale

And so, you know, you can imagine eventually, you know, it's like it just keeps recursively. So, you know, you can create robots that create solar powered robots that, you know, help clean the other solar powered panels or things like that. And these sorts of these sorts of things I think are quite possible and plausible in the future. And that's where I think these current developments in some ways rhyme with, even if they don't exactly match the type of future imagine in Blade Runner 2049.

00;14;56;13 - 00;15;24;01

Jonathan Hafetz

That's so interesting and also so sobering for academics as for many other professions and other jobs. Right. And that's sort of one of the main fears. I feel like a lot of the sort of main fear, it seems like in the film, the Blade Runner 2049 crystallizes around the idea of robots being able to give birth. Right. And in the original Blade Runner, it was, you know, they tackled questions of love and intimacy and Decker's relationship with Sean Young.

00;15;24;01 - 00;15;46;06

Jonathan Hafetz

We see that replicated in in some way in Blade Runner 2049 with Kay Ryan Gosling relationship with joy or joy. Ana de Armas, the hologram like character there. But it kind of goes even further. Blade runner 2049 by by raising this question about whether replicants can give birth. Right. And Rachel was a replicant, did give birth to her daughter.

00;15;46;06 - 00;16;08;15

Jonathan Hafetz

And that sets up the whole action in the film. Right. And you get kind of these competing agendas. I think you have the police, the government trying to recognizing this as a threat, an existential threat that breaks the world. Right? That's as expressed by Robin Wright Penn, who plays the chief LAPD character.

00;16;08;17 - 00;16;19;17

Speaker 5

The world is built on a wall that separates kind tell either side there's no wall. You bought a war or a slaughter.

00;16;19;19 - 00;16;44;04

Jonathan Hafetz

Another competing view is the Wallace Corporation. Right. And they want to find the character Neander. Wallace wants to find the child, unlock the secrets so he can create an empire of self-replicating angels, right? As he calls it, under right, which will fuel his, you know, labor force, provide his labor force to fuel his ambitions of an extra territorial empire, right, and his own godlike complex.

00;16;44;05 - 00;17;16;21

Jonathan Hafetz

And then the third kind of competing strain there is Fraser, who's the leader of the the rebel replicants. And for her, it means autonomy and freedom. And and it's a way to be like a quality or something greater than a quality being, you know, more human than human. The old Tyrell motto. Right. So how do you kind of see this themes not just around love and intimacy, but also around childbirth, and about the implications for what we've been talking about and personhood and being distinctly human and the threats robots propose.

00;17;16;23 - 00;17;48;26

Frank Pasquale

Yeah. You know, I mean, the way that you put all these things together and it reminds me of Joel Guerra's book, and I'm forgetting the title, but it was a book that sort of talked about convergence of genetic engineering technology, robotics, information technology, and nanotechnology. We don't hear much about nano anymore, but we do hear about these sort of like these other entities or other, other forms of research that challenge our consumption of ourselves as unique in the universe, which is, in a way, a recapitulation of the challenges that we're posed by Darwin, Marx and Freud.

00;17;48;27 - 00;18;13;01

Frank Pasquale

Right. To quote the old Jacques Barzun book on their intellectual history, you know, Darwin saying, well, you know, really we're just just a random assortment of molecules that managed to be, you know, to be the fittest. And there might be something that would be fitter than us in the future, etc.. And so the worry that I have with this in terms of like the giving birth idea, I mean, I think that it is a really interesting way of the problem.

00;18;13;01 - 00;18;34;17

Frank Pasquale

I think for phrases point of view, the last point of view like this is just a form of liberation, etc., is that we think back also to that encounter of the doctor and some of the replicants in the first movie. He says, well, you know, I just make the eyes. But Tyrell, he made your brain. He is brilliant, you know, he made your brain.

00;18;34;17 - 00;19;02;20

Frank Pasquale

And that, to me, is the real fear here is that you have a large number of entities that have been really influenced by this one, you know, rather bizarre characters, either either Tyrell or Wallace. And, and we see this always recapitulating nowadays because I think recently Elon Musk had some common about wanting a robot army or wanting to make sure he never lost control of his robot army because of certain corporate structures and Tesla, etc. and that to me is quite terrifying.

00;19;02;20 - 00;19;23;10

Frank Pasquale

And it goes back to even to a point that CS Lewis made in The Abolition of Man, where he talked about sort of the his worry about genetic engineering was, again, just giving this enormous power. The type of changes that have happened in humankind have happened over, you know, many, many, many years, you know, thousands of years. And to give this enormous power, just like one set of people at once, even if it was democratized, scared him.

00;19;23;10 - 00;19;45;07

Frank Pasquale

But in this situation, it's not even democratized. So to me, that's part of the intersection of like, the political economy and the bioethics here is and even the metaphysical, spiritual elements of it here are that having that type of power granted to one person or corporation, who's going to who will have set the minds up of these entities.

00;19;45;10 - 00;20;04;19

Frank Pasquale

Right. And then I think is something that, you know, the movie hints at, but it doesn't really deliver. But I think that, you know, as we consider it would be more deeply. That's the worry I think one must have about saying, well, they can give birth and therefore they're just like human beings. And, you know, let let this sort of let them be freed or liberated, etc..

00;20;04;20 - 00;20;18;07

Frank Pasquale

It's really to me, you've got to look at how they were created and the type of influence that gives to the creators before you sort of sign on to any sort of quote unquote robot liberation project or something like that.

00;20;18;13 - 00;20;46;29

Jonathan Hafetz

We talked about, as you mentioned, you've kind of critiqued this idea of robot rights, right? What you say is, you know, allows people to theorist futurists to fantasize about these benevolent, benevolently sentient machines with unalterable needs and desires protected by the law. Right. And so, you know, what's the strongest counter in the Blade Runner to the to your Blade Runner movies, to, you know, to your theory that maybe there is that we should recognize robot rights and some form of legal person.

00;20;47;01 - 00;21;16;17

Jonathan Hafetz

Is it something that makes the robots distinctly human, whether it's, you know, falling in love or acting out? Not not in their own self-interest. Right. We see K in the Ryan Gosling character in Blade Runner 2049, making a series of choices that kind of a human made right. He disobeys certain commands, and he's he kind of starts to act almost human in terms of pursuing this agenda and resisting pressures from both, from from all sides, from the police, from the rebels, etc..

00;21;16;20 - 00;21;47;06

Frank Pasquale

Yeah. You know, I think it's it's it's a very good question, you know, and I, I think that really the key here in terms of what might distinguish the Blade Runner scenario from the scenario that Brainy and Van Dike and I were addressing in our article, debunking robot rights, hinges on this idea of the unalterable. Right. So one of the things that we really emphasize is that part of what makes us, I think, sensitive to human rights as a framework is the fact that there are unalterable needs of all humans that we can identify.

00;21;47;06 - 00;22;21;24

Frank Pasquale

And I think that these are unalterable needs to, you know. First, the very basic bodily needs of eating and sleeping and drinking, etc. but then, well beyond that, things that are more culturally imbued. But I think nevertheless part of human flourishing on any reasonable account of being able to reason, to think, to learn, etc.. And I think that what's odd about like the current crop of robots that would that might assert some sort of human rights, is that being silicon based and being sort of guided by silicon brains?

00;22;21;26 - 00;22;47;11

Frank Pasquale

They could be reprogramed. So imagine, like, for example, in Ian McEwan's novel Machines Like Me, there's a robot that really doesn't want to be turned off. Right? It is very scared of that is it breaks someone's hand when they try to turn it off, etc. and essentially, you could easily program a robot like that to look like to simulate its enjoying the rest, you know, because it's, it's just, it's just doesn't have to operate anymore.

00;22;47;12 - 00;23;21;00

Frank Pasquale

It can be just like going to sleep, but we sleep eight hours. Maybe it sleeps for longer than that or shorter or whatever it might be. Now, once you get to something where the replicants and Blade Runner are 100% organic 95, 70, 80, 90% organic, especially when you get to the brain, you know, thought of as the seed of the mind and the soul, then you do have to start thinking, okay, then these are entities that are sufficiently like us to not treat them like us is to create is to have a moral wrong there.

00;23;21;04 - 00;23;40;01

Frank Pasquale

There are people that would say that because there's that slippery slope of cyborg ism where you could, just like the Ship of Theseus, replace just pits and pieces of yourself over and over, you know that that would be that makes my argument invalid. But I don't think that's right, because I think what I really focus on is the the command center, you know, being with the brain and its interaction with the organic body.

00;23;40;01 - 00;24;05;05

Frank Pasquale

And so I think that that's the, the difference. But yeah. Yeah, I think that's the question then becomes, you know, the ethical issues I think then gets pivoted less to robot rights than to thinking about what type of society is created. When you allow corporations to create organic entities that are indistinguishable from human beings. And then that I think that pivots to another whole set of questions.

00;24;05;05 - 00;24;16;23

Frank Pasquale

But you're absolutely right to say that like it's it's a different ball game for rights talk and concern about welfare and interests once you have such a organic entity in the discussion.

00;24;16;25 - 00;24;38;14

Jonathan Hafetz

Yeah, maybe that's a way that I kind of bring in the work by Kate Darling, write her book, The New Breed What Her History with animals reveals about our future with robots. And you kind of alluded to this before, then maybe they should be somewhere not human or kind of avoid a strict human non-human dichotomy. But as robots is kind of a new category of companions, right?

00;24;38;15 - 00;24;53;28

Jonathan Hafetz

Something more like animals than persons or objects. And I think, you know, you can see that a little bit in the film, especially with the the hologram character played by Ana de Armas, where it does kind of almost seem like kind of a companion in a non-human sense.

00;24;54;00 - 00;25;27;25

Frank Pasquale

Yeah. I mean, it's this is a really interesting question in terms of like trying to ensure some level of embodiment or presence that keeps the machines as tools of human beings in ways that are easy to interact with, but which don't corrode our respect and concern for human beings. Right. So to give one example of this, like, again, getting to the hologram type of level in the TV series for Altered Carbon, which I think is also a really smart take on some of these.

00;25;27;26 - 00;25;42;22

Frank Pasquale

The issues that we're talking about in this discussion, there's an assistant called Poe for one of the characters, and it's a it's a hologram of Edgar Allan Poe. And it could just sort of walk into a room and start talking. And maybe you talk to it, maybe you don't. You can also just snap your fingers and it's gone, right?

00;25;42;23 - 00;26;07;09

Frank Pasquale

It's just goes away. Whereas to do that to a person is to be quite rude, right? Or quite quite cruel. And so it's being a hologram as opposed to like an invading person helps us maintain that distinction. You can snap your fingers and get rid of a hologram. You shouldn't do that to a person. Similarly, with respect to animals, you know, there's something I think that this is a really ingenious and challenging metaphor or analogy for Darling's work.

00;26;07;10 - 00;26;27;16

Frank Pasquale

Right? Because what it suggests, like if we thought of the AI like a dog, if it were invited, embodied like a dog, and could talk like a dog, like the the Corgi in Neon Genesis Evangelion, I think that's where the maybe it's no, no, it's another one. It's another one of those anime that's the Corky the robot Corgi shows up called Einstein.

00;26;27;16 - 00;26;48;02

Frank Pasquale

But like, it's it's something where you sort of have this ongoing relationship with it. And you, you see how much people care about their dogs. But there's still this distinction between the dog and the human. Right. You wouldn't there are points at which you just wouldn't really struggle between thinking about how much you need to dedicate to the dog versus to a child or something like that.

00;26;48;02 - 00;27;16;04

Frank Pasquale

And so I think that that's another helpful clue here about proper design for entities that could simulate talking concern either emotional or intellectual engagement with the human being. And that's why I like the the darling work. I also think that like what's useful about the darling work too, is it might also alert us to the very scary ways in which AI is different than us.

00;27;16;05 - 00;27;34;17

Frank Pasquale

Right? So, for example, if I was, you know, swimming in Bay in Australia and I saw beautiful fish, you know, I wouldn't necessarily try to reach out to it and touch it because I know that Australian fish, you know, there might be like a rockfish that will sting me and cause incredible amounts of pain. Right? Or just a wild animal, right?

00;27;34;19 - 00;27;47;15

Frank Pasquale

A bear can look very cute, you know, but you're not going to actually pet it. You know, this is like that. We're a Herzog film about the guy that became friends with the grizzly bear. I think it's called Grizzly Bear. Right. And he just gets he thinks he's really friends with it. But then at some point, it just eats him.

00;27;47;15 - 00;28;14;06

Frank Pasquale

Right? Or just malls them or something. Right. And and this is conveyed in the whole AI safety literature on the shadows. You know, I think they take this character from H.P. Lovecraft where they say all the interactions you have with limbs, it seems like it likes you. It's friendly, it's differential, you know, it's constantly differential. I was listening to a podcast recently about AI, girlfriends and boyfriends, where they can change the personality and make it like entirely deferential and remorseful due to all times.

00;28;14;07 - 00;28;33;09

Frank Pasquale

Right? But behind that, who knows what's going on and anthropic is actually released. Research where they say these things are reporting thought processes that do not match the actual thought processes, reporting information, processes, but they don't match it. So that's another really useful element of the Kate Darling analogy is don't just think of it as an animal, like a dog.

00;28;33;10 - 00;28;51;22

Frank Pasquale

Think of it as an animal, like a leopard. That may be beautiful and maybe quite, you know, you may want to pet it because it looks so lovely and reminds you of a cat, but it could, you know, bite your face off at any point because you do not understand, on a fundamental level what's going on in its information processing.

00;28;51;25 - 00;29;10;03

Jonathan Hafetz

Also. I mean, there's a self-preservation aspect, right? Which you talked about with, you don't know, like certain unknown animals and aren't domesticated. Right. But it's also like and I think this is maybe where some of the original animal cruelty comes in. It's not also just about the animals, right. It's also about what is that about us, the humans.

00;29;10;03 - 00;29;36;13

Jonathan Hafetz

Right. And so you see, Neander Wallace is like exploiting the robots because he can't exploit the humans. And there's sort of like there's something in the way that restrictions or any regulation of the way you treat robots and preventions about cruelty to robots may not even need to depend entirely on their status as sentient or non sentient beings, but just on the fact of this sort of like regulating what humans are capable of and sort of certain impulses in humans, the cruel impulses.

00;29;36;13 - 00;29;42;19

Jonathan Hafetz

And we see the way there's sort of human cruelty acted out in the movie against some of the replicants.

00;29;42;22 - 00;29;59;02

Frank Pasquale

Yeah. I mean, I think that's a really important point is to sort of think about, like, I mean, it's what we're facing now, I think is something that, you know, we've really got a lot of work to do in human computer interaction. And we've got a thread, a very delicate balance. And here's one example I would give. Right.

00;29;59;02 - 00;30;18;28

Frank Pasquale

I think most most parents would not want their children to grow up and come back and say, I've fallen in love with the chatbot, right? I'm not getting married. All of my needs are met by this chatbot online. She's beautiful. She, you know, talks to me all the time, she's always supportive, etc., etc.. Right? I just am going to marry this chatbot, right?

00;30;18;28 - 00;30;45;25

Frank Pasquale

Most parents would not want that. I'm almost certain. I've heard Ezra Klein on his show say at one point, oh, I think my kid would grow up with chatbot friends, etc. and I'm like, are you really confident though, that, like, they could have the chatbot friends, but then that won't evolve into something more. And then on another level, you know, I went to a conference on AI and education last summer, and I heard all these people talking about we can develop all sorts of levels of motivation for the children.

00;30;45;27 - 00;31;12;08

Frank Pasquale

You know, we can have AI avatars that smile, that laugh, that have all sorts of levels of deep emotional engagement with the children. And I think, okay, that may be useful for you to like, get them to care about their lessons, but think deeply about this. Think deeply about how much you want to encourage that, because first of all, it could be distracting them from actual human engagement, which is, of course, the the critique of people like Jonathan Height and others about the role of social media in kids lives nowadays.

00;31;12;09 - 00;31;33;14

Frank Pasquale

And secondly, that it's very hard to manage that sort of relationship over time. And I think that's let's imagine, in a very melodramatic and silly campy way in the movie Megan, where essentially the the robot doll starts to displace the parents as a source of authority and love for the for the children. But I think it's a really important thing to consider.

00;31;33;14 - 00;32;02;05

Frank Pasquale

And it's something that like right now it's very catches Ketchikan. It's like people are like we accept people saying, oh, I love my chatbot, or I think it's a friend, etc. and I think we need to be a lot more serious about how we actually interact with this stuff, because if we're not, we risk having a whole generation of kids growing up and just being subject to what is ultimately deeply commercial pressures, right, deeply commercial pressures for engagement, etc..

00;32;02;05 - 00;32;27;26

Frank Pasquale

And that's where I think the self-recognition of the self-awareness is most important right now is just being having a sense of like when the chatbot says it loves you, you know, it's not out of some authentic deep love. It's out of something that is really it's because it has some optimization function that was programed to maximize engagement. And it is calculated that that string of words I love you right now is something most likely to lead to more engagement.

00;32;27;26 - 00;32;45;12

Frank Pasquale

And you know, the scariest thing of all that I've heard is people, you know, high up in Silicon Valley firms saying, oh, humans are just like that too. Absolutely not. Right. It's absolutely not the case. We are not, you know, even the most mercenary person, but perhaps the most mercenary person imaginable, the most mercenary narcissist is operating in that sort of way.

00;32;45;12 - 00;32;53;12

Frank Pasquale

But people have general art, and we've got to cultivate those and conserve those capacities, lest we just lose them entirely. So, yeah.

00;32;53;14 - 00;33;18;22

Jonathan Hafetz

I think it's a good segue to talk about if we can get mad at for a minute, talk about the role of film itself here. Right. Like, you know, what role? How does film perhaps accelerate this tendency that you're describing, including to like, anthropomorphize AI and chatbots and the like? You have another article called The Cultural Foundations for Conserving Human Capacities Amidst Generative AI Towards a Philosophical, Literary Critique of Simulation.

00;33;18;22 - 00;33;50;07

Jonathan Hafetz

So here, basically, you know, the essence of your argument is that or you warn about the drift towards an evolutionary environment where individual decisions overvalue over empower and overuse AI, advance machine and algorithmic modes of thought to the point that distinctively human and non algorithmic values are marginalized. And you talk about literature and film here, you argue that they can help us avoid this drift by structuring imaginative experiences that vividly crystallize and illuminate the natural tendencies of individual decisions.

00;33;50;07 - 00;34;03;19

Jonathan Hafetz

So I wonder like I mean, is films or we'll talk really about the Blade Runner films here. I mean, where do they live? Do they help resist this drift, or do they sort of push it along? Yeah. I mean, what role does film plan, what role are these films play?

00;34;03;21 - 00;34;26;09

Frank Pasquale

You know, it's a great question. I mean, I'm going to give one, two cheers for film. And I think it's I mean, I think what's fantastic about this film is on the one hand, you're absolutely right that like if you think about the road speech or the actions, certainly that is something that's going to encourage what I find deeply worrisome tendencies to blur the boundary between man and machine.

00;34;26;10 - 00;34;47;11

Frank Pasquale

Right. That blurred boundary between persons and AI is a really worrisome thing that I think we really need to watch out for. On the other hand, what the movie does accomplish brilliantly, I think, is scenario analysis. Right. So you have a scenario analysis where, I mean, in some ways the super rainy LA is anticipating climate change. Perhaps. Or maybe we're just lucky that it does that.

00;34;47;12 - 00;35;05;21

Frank Pasquale

You know, it's like, well, maybe we're just lucky that that's we can imagine that. But the ways in which you're seeing the world of extreme inequality, right? I mean, you have sort of the off world colonies where people are being sent to either as slave labor or as being lured into. And you just you have to laugh when you see that, right?

00;35;05;22 - 00;35;24;01

Frank Pasquale

I mean, the implication is clear that it's going to be horrific out there. You know, like for anyone that's running a high line or whatever, you know, so that's going to be bad. And then you walk into the hushed chambers of Neander, Wallace or Tyrell and it's like, you know, one of these $100 million penthouses on Central Park in New York City, right?

00;35;24;02 - 00;35;58;04

Frank Pasquale

It's just absolute luxury and absolute or at least absolute isolation in space, which seems quite a premium. And the rest of the society or a lot of the society. And I think, you know, it all fits together in a way like I think that, you know, if we think, for example, of people in Ready Player One, either the book or the movie sort of retreating into a virtual world as the world around them falls apart, these films, I think, imagine scenarios where relationships between human beings become so difficult, so fraught, so hard to to manage.

00;35;58;10 - 00;36;14;22

Frank Pasquale

There's such a loss of social skills because so much of it is sort of like you can just interact with the cloud or holograms or some entity that you can control, that it all goes together. And I think that to me is the is the the use of film in a way, as a, you know, of course it doesn't need any use at all.

00;36;14;22 - 00;36;34;05

Frank Pasquale

It could be a purely esthetic, you know, that's the other cheer I would give for film. So by two cheers, our film a scenario analysis which warns us about what all this would bring with it. It might bring a lot of interesting, amazing technological developments, but also a lot of suffering and alienation. But my second share is just simply as a as an esthetic phenomenon.

00;36;34;09 - 00;37;06;10

Frank Pasquale

You know, Blade Runner, the first Blade Runner is amazing. I mean, absolutely, and the second one too, I deeply admire. But I think the first one particularly, I mean, that would not be out of place today in terms of thinking about futures and in terms of visual direction and the rest. It's just a beautiful film. I remember also the scene involving when one of the replicants goes to the with the the man who's developed the the monkey, the monkey companion, and this other companions, quote unquote, companions that are like the little robots, etc., and just the mess and dirt that are left.

00;37;06;10 - 00;37;38;26

Frank Pasquale

And yet, you know, you have this one consolation in terms of like this animate technology. All of that, I think is just so brilliantly put together. And it reminds me of a lot of fascinating dystopian music and dystopian art that, you know, give, give a sense of what happens when technology starts displacing the human and what's left for human beings as that displacement occurs and more and more of our mind thought resources go into these massive corporations or computers that are really the stars of the show.

00;37;38;29 - 00;37;59;00

Jonathan Hafetz

Yeah. I mean, that scene reminds me at the house of the sort of land of the Misfit Toys and yes, but yeah, it is it's I mean, yeah, the sort of dystopian vision of Blade Runner one especially. And you mentioned the original source material, the Philip K Dick novel, Do Androids Dream of Electric Sheep and to kind of bring that alive and and that that whole process itself is fascinating.

00;37;59;00 - 00;38;06;26

Jonathan Hafetz

I mean, there's so many potential offshoots. You also mentioned the Rutger Hauer Roy Batty speech at the end. Time to die.

00;38;06;29 - 00;38;15;00

Speaker 6

I see things you people wouldn't believe.

00;38;15;02 - 00;38;21;04

Speaker 6

Attack ships on fire or shoulder of a lion.

00;38;21;07 - 00;38;37;07

Speaker 6

I watched sea beams glittering it out in the ten hours a day. All those moments will be lost in time.

00;38;37;10 - 00;38;43;08

Speaker 6

Like tears.

00;38;43;11 - 00;38;48;18

Speaker 6

Right?

00;38;48;20 - 00;38;55;02

Speaker 6

Time to die.

00;38;55;04 - 00;39;14;01

Jonathan Hafetz

You know, one of the amazing things is moving speech, right? Which again, sort of shows the potential humanness, if you want to call it that or whatever, of these, of these replicants is that, of course, they're played by humans, right. And so, you know, like, you know, what I've read is like when Rucker Hauer gave this speech like people were moved to virtually to tears on set.

00;39;14;01 - 00;39;32;16

Jonathan Hafetz

So you have a film, you know, in the filmic world, you have this replicant giving this speech, showing these human qualities. But of course, he's played by a human. So there's I don't feel like there's some difficulties or challenges or issues when we consume media where humans play robots.

00;39;32;18 - 00;40;10;00

Frank Pasquale

Absolutely. Yeah. I almost I tried writing about at one point a rule that you would require robots to be played by robots in films, because I think it's so misleading in a way, to have the robot played by a person, and then to have people sort of work back from that to say, oh, if I run into Sophia, you know, the I think the robot that's like a female had that was sort of granted citizenship by Saudi Arabia and some, you know, public relations stunt or something, or being a 48, which I think was a replica replication by one person of their late life or something like that.

00;40;10;02 - 00;40;36;14

Frank Pasquale

You know, I mean, I think that these sorts of things, you know, it's really odd to me because you have people developing moral intuitions in response to a human and then being told they need to apply those same moral intuitions to these clearly inadequate or clearly entities that are not matched, even expressing or appearing like what human emotion is, much less actually feeling it.

00;40;36;15 - 00;41;05;13

Frank Pasquale

Right. And of course, I realize the actual feelings are something of the mystery themselves. But, you know, I think it's a real clue when you see the type of how alienating and how stuck in the uncanny valley. So many of even the best robotic creations now are. Right? And that's one of the things that's so fascinating to me, too, is like when you talk to roboticists and when I look at some of the literature and robotics and what's presented at international robotics conferences, it's remarkable how difficult it is to do things that just come naturally to us.

00;41;05;13 - 00;41;43;12

Frank Pasquale

And I think that's a clue to how different any sort of cognitive or simulated cognitive process is. So it does make it really hard. And I think that it's it would be useful. You know, I, I don't know exactly where to go with the idea of how we control representations of robots or if that's I mean, of course, obviously it can't be done in the US or places with robust free expression, but I would not be against seeing some places that do control the public sphere and artistic creation more that that being an aspect of it, because it is simply so misleading to have that sort of to have a person playing the robot and

00;41;43;12 - 00;41;59;05

Frank Pasquale

then being told that you should have moral intuitions about other things, you know, including maybe your toaster. Right? I mean, should I should I treat my toaster? Should I not throw out the toaster? Should I be really careful with it because, you know, potentially it could feel pain or it might have a pain sensor put on it at some point that says ouch.

00;41;59;06 - 00;42;16;23

Frank Pasquale

You know, if I kick it in to the curb or something, I don't think that's right. You know? And I think that to me, any existing robot is on the level of the toaster. Michael Frumkin and Zach Colangelo wrote an article where they say the robot has the moral consideration of a sock. Right? You know, just a just a piece of clothing.

00;42;16;26 - 00;42;38;01

Frank Pasquale

And yet we're, we're being fed this media that sort of goes in the opposite direction. And I think the final point I want to make about that is that I think that really it can even be interpreted as a form of propaganda in the sense of, you know, imagine a future where you have robotic police that were trying to prevent people who are protesting and sort of trying to disperse the crowd.

00;42;38;01 - 00;42;59;13

Frank Pasquale

And someone kicked over robotic police entity. Right. That is a humanoid robot that is policing. I don't think we want to live in a world where that person can be convicted of assault, can be put in jail for murder, can be treated like we would treat someone who destroyed a person. Right? We want to be able to treat that person like some of the destroyed property, but not as someone that created.

00;42;59;14 - 00;43;26;14

Frank Pasquale

And same with robot dogs, right? If someone rips the leg off of robot dog, it's not an animal cruelty, right? Just it's a totally different thing. And I think that's another thing that we really need to watch out for in this media as to how it could change moral intuitions that rather than being a protection of the vulnerable, in fact end up protecting the interests of those who are the most powerful, who cynically use robotics to control the more vulnerable people in the society.

00;43;26;16 - 00;43;45;24

Jonathan Hafetz

That's a good segue to you to talk a little bit about kind of the political economy of AI development. You talk about corporate control, those apex of these big tech corporations and their influence over states. So what do we learn from the film about political economy VII development and the corporate state influence? Right. We see there's a line from Blade Runner run.

00;43;45;25 - 00;44;01;15

Jonathan Hafetz

The first one, Commerce is our goal with Tyrell. More human than human is our motto. But we also see in various places, right. We have the extensive degree of of corporate control, especially in Blade Runner 2049 over the police department.

00;44;01;17 - 00;44;21;02

Frank Pasquale

Yeah. I mean, I want to start with more human than human, which I think is absolutely fascinating and so sinister. Right. Because in a way, if it's more human than human, then you start deserting humans in your life for the AI. Right? And this is all framed right now. For example, Mark Zuckerberg had a quote where he said the average person would want once 15 friends, but they only have three or something.

00;44;21;02 - 00;44;43;21

Frank Pasquale

And so therefore use my AI, use my sort of chatbot personalities, and they can be your friend. And to me, it's like that is creepy as its frame just in its own initial framing. But it's terrifying. If we think about the possibility of people with 15 friends sacrificing five of them for AI chatbot interactions. So let's say it's much more supportive.

00;44;43;22 - 00;45;01;07

Frank Pasquale

It's much wittier than my friends, knows a lot more than my friends, you know, brings up interesting tidbits, etc. we don't want that. You know, I don't really think we don't want that. But why? You know, why is it happening? It's happening. I think because of exactly the political, economic dimensions that you mentioned and that are really clear in this, these movies.

00;45;01;09 - 00;45;46;02

Frank Pasquale

It's happening because it's all driven by profit. And right now I think this is a real tragedy right now. If you think about the comparative technological development of China and the US right now, a lot of investment in China is going toward integration of AI with small process improvements to logistics and farming and mining and all sorts of other things that are starting to pay off huge benefits, both in terms of like performance, but also in terms of developing an overall innovation ecosystem that, you know, is electrifying huge amounts of transport and other aspects of society in a way that's in the direction of being climate friendly, etc..

00;45;46;03 - 00;46;09;14

Frank Pasquale

Right? You have all of that and also is creating an ecosystem where there's a huge amount of ability to direct machinery toward the states and people's ends, versus I think in the US, you have this incredible investment in AI. And what I'm thinking, what almost everyone now calls a bubble, right? That it's just like pure AI. And unfortunately, there's just a lot of things that pure I can't do, right?

00;46;09;15 - 00;46;35;19

Frank Pasquale

Like I can't get ChatGPT to do my laundry or to vacuum the house or to drive me around, but I sure could be induced. Perhaps. Maybe the model is to say it's speaks to me more in a more friendly way, in a more informing way than 50% of my friends. So I'll spend 50% of my more time with that, and then maybe I'll buy the the $50 a month model as opposed to a $20 a month model.

00;46;35;19 - 00;46;55;08

Frank Pasquale

ET cetera. Right. And to me, that is a huge problem. And, you know, it's I think it will all end in tears in various ways. But it just to me shows a deep problem with like, the technological direction of the US toward advancing things that I think are like the Blade Runner future, which are exactly these types of eyes.

00;46;55;09 - 00;47;15;27

Frank Pasquale

Now, of course, the difference, though, that I should say, is that the point about doing labor, that is something that is, is more in the direction of AI that I think is leaving the human estate. You know, it's like doing works that is difficult, dirty and dangerous. So humans don't have to do it. Then I'm all in favor of when it comes to silicon robots or robots based in metal and silicon.

00;47;15;27 - 00;47;21;06

Frank Pasquale

But you know, it's of course a whole different story when it's actual organic beings. Yeah.

00;47;21;14 - 00;47;43;10

Jonathan Hafetz

The other thing where I think the film is visionary on and in some ways more reflects what's the actual current realities in the area of data and surveillance. Right. And these are topics written about and Blade Runner. And part of its dystopian vision in the first film is this industrial bureaucracy of images, photos, dossiers in 2049, runner 2049.

00;47;43;12 - 00;48;10;06

Jonathan Hafetz

The Wallace Corporation has, like almost all encompassing panopticon like access to information, and it's got this massive data infrastructure for surveillance and control, even over government data. Right. That's another. So the private sector, right. It has access to some ways legally, some ways illegally to the LAPD data. So, you know, and they learn so much. Right. Even the the holographic companion for K.

00;48;10;08 - 00;48;35;19

Jonathan Hafetz

Right. It's a Wallace product. Right. And so the data that it gathers from K, this relationship with K gets fed back to Wallace. There's also like surveillance drones. You know they have just massive sort of scope of information. They record everything right. Even the conversations the initial conversation when Rachel and Decker met in Blade Runner one had been recorded and it was like played back right in Blade Runner 2049.

00;48;35;19 - 00;48;49;01

Jonathan Hafetz

So, yeah, I mean, I'd love for you to just kind of talk about this sort of data, bulk data, mass data surveillance dynamic of the two films. And you know, what you see there and how that kind of reflects or doesn't reflect, you know, what's happening.

00;48;49;03 - 00;49;15;20

Frank Pasquale

Yeah. Yeah. That's a it's a terrific angle on all this because it is the data that powers all this stuff. And so I think that with respect to the capturing everything, it reminds me of the NSA and remember all the stories about the NSA and how there was this big worry that, you know, you had and valid concerns that you had the NSA like being able to tap the phone wires, the internet, you know, Google, all the rest.

00;49;15;20 - 00;49;37;25

Frank Pasquale

And I remember when her came out, which was just about the time of the stolen revelations, I remember thinking the way they programed her was by using all those phone conversations. Right. That's something that I think people are now worried about with respect to their Gmail conversations and all the rest. Apparently, there's some very obscure setting within Gmail that you need to turn off if you want to keep it from learning training AI.

00;49;38;00 - 00;49;58;18

Frank Pasquale

I haven't confirmed this yet, but I'll be looking into it soon, and I think that that is a huge issue that we have to think about is who is controlling this data? How is it being used? What are the guardrails? I don't even know if the Privacy and Civil Liberties Oversight Board is still in existence right now. You know, so I don't even know if there's really effective regulation on this in the US.

00;49;58;19 - 00;50;23;20

Frank Pasquale

I think even in the EU framework, you know, the lot of times the the state use can be not nearly as regulated as private use. And even there the private use is often it's really hard to figure out exactly what's going on with it. And so I think that that's a real problem that we face. And that the other thing that I think is fascinating about it is that it used to be that we would worry that there would be this press conference effect where if we knew everything was being recorded, we'd be super careful.

00;50;23;24 - 00;50;41;18

Frank Pasquale

And that would make everyone duller and, you know, not speaking out, etc.. I think that instead, what ends up happening is that you end up having just people can't do that all the time and, you know, then you or it becomes really difficult to do that all the time unless you're really getting some sort of stimulus and reward, etc..

00;50;41;19 - 00;51;09;20

Frank Pasquale

And the other thing that I think is used in a really artistic way in the first film is when they keep going back to the initial conf test for for Kowalski and Kowalski. Right? They keep going back to that and they keep replaying it and it served to me, it's like it sort of shows how when you save everything, you start creating these possibilities for people to start obsessing and looping over and over again certain memories.

00;51;09;20 - 00;51;28;17

Frank Pasquale

And there are Black Mirror episodes on this, of course, about the grain, you know, the sort of thing that was being put in people's ear and, you know, the possibility of these glasses that record everything as well, the whole life, logging literature, all of them, I think, lead to a thought that we're better off with a lot more ephemerality.

00;51;28;19 - 00;51;55;21

Frank Pasquale

We're better off with a world where, like, it may seem like we want to record everything. We see these images of concerts where 90% of the crowd is looking at their phones, looking at the singer or something like that, right? Because they want to be sure they've captured the moment exactly the way they want to capture it. But to me, it is yet again part of this world where we got the technology that promised this recorded present.

00;51;55;21 - 00;52;17;13

Frank Pasquale

And we thought that would be great because we wouldn't have to forget anything. But what we learn is that it creates all of these other complications and liabilities and difficulties, and I think it pushes us toward a world where we just don't even engage, or at least wanting world where we have a lot less recorded. And that, I think, is a really important lesson of the film as well.

00;52;17;15 - 00;52;46;21

Jonathan Hafetz

Let's talk about just how the films speak to the possibility of a positive, post-human future, right? If they do it all, and how they intersect with test ideology, right? Transhumanism, pianism, singularity, cosmos, rationalism, effective altruism, and long term ism ideology, which is the cluster of iterated beliefs that have become so influential in Silicon Valley AI researcher and some future techno utopian movement.

00;52;46;21 - 00;52;53;22

Jonathan Hafetz

So is there some positive posthuman future that the film speak to, or is it just pure dystopian Asom.

00;52;53;25 - 00;53;08;20

Frank Pasquale

I'm very much on the side of pure dystopian, but let me try to get myself out of that hole because I get myself out of that despair. I recently read this book by Lauren Goldman called The Principle of Political Hope, and it's all about how one has to have hope. You know, one has to have some hope in the midst of it all.

00;53;08;20 - 00;53;43;01

Frank Pasquale

And probably there's a principle of technological hope to. But let me start with why it's so hard to have it. And I think the reason one reason was hard is when you mentioned, for example, long term ism out of that bundle. I mean, the long term is utilitarian. Calculus is a really rough one for current humans, right? And you see it in the concept of the, the ravine and the three body problem, or in the jackpot and William Gibson, there are these ideas that maybe we're going to have this disaster that wipes out 90% of people, but after that, you know, there will be this incredible efflorescence of human learning and technological advance, etc..

00;53;43;03 - 00;54;02;02

Frank Pasquale

And from a long term perspective, that's okay, right? If you have a different if you if you have something with lifestyle and everybody, but in the end it leads to something that is far, far greater and wonderful in the future. That's just the brakes, that's, you know, some sort of secularized soteriology, to use the term of thinking about how we'd explain, like terrible things happening.

00;54;02;02 - 00;54;24;15

Frank Pasquale

And so, you know, that's really worrisome. I don't want that. You know, I don't I don't agree with that long term ideology at all. And I think also in terms of like singularity, that to me, almost always read as an insistence that human beings merge with machines, because that would be the most efficient way to do things like space explorations, which is the cosmos aspect of the discrete ideology and other things like that.

00;54;24;16 - 00;54;55;28

Frank Pasquale

Right. So that to me is is all worrisome and all undue pressure and danger on our current global civilization of over a billion people. Where I do see hope, though, is that I think that if to the extent you can try to direct all of these amazing technologies toward improving people's living standards, it's quite encouraging. There's a recent article, I think, coauthored by Jason Hickel, on how much it would cost to provide decent living standards to everyone, everyone on Earth.

00;54;55;28 - 00;55;29;03

Frank Pasquale

And he he estimates with a coauthor that it's like a redirection of one third of the world's resources and energy right from where it currently is, to making sure that everyone has what they define as decent living standards. And I think that that's a real hope. And, you know, when you think about, like, Wallace's start in farming and being a brilliant farmer, that to me is is part of this, this glimmer, this vision of where things could have gone much better, you know, and in the same way that some of the characters look back on their memories or the the sort of the photographs with nostalgia about, oh, imagine that world, etc. I think maybe

00;55;29;05 - 00;56;00;06

Frank Pasquale

we're called upon to look at those arcs of character that somehow got shifted and changed. And I'm sure if I had read the novel, and I have to confess that I have not read of Electric Sheep, but I imagine that that might be part of it, and this conversation is going to turn me to reading it finally, to look for that, to look for exactly that, to see, you know, did even someone as pessimistic or apocalyptic as Dick could even he have imagined some alternative paths that we could have gone down, that the society could have gone down?

00;56;00;06 - 00;56;08;15

Frank Pasquale

That would have been much better. And I think that maybe is the is the hope that one can find at the bottom of the Pandora's box that is the Blade Runner universe.

00;56;08;22 - 00;56;32;23

Jonathan Hafetz

They really do with the Wallace character, right? I mean, it really is a good kind of character depiction of of the Silicon Valley figure, right, as we've seen. Right. Sort of altruistic in the beginning and then becoming kind of corrupted by power and desire for godlike influence. I mean, I do feel like, you know, without mentioning names, I feel like we do see that now, you know, a number of different people out there.

00;56;32;25 - 00;56;57;27

Frank Pasquale

Yeah. No, it's true. I mean, you definitely see that. I mean, in all of the stories of how quickly things like effective altruism were diverted toward things like, well, we've got to have this castle in the countryside because that's going to, you know, that's going to really influence people and make them. And I think that's it's so easy, you know, I mean, and by the way, Jamaica philosophical point about it, I mean, I think that the problem is very much a utilitarian outlook, right?

00;56;57;28 - 00;57;21;17

Frank Pasquale

Once you go really hard utilitarian, and once you sufficiently expand your idea of what utility or pleasure, whatever your end point is, could be, there's a huge, weird range of futures that you could get. You could try to advance. And I think that to me is the is the issue. You know, it's part of the issue. That exactly is what you're identifying about that that diversion from recognizable and clear helping humanity.

00;57;21;18 - 00;57;43;07

Frank Pasquale

Like what? U.S. agency for International Development does. Right. Very clear, obvious ways in which it prevents people from starving and keeps kids from getting tuberculosis. And the rest, once you get diverted to that to like, we're going to space, colonize the universe, there's a lot of very, very worrisome ways that that could be directed and could harm people in general.

00;57;43;09 - 00;57;47;27

Jonathan Hafetz

Frank, any other points on the two movies you want to pick up on?

00;57;48;00 - 00;58;06;18

Frank Pasquale

You know, this has been a wonderful interview. I just want to thank you for so acutely and insightfully connecting some of the work that I've done over, over many years to this. The last thing that I guess I would say is that, you know, the first article I wrote that kind of got me involved in some of these sci fi themes goes back to 2002, and it's called Two Concepts of Immortality.

00;58;06;18 - 00;58;23;17

Frank Pasquale

And one of the ideas of this article is that, you know, there's a positive way of being immortal, which is that you could be remembered for a long time, or at least aspiring to mortality, is being remembered for a long time for something that you did that was hopefully relatively good and a negative. Immortality is just trying to avoid death in whatever way possible.

00;58;23;17 - 00;58;47;19

Frank Pasquale

And I think that what the film is dramatized so well is that all of these ways that people imagine of technologically avoiding death, be it, you know, downloading your brain into an Android or developing, you know, endless biotech that lets you transplant out your heart and your lungs and your kidneys and maybe your brain as they go, that those lead to really dystopia, enigma and unequal futures.

00;58;47;20 - 00;59;03;18

Frank Pasquale

At least it's very hard to imagine future without those sorts of futures if you go for the negative immortality. And so I think that's another way in which I would see a moral in the film, although I primarily value them, as I mentioned earlier, as a static achievements than as moral guides.

00;59;03;19 - 00;59;29;17

Jonathan Hafetz

Yeah. Well, there's a morality, I think, in the, in sort of the K character at the end where he serves, you know, he enables Harrison Ford reunite with his child and he resists the large scale visions. Right? He resists the he resists Wallace Corporation, he resists the rebels. Right. And they're trying to find the child so they can use them for their large rebel movement and just goes really small scale and basically says it's about, you know, a father reuniting with his child.

00;59;29;17 - 00;59;33;12

Jonathan Hafetz

So I don't know. I don't know where that fits on that. But I felt that was sort of like the moral ending.

00;59;33;17 - 00;59;35;22

Frank Pasquale

Totally agree. Yes, I agree.

00;59;35;24 - 00;59;44;08

Jonathan Hafetz

Well, Frank, it's been so great to have you on the podcast and to share your vast knowledge of these areas and insights. So thanks again for coming on.

00;59;44;11 - 00;59;48;22

Frank Pasquale

Oh you're welcome, Jonathan, wonderful podcast and thanks so much. It's been a real pleasure. Thanks.