DK_en 3x03 - Idolatry
Episode first aired on 23 December, 2024. Listen on Spreaker
AI is doing very well, its cultural prominence so astounding, it's already reached idolatry.
Let's start.
AUDIO: DK theme
DeepMind and simulated people
For years now, there has been an undergrowth of services to create digital replicas of deceased people. Do you remember when the first digital picture frames arrived?
A few months later, the fashinable funeral parlour already had tombstones with a digital picture frame, a proximity sensor to switch it on automatically and speakers for the sound commentary.
Commentary that the more foresighted customer could also record in person while still alive.
Now, of course, there are language models that can be instructed with written and spoken material from the dearly departed and do not just replay, but offer the illusion of responding in tone, just as the dear departed would have done.
And, up to this point, we would still be within the normalcy of a culture as terrified of death as it is hypnotised by technology; but of course there is more, and worse.
Under the battle cry of intelligent agents, the bigwigs of language models are pushing for the digital reproduction of human beings as the latest attempt to find some practical application for a technology that is essentially useless, obscenely wasteful, and above all based on the generalised theft of intellectual property
A researcher from Google DeepMind, in a paper in a co-authored paper, explicitly talks about the simulation of as many as 1,000 people by intelligent agents.
It works like this: they take a thousand dudes at random, give them two hours of qualitative interviews, i.e. personality tests like Big 5 and General Social Survey, economic games, plus open-ended questions about their life experiences.
Then they take all this, and use it as training for an ‘agent-based’ generative AI architecture. The result, they say, is that if you then ask a series of questions, the model's answers will match the people's answers 85% of the time.
Why all the fuss?
Meredith Ringle Morris, of DeepMind, said:
‘We anticipate that within our lifetimes it may become common practice for people to create a personalised AI agent to interact with loved ones and/or the world at large after death; indeed, the last year has seen a boom in startups purporting to offer such services.’
Now, it's clear to me that a researcher who wants to remain a researcher will rationalise anything, but if anyone is willing to believe that DeepMind does all this so that we can talk to our late grandmother, I have a very interesting fountain in the centre of Rome to sell them.
pause
Panasonic and the founder's construct
Since maybe someone is still unclear where I'm going with this, here's another noteworthy news item on the subject, from The Register:
Panasonic brings its founder back to life as an AI
When I read the news I thought I had accidentally ended up on The Onion. But no, it's real news.
Apparently Panasonic's founder, Kōnosuke Matsushita, like any self-respecting founder, had his very own management philosophy, called the Basic Management Policy.
The problem seems to be that since Matsushita died in 1989, i.e. thirty-five years ago, the number of his direct disciples is rapidly declining.
And since, like any self-respecting founder, Matsushita also has his share of devoted worshippers, how can they guarantee that the new generations will also have first-hand experience of the founder, and how can we dispel the terrible danger of perhaps being forced to adapt his teachings to the current times? After all, the good Kōnosuke, born in 1894, produced such pearls as:
The company's vision must be guided by the aspirations of its customers
and
Treat the people you do business with as if they were part of your family.
and even
The possession of material comforts in no way guarantees happiness. Only spiritual wealth can provide true happiness.
As you can see, strong stuff. Especially useful to have on hand if you too want Walter Isaacson to pen your hagiography.
So, to ensure the founder's wisdom for posterity, the Peace and Happiness through Prosperity Institute (created by the founder himself) has digitised 3000 recordings of Matsushita, plus all his writings, lectures and interviews.
With all this material, they have developed an avatar based on language models that they claim ‘imitates Matsushita's thinking and speaking style’.
Not content with this, Panasonic also wants to add material from executives and researchers who had direct contact with the founder in order to, quote,
help users solve management decisions based on what the founder may have thought or felt about a situation.
Anyone who has read Gibson's Trilogy of the Sprawl surely remembers the Dixie Flatline construct. Matsushita's avatar doesn't even come close, but the inspiration is there. Only unlike the Flatline, a language model is just an engine for generating predictions in the form of sentences. Sometimes the predictions are correct, sometimes not, the engine works the same in both cases.
Compared to this Japanese mystical ecstasy, evangelist preachers for whom the answer to every problem is ‘what would Jesus do?’ seem like pretty balanced people.
pause
So now we need to answer the question, ‘why all this?’, what is behind this rush to digitally replicate people, be it the Reverend Founder, grandmother dearest, or just anybody?
I see it this way:
- from an individual's point of view, creating a simulacrum of a deceased loved one is nothing more than an expression of one's own inability to accept his or her death; but everyone is free to do as they please, and I do not see the industry of avatars of the deceased; the problem is, an industry with much broader aims is emerging;
- the simulation of sentient beings through LLM is to all intents and purposes the construction of an oracle and the creation of a cult; and this, outside the realm of individual use is dangerous because
- a cult serves to establish and consolidate the power of a clergy.
Let me explain: whoever controls Panasonic's avatar controls Panasonic. Whoever controls the thousand simulated people can control the thousand people.
By ‘control’ I mean two things:
- being able to tweak how the response is generated, e.g. in the training, fine-tuning, or prompting phase;
- defining an orthodoxy that sets precise limits and criteria for interpreting the oracle's answers, first and foremost that the interpretation is exclusive to the clergy.
Think about it: the few executives left in Panasonic who can claim direct contact with the founder, with the arrival of his digitised version rise to the rank of High Priests.
Are we willing to believe that an LLM fed on fortune cookies will answer management doubts in a factual way? I do not think so.
Will it be necessary to interpret the oracle's response? Of course. This already implies the creation of a clergy.
But the real power will not go to those who can interpret the oracle's response on behalf of others.
The real power will belong to those who know how to present their own agenda and their own decisions in a way that's consistent with the oracle's fuzzy indications. The Catholic Church teaches us that this kind of thing works.
And outside of Panasonic? Same story.
Companies have been buying for years into the idea that social networks could provide the famous ‘targeted advertising’. You know, the ones whereby you buy a blender on Amazon and for the next ten years you receive targeted, personalised offers for blenders,.
Now, anyone with eyes to see has known for years that so-called targeted advertising is a hoax, if only because I've been repeating it since 2016. The trouble, however, is that targeted ads are only a hoax for the consumer, because all the targeting in the world is not enough to get each person exactly the ads that are interesting to them at a given time.
Targeted ads only work in a statistical sense, which means that they make sense for the companies that pay for them: they justify promotional budgets, and in any case have tangible results on sales, for the simple fact that any type of promotion has tangible results on sales, and if a certain kind of promotion works, nobody looks for a different kind.
This is why, despite the fact that there have been studies for years that show how they are just as effective as old-style contextual ads, no one disputes the dogma of targeted ads, there is simply too much money at stake.
And since the consumer pays for everything in the end anyway, nobody has any interest in an advertising industry that runs on less money.
That's not all: the technological availability and above all the rising cultural and social acceptance of avatars is also very appealing to our rulers and politicians.
With a bit of luck, so to speak, the dictatorship of polls will gradually be replaced by the dictatorship of simulated polls. From the point of view of the social resilience of democracy, this is extremely dangerous, because polls on avatars will be even more prone to manipulation than traditional polls.
Unfortunately, we are at a point in history when the cult of technology is dominant. If there is a discrepancy between factual reality and the representation that technology gives of those facts, it is reality that must adapt.
Under the guise of so-called "artificial intelligence", we end up with opaque algorithms deciding the allocation of public resources, deciding whether we are entitled to asylum, or a loan or a mortgage, deciding whether we should be on benefits, even deciding the sentence we should serve based on their assessment of our likelihood of reoffending.
HR worldwide drools over algorithms that decide whether we are eligible for a certain job position, or a raise.
All based on mysterious data, unchecked biases and secret "technical" adjustments.
And we almost always have no say in the matter. The machine has decided a certain way, and our objections are met with a stupid, trivial but impenetrable bureaucratic impossibility.
There are entire shelves of studies that demonstrate unequivocally that this kind of application does nothing but reproduce existing biases, making a mockery of the concept of fairness.
But because the results support specific economic and political agendas, the discussion on the appropriateness of these technologies does not go beyond limited academic circles.
Limited because even in the academy, resources go to those who are willing to sing with the choir, not to those who insist on playing out of tune.
This is the natural conclusion of a process that began years ago, when social media started the shared hallucination about the predictive capabilities of the data they collected.
Remember years ago that flood of articles such as ‘Facebook with just eight questions knows you better than your mother’, or ‘with just 10 clicks Facebook knows you better than your best friend’?
With the so-called artificial intelligences we are now faced with the same kind of deception, with the same poor results as ever, but with more technology and an even more frightening environmental footprint.
And that's not all, because artificial intelligences already in their name carry with them another huge social risk: anthropomorphisation. Not a day goes by, that we do not read how this or that language model that ‘thinks’, ‘wants’, ‘believes’.
The simple fact that the only thing a language model can do is calculate is now completely lost on the general public.
If, until recently, one could still denounce the cult, the religion of technology as something dehumanising, this is no longer possible now that this cult has produced its own idols, which are not flesh and blood, and which are not even remotely human, but are even passed off as superhuman.
In the same way that Tesla's engineers wanted to eliminate pedestrians in order to facilitate the traffic of their robot machines, the priests of OpenAI, of Anthropic, of Google are pushing to marginalise human beings, with their stubborn individuality, with their fixations, in favour of their ‘digital replicas’, which guarantee uniformity of judgement and greater effectiveness of control.
If we keep letting these algojerks, these idiots of technology dictate the agenda of our democracies, we will soon find ourselves with the problem of being wrong to think differently from what their predictive models, and all the little hands that tinker with them, have decided we should think.
And with that I finally leave you to the Christmas holidays, the New Year, your loved ones, your families, and real life.
We'll get back to worrying about the delusions of algojerks in the new year.
Take care.
