‘And when that happens, what can we expect?’ Bello asked.
‘Within two or three hours, the planet will be entirely in the hands of the Benign Superintelligence. Human affairs will be re-organised, within seconds, into their optimal state: no more war, no more sorrow, no more death.’
‘But how can we be sure of that?’ Bello probed fearlessly. ‘Computers are capable of all kinds of errors and mistakes.’
‘Computers built and programmed by humans, yes,’ Esch conceded. ‘But remember, by definition, every element in the ascending chain of Godlets will be superior to its predecessor, in both intelligence and benignity. We’ve done the theoretical groundwork; now we’re assembling the final pieces that will start the chain reaction. The endpoint is simply a matter of logic: God is coming into being. There is no disputing that, and there is no stopping it.’
Nasim said, ‘Which of these reminds you most of a cat?’
Fariba examined the four photographs: the Eiffel Tower, a parrot in a cage, the Great Pyramid of Giza and the Empire State Building. A patch of light on the screen tracked her focus of attention as it moved from image to image.
‘The pyramid most of all,’ Fariba said, ‘because of the thing Egyptians had for cats. Second is the bird, because it’s a pet too, and because the cat might want to eat it.’
Nasim gestured for a reset before the urge to say ‘thank you’ or ‘good’ or ‘correct’ could start to nag at her. Four new pictures appeared: a bicycle lying flat on the road with some scattered groceries around it; a wilted flower; a wrecking ball swinging towards an apartment block; and a very young girl holding the hand of an old woman bent over in pain.
‘Which of these pictures is the saddest?’ Nasim asked. Fariba, with no memory of the previous test, contemplated the images for a few seconds before replying, ‘Maybe the bicycle. The girl with the grandma is stronger emotionally, but I’m not sure if its sadness and its sweetness should cancel each other out, or stand alone!’
Nasim gestured reset and halt, and turned to Bahador.
‘I spend a few days playing football,’ he said, ‘and look what happens behind my back.’
‘A few days? It’s been five weeks!’ They’d had to fit water dispensers into the ghal’eha so all the Azimi groupies didn’t dehydrate as they helped beta-test the game. ‘What do you think?’
Bahador smiled. ‘It’s amazing. How many women did you side-load? ’
‘Twenty. Students mostly.’ Each recruit had lain in the MRI for ten hour-long sessions, going through a series of undemanding tasks: looking at pictures, reading short essays, listening to recorded speeches, answering simple questions. At night, the side-loading software had prodded and kneaded the inarticulate Blank Francine – assembled by Nasim from the HCP women – into her less tongue-tied Iranian cousin, Farsiphone Fariba: fluent in the written and spoken language, brimming with word associations, conversant with thousands of commonplace facts.
Fariba would never be mistaken for a philosopher or a poet. Conventional Proxies – which improvised around elaborate, branching scripts – could have far deeper, more convincing interactions with people, at least in situations for which they’d been tailor-made. But Fariba wasn’t meant to do anything as a stand-alone system. Conventional software would still provide the back-story, goals, memory and context, but Fariba would make it a thousand times easier for developers to construct a flexible character who wouldn’t lapse into embarrassing silence if the conversation moved beyond the range of possibilities envisaged in the script.
‘Can you make a few different versions?’ Bahador wondered. ‘With different weightings for the various side-loading subjects? That way the responses will always make sense, but they won’t be the same for every character that uses the modules.’
Nasim thought it over. ‘That’s a good idea. For a composite Proxy it will still be the script that determines most of its personality, but it will be an added attraction if we can offer a kind of library of low-level variations.’ Developers could decide for themselves whether they preferred a particular character to associate cats with birds, or with pyramids. Even if nothing crucial hung on the distinction, that would give them more control over the tone of the game.
Bahador was dripping sweat onto the carpet; he’d just come from a game and Nasim had called him in as he’d passed her office on the way to the showers. ‘I should let you go,’ she said.
He looked down at his damp shirt. ‘Sorry, I must stink.’ He headed for the door. ‘What you’ve done is terrific. And so is the Azimi game. We’re going to make a fortune!’ He waited until he was out in the corridor before adding, ‘I expect a pay rise.’
‘Maybe when you start doing real work again,’ Nasim called after him.
She sat playing with the demonstration module. She’d already run rigorous automated tests on it, yielding no surprises or major problems – but there was something addictive about chatting away to Fariba in person.
‘Which colour here makes you think of warm weather?’
‘If you could take only one of these items to a desert island, which one would you choose?’
‘Which of the first three pictures tells a story that’s completed by the fourth?’
The tests didn’t always have a single right answer, but Fariba always managed a sensible response. She possessed no narrative memories, and no sophisticated beliefs – but all the words and concepts she’d acquired were wired together in a perfectly reasonable way. If she had no depth to her, she sounded less like a simpleton than an amnesiac who hadn’t yet noticed her own plight.
The women who’d answered Nasim’s advertisement had been happy enough with the modest payments they’d received for their work – and it hadn’t exactly been arduous for them, once they’d grown used to the confined space of the scanner. Nasim didn’t feel that she’d exploited them, that she’d pillaged their brains in return for loose change; their language skills, common sense and general knowledge, however vital, had hardly been rare commodities. Literally millions of Tehrani women could have done the same job equally well.
Nevertheless, a little more than raw vocabulary and dry factual knowledge had rubbed off. Sometimes Fariba exhibited quirks of phrasing that came straight from Asa, or offered witticisms that Azita would not have disowned. Sometimes she seemed as warm as Farah, or as acerbic as Chalipa.
So what was the bottom line? Fariba had no long-term memory, and no sense of herself. When Nasim reset her after every test, she lost nothing, because there was nothing to lose. Even if she’d run uninterrupted for an hour or a day, the passage of time would have left no mark on her. It would be crazy to start treating her as if she had interests, goals and rights.
But was she conscious – as much as the women who’d helped build her would have been conscious if, for a few seconds, they’d forgotten themselves and focused entirely on their simple tasks: thinking of a word, matching a picture?
Nasim wasn’t sure. She was moving into territory where that prospect was no longer unthinkably remote; she had to tread carefully.
Still, at most it could only be a transient form of consciousness – with no conception of itself to underpin a fear of extinction. Splicing Fariba, and a thousand variants of her, into narratives in which they played no active part wouldn’t bolster their fragmentary minds into something more substantial; that was just the illusion that human players would receive. The Faribas would still live – if they lived at all – in an eternal present, doing their simple tasks over and over again, remembering nothing.