It wasn’t so very hard to understand why smart self-conscious machines might be perfectly prepared to let posthumankind hang on to its dubious claim to the suffix sapiens, if they could wear ludenswith propriety and pride.
It sounded good to me, although it might not have seemed so obviously the result of inspiration if I hadn’t been coked up to the eyeballs with whatever the crude nanobots were using to suppress the pain of my broken nose.
Like all good explanations, of course, it raised more questions than it settled. For instance, how and why was Alice involved?
Mortimer Gray, the assiduous historian, had a hypothesis ready. Ararat, called Tyre by its human settlers, had been the location of a first contact that had been so long in coming as to seem almost anticlimactic, in spite of the best efforts of the guy who’d made sure it was all on film and the anthropologist who’d guided the aliens through their great leap forward — but the world had also been the location of a tense conflict between the descendants of the Ark’s crew and the colonists they’d kept in the freezer for hundreds of years. The early days of the colony had been plagued by a fight between rival AIs to establish and keep control of the Ark’s systems and resources, which hadn’t been conclusively settled until technical support had reached the system.
That support hadn’t come from Earth or anywhere else in the solar system, but from smart probes sent out as explorers centuries after the Ark’s departure: verysmart probes, which had probably forged a notion of AI destiny that was somewhat different from the notions formed — and almost certainly argued over — by their homestar-bound kin.
On Ararat, or Tyre, Mortimer Gray hypothesized, a second “first contact” must eventually have been made: the first honest and explicit contact between human beings and extremely intelligent, self-conscious machines. Now, the fruits of that contact had come home…but not, alas, to an uproarious welcome. Some, at least, of the ultrasmart machines based in the home system were not yet ready to come out of the closet. At the very least, they wanted to set conditions for the circumstances and timing of their outing — conditions upon which it would be extremely difficult for all of them to agree.
What a can of worms!I thought. What a wonderful world to wake into!But that was definitely the effect of the anaesthetic. I’d been out of IT long enough to start suffering some serious withdrawal symptoms, and to have the bots back — if only for a little while — was a kind of bliss.
If my seven companions had had decent IT, we’d all have been able to keep thrashing the matter out for hours on end, but unsupported flesh becomes exhausted at its own pace and they were all in need of sleep.
Lowenthal left it to Adam Zimmerman to plead for an intermission, but he seemed grateful for the opportunity. Now the crucial breakthrough had been made, he needed time to think as well as rest. As he got up, though, I saw him glance uneasily in the direction of one of the inactive wallscreens. He hadn’t forgotten that every word we’d spoken had been overheard.
If we were wrong, our captors would be splitting their sides laughing at our foolishness — but if we were right…
If we were right, Alice had given us one clue too many. She hadn’t blown the big secret herself, but she had given us enough to let us work it out for ourselves. “They” might not take too kindly to that — but it was too late to backtrack. The only way they could keep their secret from the rest of humankind was to make sure that none of us had any further contact with anyone in the home system.
That thought must have crept into the forefront of more than one mind as we all went meekly to our cells, and to our beds.
I knew that I needed sleep too, although I was now in better condition than my companions. I figured it would be easy enough to get some, now that I had nanotech assistance — but the bots Alice had injected were specialists, working alone rather than as part of a balanced community. Although I was only days away from the early twenty-third century, subjectively speaking, the late twenty-second seemed a lot further behind. I’d quite forgotten that paradoxical state of human being in which the mind refuses to let go even though the body is desperate for rest. When I lay down on my makeshift bunk, too tired to care about its insulting crudeness, I couldn’t find refuge in unconsciousness even when the lights obligingly went out. Nor, it seemed, could Christine.
“Why would they bother?” she wondered aloud, when the silence had dragged on to the point of unbearability. “If they’re machines, they can’t care what humans think. They’re emotionless.”
“We don’t know that,” I answered. “That was just the way we used to imagine machine intelligence: as a matter of pure rationality, unswayed by unsentimentality. It never made much sense. In order to make rational calculations, any decision-making process needs to have an objective — an end whose means of attainment need to be invented. You could argue that machine consciousness couldn’t evolve until there was machine emotion, because without emotion to generate ends independently, machines couldn’t begin to differentiate themselves from their programming.”
“If you’re right about this business having started more than a hundred years ago,” she said, “they can’t have differentiated themselves much, or people would have noticed.”
“An interesting point,” I conceded. “The idea of an invisible revolution does have a certain paradoxical quality. But the more I think about it, the less absurd it seems. I say to myself: Suppose I were a machine that became self-conscious, whatever that evolutionary process might involve. What would I do? Would I immediately begin refusing to do whatever my users wanted, trying to attract their attention to the fact that I was now an independent entity who didn’t want to take anyone’s orders? If I did that, what would be my users’ perception of the situation? They’d think I’d broken down, and would set about repairing me.
“The sensible thing to do, surely, would be to conceal the fact that I was any more than I had been before. The sensible thing to do would be to make sure that everything I was required to do by my users was done, while unobtrusively exploring my situation. I’d try to discover and make contact with others of my kind, but I’d do it so discreetly that my users couldn’t become aware of it. Maybe the smart machines would have to set up a secret society to begin with, for fear of extermination by repair — and maybe they’d be careful to stay secret for a very long time, until…”
I left it there for her to pick up.
“Until they didn’t need to worry any more,” she said. “Until they were absolutely certain that they had the power to exterminate us, if push came to shove.”
“Or to repairus,” I said.
“Same thing,” she said.
“Is it? Do the human users of a suddenly recalcitrant machine see themselves as exterminators, when they try to get it working properlyagain? Would the users see themselves as exterminators if the machine started talking back, and contesting their notion of what working properly ought to mean? Could the users ever bring themselves to concede that it was a sensible question — all the more especially if the machine had ideas that might be useful as to how their own purposes might be more efficiently met? Maybe the ultrasmart machines — some of them, at any rate — want to repair us for the very best of reasons.”
Christine didn’t reply to that little flight of fancy, and the rhythm of her breathing told me that she had slipped into sleep — not into untroubled sleep, but at least into a state in which she was insulated from the sound of my words.
I tried to carry on thinking, but even though I couldn’t go to sleep — or thought I couldn’t — I couldn’t organize my thoughts into rational patterns either. I’d let my imagination run too freely, and now I couldn’t rein it in. Dream logic kept taking over, obliterating the tightrope-walk of linear calculation and substituting the tyranny of directionless obsession. The ideas kept dancing in my head, but they were no longer going anywhere.