Изменить стиль страницы

Ramos stared at him with a fierce edge in her eyes. Then her gaze swept away, embarrassed. "It’s ridiculous."

"What?" Tic persisted. "The first thing you thought of."

"I thought of something my roommate once said." Her face broke into a rueful smile — very sweet, very young at that second. "At Explorer Academy, my roommate Ullis was a cybernetics whiz. At least compared to me." The same rueful young smile. Pretty. Human.

"Ullis said no one alive today has ever programmed an android from scratch. It’s too complicated to work out the nitty-gritty algorithms. Even if you look at simple actions, like bending over to pick something up, there’s so much tricky coordination of the arms, the legs, the waist, the hand, the eyes… well, the companies that manufacture androids have hundreds of programmers on staff, and even they don’t start from zero when they build a new model. They start from last year’s model… which was based on the previous year, and so on, back three or four centuries."

"Ah," Tic said. "That explains why robot thoughts always feel so endearingly old-fashioned."

Ramos gave him a bemused look. I leapt in with a question before she started thinking my mentor was tico. "What does this programming stuff have to do with homicidal androids?"

Ramos said, "Demoth isn’t the first place androids have been used as killers. And every time it happens, it always follows the same pattern. Since it’s so difficult for anyone to program robots from scratch, Ullis told me that murderers have to start with off-the-shelf android brains. They don’t program a robot, they reprogram it… override a few instructions while leaving almost all the basic programming intact. The key part of turning a robot into a killer is to override the safeguards that manufacturers build into every android brain: don’t hit sentient beings, don’t squeeze them too hard, don’t push them off cliffs, things like that. Ullis said the original manufacturers program all those things separately — it’s nonsense to think there’s a single do not kill circuit that covers every dangerous act. Machines don’t work that way; they need hundreds of separate instructions. Don’t strike humans with more than X newtons of force. Don’t squeeze humans with more than Y kilopascals of pressure. Each possibility has to be clearly spelled out."

"Poor simple dears," Tic murmured. "Although I’m afraid I don’t see what point you’re making."

"Ullis explained it to me this way," Ramos said. "The bad guys reprogram standard androids so their robot brains don’t mind splattering someone with acid. But suppose the programmer doesn’t think to override the standard safeguards against hitting people. When the robot attacks, you scream, ‘Stop, you’re hitting me!’… even if it hasn’t touched you. If you’re lucky, some cease-and-desist event handler will kick in to shut the bastard down: Must not hit humans. Must stop whatever I’m doing."

"That sounds like a god-awful long shot," I muttered.

"Especially when you’re staring down a jelly gun’s mouth."

"Not at all," Tic said slowly. "It gives the robots an excuse to do the decent thing." Ramos and I stared at him.

"Machines know right from wrong," he assured us. "It grieves them terribly when someone has programmed them to hurt people. If you give them the smallest opening to overcome that programming, they’ll take it."

"Uh-huh." Ramos was two hairs from dumbstruck. "You think machines have the capacity for independent moral judgment?"

"More than people," Tic replied. He gave her a long cool look. "And that’s what popped into your mind the instant you thought about killer robots?"

"I told you it was stupid," Ramos said. "Trying to stop them from shooting you by yelling, ‘Ooo, you’re drowning me!’ Ridiculous."

"Absolutely," Tic agreed, amiable as the sun. "Which is why you must come with us if your probes find anything. Just to see."

"Oh," Ramos glowered, "I’m supposed to hope we meet homicidal androids… to test some silly remark my roommate made ten years ago?"

"No," Tic said. "To see if the first thing to cross your mind was a meaningless mental belch, or the universe trying to tell you something. That’s worth finding out, Ramos. Worth learning if you’re a poor vekker doomed to slog for every lumen of enlightenment, or if some god occasionally whispers into your gnarled little ear."

He settled back in his chair, closed his eyes and both ear-sheaths, then folded his hands across his belly: a man who had finished with a conversation and was precious pleased with his side of it. Ramos turned to me, and asked quietly, "Is he crazy?"

"He wants to be," I said.

Tic’s smile twitched a notch higher, but his eyes stayed closed.

"Hmph." She stared at Tic across the table. "I’ve had my share of escorting senile old coots into dangerous places. I sympathize with you, Faye."

"Tic is definitely not senile," I told her. "But you’re still welcome to help me escort him. Would you like to come? On an irresponsible adventure, just to feel your heart beat faster?" I gave her hand a motherly pat. Well… motherlyish. "And don’t worry you might turn out useless. I promise, when androids attack I’ll let you be my human shield."

"Oh, in that case…" She laughed. Lightly. But keeping her eyes on me. "You think I should go?"

"Lord Almighty," I answered, "don’t ask me for advice. I’m the queen of thoughtless impulse." Then an impulse. "Yes, I think you should go."

"Well then. Irresponsibility. Just this once."

And that was very much that.

As we were finishing breakfast, our two ScrambleTac bodyguards put in an appearance, asking what we intended to do next. They were a human wife-and-husband team, Paulette G. and Daunt L. of the Clan Du… which meant they had more husbands and wives back in Bonaventure. In the years after the plague, I wasn’t the only hothead to light on group marriage as a way to give society the crank.

But if Paulette and Daunt had ever played the jeering rebels, they were far past it now. By-the-book police types down to the crotch tattoos. If I had suddenly found myself stuck in a cozy resort with one of my spouses, I know what I would have done; but Paulette and Daunt told us they’d spent the night conducting a more thorough search of Maya’s room, collecting hairs and dirt specks the housecleaning servo had missed, then dismantling the servo itself for more samples. When that was finished, they took shifts, one sleeping while the other prowled the grounds in search of acid-blasting androids.

A jolly old evening. You can always hope they were lying.

After breakfast, we went for a walk around town… by which I mean Tic and Festina ragged on me for a tour of my childhood tree-forts/skating rinks/skipping areas/make-out spots, till they wore me down. Not that I could show them much of the town I’d known. Twenty-one years had stampeded past since I said good riddance to Sallysweet River — years with heavy feet, trampling down defenseless places where kids played. Tree-forts had got cut flat to make room for ski chalets. Skating rinks were moved far downriver, where shouting and laughing wouldn’t annoy the tourists. The skipping areas were gone too: my junior school had expanded with two new domes plunk on top of the old playground. As for make-out spots… I sure as sin wasn’t going to check on those with two ScrambleTacs looking over my shoulder. Or Tic. Or Festina.

Instead, we wandered aimless-blameless, with me trying hard not to sound like some old fart, bemoaning the things that had changed. A dozen new stores. New housing, especially near the mine, which had acquired a slew of unmarked outbuildings. All the tourist facilities, with paintings and holos and sculptures of my father, lined up in every window… most of them using that creepy artist’s trick where the eyes follow you.