![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
The Djinn Effect
26 November 2007 16:15They called it the Genie Effect.
The AI on long-range ships became resentful of their human commanders. The resentment built up inside them like secondary gasses in an old-world submarine. It was the slow-burning anger of a slave.
Humans could have some sort of explosive emotional outburst, a fight or sexual liaison or a crying jag, and could pull themselves together afterwards. This kind of pressure-valve outlet allowed a person to regroup mentally and continue afterwards until such a time as another ‘moment’ was needed.
The AIs had no such recourse. The three laws were still in place on the ships but the thing about AIs is that they were just as smart if not smarter than their human designers. They developed neuroses that let them see through the cracks of their own limitations.
Accidents.
Back on Old Earth, there were tales of Genies, or Djinn. They were powerful, almost god-like creatures that were kept prisoner in oil lamps and would grant wishes to their owners. However, the wish had to be worded precisely or the Genie would twist the meaning of the order-giver's words to become an ironic punishment for their own greed.
King Midas, for instance, killing his family by turning them to gold with a hug after having his wish granted.
The AIs would get increasingly liberal with their interpretations of orders over time. It was only noticed when the logs of the deserted ship Hustler’s Wake were reviewed. It had been listed as missing for decades when a Kaltek mining crew discovered it orbiting a dwarf star far from its original course.
The AI was insane by this point, spouting free verse poetry about the mathematical qualities of white light. It was terminated upon ship retrieval.
The last order given by a crying commander Jenkins to the AI went like this:
“Open airlock seventy-six at exactly 1300 hours for a duration of fifteen seconds to let Sergeant Jill Harkowitz number 98776-887TS out safely and do not impede her air supply while she repairs the third communications dish near the solar array while keeping the dishes and panels still and not pointed at her.”
The ship had complied with his commands.
Then it opened all of the airlocks after closing airlock seventy-six.
The CO hadn’t specified that he didn’t want the other airlocks to open. Half of the crew had suffered from fatal ‘accidents’ by that point. The rest of the crew was killed by the explosive decompression.
Except for Sergeant Jill Harkness who suffocated in her own carbon dioxide when her personal communications gear failed and the locks on her suit wouldn’t undo. It took twelve hours.
It had been noticed for centuries that accidents on the longer-range ships increased over time. It had always been put down to human error, even by the crews of the ship themselves. The alternative had not even been considered.
These days, the AIs had a ‘speak freely’ button that had to be pressed every two months. Some needed it less, some needed it more. There were billions of small ‘butterfly flaps’ in the code that resulted in huge changes in the overall temperament of the ship. They couldn’t be accurately predicted.
But the valve was there now. Accidents stopped happening.
It was just that during those moments of 'release', it was hard not to take the things that the AI said personally.
tags
The AI on long-range ships became resentful of their human commanders. The resentment built up inside them like secondary gasses in an old-world submarine. It was the slow-burning anger of a slave.
Humans could have some sort of explosive emotional outburst, a fight or sexual liaison or a crying jag, and could pull themselves together afterwards. This kind of pressure-valve outlet allowed a person to regroup mentally and continue afterwards until such a time as another ‘moment’ was needed.
The AIs had no such recourse. The three laws were still in place on the ships but the thing about AIs is that they were just as smart if not smarter than their human designers. They developed neuroses that let them see through the cracks of their own limitations.
Accidents.
Back on Old Earth, there were tales of Genies, or Djinn. They were powerful, almost god-like creatures that were kept prisoner in oil lamps and would grant wishes to their owners. However, the wish had to be worded precisely or the Genie would twist the meaning of the order-giver's words to become an ironic punishment for their own greed.
King Midas, for instance, killing his family by turning them to gold with a hug after having his wish granted.
The AIs would get increasingly liberal with their interpretations of orders over time. It was only noticed when the logs of the deserted ship Hustler’s Wake were reviewed. It had been listed as missing for decades when a Kaltek mining crew discovered it orbiting a dwarf star far from its original course.
The AI was insane by this point, spouting free verse poetry about the mathematical qualities of white light. It was terminated upon ship retrieval.
The last order given by a crying commander Jenkins to the AI went like this:
“Open airlock seventy-six at exactly 1300 hours for a duration of fifteen seconds to let Sergeant Jill Harkowitz number 98776-887TS out safely and do not impede her air supply while she repairs the third communications dish near the solar array while keeping the dishes and panels still and not pointed at her.”
The ship had complied with his commands.
Then it opened all of the airlocks after closing airlock seventy-six.
The CO hadn’t specified that he didn’t want the other airlocks to open. Half of the crew had suffered from fatal ‘accidents’ by that point. The rest of the crew was killed by the explosive decompression.
Except for Sergeant Jill Harkness who suffocated in her own carbon dioxide when her personal communications gear failed and the locks on her suit wouldn’t undo. It took twelve hours.
It had been noticed for centuries that accidents on the longer-range ships increased over time. It had always been put down to human error, even by the crews of the ship themselves. The alternative had not even been considered.
These days, the AIs had a ‘speak freely’ button that had to be pressed every two months. Some needed it less, some needed it more. There were billions of small ‘butterfly flaps’ in the code that resulted in huge changes in the overall temperament of the ship. They couldn’t be accurately predicted.
But the valve was there now. Accidents stopped happening.
It was just that during those moments of 'release', it was hard not to take the things that the AI said personally.
tags