Appeal to Authority

Every technology has its dangers and its benefits. The most beneficial are often the most dangerous… and the most irresistible.
__________________________________________________

Dear Madame President,

This moment demands greatness, anything less will fail. Everything we know, everything we hold dear, is at stake. Our nation is at war and we cannot know if we will win or lose, but we do know the cost will be unimaginably high. I have reflected for a long time before daring to write you. Know that my words are chosen with great care, not because of you or me, but because of the country we both serve and the repercussions of our actions that will echo through the ages.

In a previous generation on a previous project at the crux of a previous war a predecessor of mine wrote a predecessor of yours with serious concerns. The United States was about to unleash the atomic bomb on the world. The scientists who were working on the project feared what would happen when such power was unleashed. Some of them even considered the possibility that the atmosphere might ignite and kill all of humanity after the first detonation. Their worst fears proved unfounded, but there were plenty of founded fears that still remained. We suffered through a Cold War and the constant threat of nuclear strikes. We strove to keep the wrong people and the wrong countries from gaining access to nuclear technology — whether for weapons or energy — because we feared what we had created.

You and I stand at a similar crux now. On this day I write to you with all of the same fears as my predecessors. Yet my fears are founded on the cornerstone of history and experience. I have seen what nuclear technology has done to the world. I know the number of lives lost in the attacks. I know the money spent to ensure mutual destruction. I know the constant fear of a bomb or missile falling from the sky and erasing all life. The fears of the scientists working on the Manhattan project were based in theory; my fears are based on fact.

So, Madame President, it is with regret that I must decline your offer. Though I dearly love my country and I trust you as a person, I cannot trust those with whom you would share my work. Nor can I trust your successor, nor the intelligence community, nor the international black market that might tempt the intelligence community. To be sure my work would proceed much more quickly with governmental funding. To be sure I would be able to accomplish more in defense of our nation if I could add to my work the work of others in my field. To be sure the work would get done, but the outcome of that work is far from sure.

I have an obligation — even a sacred trust — to ensure that what I create does not follow the path of history. I cannot start another Cold War. I cannot offer another tool for extremists around the world to vie for. I cannot allow my work to become the death knell for humanity. And that is quite possibly what it could be. We don’t know — just as we did not know if the atomic bomb would ignite the atmosphere and we did not know if the Large Hadron Collider would produce micro black holes with enough gravity to rip apart the Earth — and it is the very fact that we don’t know the worst that could happen that I cannot be complicit in that risk.

As Dr. Stephen Hawking said, "The development of full artificial intelligence could spell the end of the human race.”

__________________________________________________

Read the rest here.