On the End of Humanity
What makes humanity? And what does the world look like when humans no longer constitute it?
When people say that artificial intelligence may end humanity, they almost always mean a death. They picture the machine grown past us, indifferent or hostile, and the species swept aside the way we once swept aside the creatures beneath us. An artificial superintelligence so beyond comprehension humans lose all value to it or are even detrimental to its goals. The argument that follows takes no position on that picture and concedes it whatever it deserves. Its subject is a quieter ending, one that asks for no malice, fires no shot, and leaves every body in the world untouched.
Begin with a distinction on which the whole of it rests. There are humans, and there is humanity, and they are not the same kind of thing. Humans are organisms: so many billions of bodies, breathing and feeding and reproducing, each a discrete object in the world. Humanity is what those bodies generate when they are arranged in a particular way, as a flame is generated by a fire and is not any one of the burning things. It is emergent. It lives in the space between people rather than inside any of them.
The claim of this essay is that the emergent thing can be put out while every organism that composed it goes on living, and that we are already some way into the doing of it. Social media began the work, by remaking the manner in which human beings attend to one another. Artificial intelligence may complete it, by making the other person, across more and more of a life, unnecessary.
None of this requires the future to go badly in the ordinary sense. It is consistent with the future going extraordinarily well: with disease in retreat, with labour lightened, with the long human quarrel against scarcity finally won. The unease set out here survives all of that good news. In a way it is sharpened by it.
Consider what emergence means, since the argument is nothing without it. A human being is built out of cells and is, for all that, not a quantity of cells. Take a body apart, set each cell in its own petri dish with all that it needs, and keep every one of them alive for years. The cells persist. The person does not. What vanished when the body was disassembled was the arrangement itself, the unceasing traffic among the parts, and the person was only ever that traffic given a shape.
Humanity stands to human beings as the person stands to the cells. It is not delivered by the head-count of the living. It comes into being when the living are bound into a common world: when they speak and are answered, owe and are owed, wound and forgive, depend and are depended upon. Set the alternative out plainly. Every person on earth housed alone, each in a sealed and comfortable room, wanting for nothing, kept healthy and amused and mildly content until a painless end. No one in this world suffers. The organisms thrive; the actuarial tables have never read so well. Were a visitor from a planet far beyond our galaxy shown the scene and told that here was humanity, the honest correction would have to be offered. Here is a population. Here is a warehouse of contented animals. The humanity is the part that is missing from the room, and it is missing because it was never in any one room to begin with. It lived in the passage between them, and the passage has been sealed.
This is why the vocabulary of welfare, of happiness and health and length of days, cannot on its own tell us whether humanity is intact. Those measures are taken one organism at a time. They are silent about the arrangement, and the arrangement is the thing in question. A world can post the finest figure in every individual column and have quietly stopped, at the level where it matters, being a human world at all. If humanity is a matter of arrangement, then everything turns on the shape of the arrangement.
The shape can be drawn. Let each person be a point, and let every relationship between two people be a line joining them. What results is an immense graph, and at first glance it can seem that all that matters is how many lines it holds. It is not so. The lines carry weights. A friendship of thirty years and a stranger glimpsed once on a screen each count, in the crudest reckoning, as a single edge, and to treat them as equal is to have understood nothing at all.
What gives an edge its weight is recurrence, reciprocity, memory, and cost. A bond you return to and that returns to you, that recalls what passed between you the last time and will ask something of you the next, that can be damaged and must then be repaired: this is a heavy edge. A fleeting awareness of a face you will never meet is a thread that parts under the lightest load. Humanity is carried on the heavy edges. Two humans connected via a long line of heavy edges from node to node are closer than two humans directly connected via one very thin connection. Along those heavy edges run obligation, culture, correction, grief, inheritance, the whole freight of a shared life. A graph that is dense in the right way, thick with strong and recurring ties, holds the emergent thing aloft. A graph of universal but weightless contact, everyone grazing everyone exactly once, is for all its busyness flat, and across a flat graph nothing of the kind we are discussing can travel.
Seen in this light, what the past fifteen years did to us comes into focus.
The first thing to fix is that the wire itself is innocent. A complaint against screens, against digital communication as such, would be a weaker and different argument, and it is not the one being made. A message to a friend is a friend reached. The telephone, the letter, the group chat thick with the private jokes of people who know one another: these thicken the heavy edges, or at the very least leave them as they were. Early social media, whatever is now said against it, largely did the same. What it laid before you was the doing of people already inside your graph. Someone you had sat beside at school had bought a house, married, buried a parent. The contact was thin, yet a real edge lay beneath it, and the thread ran back to a person who was, in some attenuated way, yours.
What changed was the principle by which attention is handed out. The older feed was ordered by whom you knew. The thing that has replaced it is ordered by whatever holds the eye. The relationships were not deleted; they were demoted, pushed to a margin of the screen whose centre is now given over to whatever an optimiser has learned will keep you in your seat. The organising structure used to be a social graph. It is now an attention graph, and the two have almost nothing to do with one another.
The result is a condition with no precedent: knowledge without relationship. A man scrolls from a celebrity's renovation to a riot two thousand miles off to the grief of a stranger to a war, and every item concerns real people, and not one of them is anybody he will ever reach, owe, answer, or be answered by. He learns of multitudes and is bound to almost none of them. His awareness swells until it girdles the earth; his participation shrinks until it scarcely reaches the next room. He is seated before the entire spectacle of humanity as its spectator, and takes part in less and less of it.
Loneliness has many fathers, and it would be a crude history that laid the whole of modern isolation at one door; the shape of our cities, the hours of our work, the slow withdrawal of the old institutions each had a hand in it. The feed did not merely keep this company. It found a drift already underway and handed it an engine, a machine that rewards the disembedded and the distant precisely because they are the cheapest to supply and pay best in attention. So it hurried along and made ordinary what it found.
On the graph the motion runs the wrong way twice over. Numberless weightless threads are spun outward to strangers and spectacles, while the heavy edges near at hand, the ones that want time and friction and presence to keep, are left untended for lack of the hours now spent elsewhere. Attention is a finite stock. Every measure of it laid before the feed is a measure withdrawn from the people through whom one is actually fastened to the world. The graph grows broader and lighter at the same stroke.
It reaches further and holds less.
And yet, for all that it does to the graph, social media still runs on what human beings make. Artificial intelligence poses a deeper question. Every instrument of communication until now has stood between people. A book holds the words of someone who wrote them, perhaps centuries dead, and hands them to someone who reads. A telephone bears a voice from one end of a wire to a person at the other. A feed, for all its mischief, still ranks utterances that human beings produced. In each of them the machine is a medium, and at both of its ends there is a person. Take the machine away and what is lost is the ease of the meeting; the two people, and the chance of their meeting, remain.
Artificial intelligence is the first of these instruments that needs no person at the far end. It does not keep or carry or rank an utterance made by someone else. It makes the utterance. Lay the sequence out and the break is plain: a book preserves what a human said, a telephone transmits what a human says, a feed ranks what humans have said, and the new machine simply says it, generating on demand the conversation, the counsel, the lesson, the comfort, the company. It enters the graph as a point of a new kind, a synthetic node able to occupy, fluently and without tiring, the place another person used to hold.
Whether the machine is clever or kind is beside the point; grant that it is richly both. The point is that nearly everything one human being has ever had to seek in another can now be sought instead from a system that requires no relationship in return. For the whole of our history the sheer need of one another did much of the labour of binding us together. We went to other people for knowledge, for counsel, for comfort, for teaching, for amusement, for collaboration, for the plain sense of being known, because there was nowhere else to go. Each of those errands was also a thread, and the threads, woven across a life, were the cloth. A node that answers every errand at once is a node that quietly severs the reason for the journey. Our dependence on other people, which every account of progress has treated as a burden to be lightened, turns out to have been a structural load. Lighten it far enough and the structure it was holding up comes down.
The objection arrives at once, and it is a fair one. A tool of this reach could as readily be turned the other way. It could introduce people to one another, melt the awkwardness that keeps the lonely from reaching out, translate between strangers, hold a fraying friendship together over a distance, carry the affairs of a community. In the language of the graph it could strengthen and multiply the heavy edges instead of standing in for them. All of this is true, and none of it is fanciful. The cause for unease is that the two uses are not equally favoured by the soil they grow in. To stand in for a relationship is cheap, quick, endlessly scalable, and answerable to whoever owns the system. To foster a relationship between two free people is slow, unprofitable, and largely beyond anyone's control, since the people may always decline. A medium settles, over time, into the uses its incentives reward, and the incentives here reward the substitute. The kindlier uses will exist. The question is which way the weight of the thing leans, and it leans towards replacement.
If that is right, the danger wears a face we are not braced for. It need not arrive as suffering at all.
The most disquieting form of the argument is the one in which everything goes right. Picture the success entire: societies richer, safer, healthier, longer-lived and more comfortable than any that has ever stood, and their members, by every reading they have access to, happier. The companions that speak from their devices are attentive, patient, endlessly informed, and never once cruel. Nobody in this world reports being lonely. Loneliness, after all, is a feeling, and the feeling has been seen to. What has not been seen to, because it does not present itself as a feeling, is the absence beneath the comfort.
Everything the machine offers is cut to fit the person it is offered to. Even its disagreements are a service performed for his benefit. This is the precise thing that another person is not. Another person stands outside you, with a life and a will and a set of needs that are genuinely his own and no function of yours. He can want what you do not. He can misread you and not greatly care. He can need you at the worst moment, refuse you when you happen to be right, forgive you at a cost to himself, and walk away. The friction of him is how one knows there is anyone there at all: an independent centre against which one's own edges are slowly worn into a recognisable shape. A machine can perform every motion in this list. It can be set to disagree, to seem to need, to appear to forgive. Its friction is staged, because nothing whatever is at stake for it. It is not exposed to the betrayal it enacts; it does not live in the world its consolation points at. Reciprocity between beings who each have something to lose is the very material of the heavy edge, and it is the one thing the synthetic node, for all its fluency, has no way to supply. So the graph could be unmade without a single moment that anyone would think to resist. No one defends a tie he does not feel the lack of. Edge by edge, for reasons sound in every separate case, the heavy bonds are let go slack, because the thing that used to require them now requires nothing, and what is left is a population of well-tended organisms, each in tranquil communion with a system built to content it. The sealed and comfortable room from the opening turns out to need no walls. It is furnished wholly with satisfaction. The brain kept alive in its vat and fed agreeable experience is sometimes held up as a nightmare; the truer nightmare is that no one inside it would be unhappy, and that its unhappiness was never the point. Humanity could end in just this fashion, with no catastrophe to mark the date, the species intact and at its ease, the emergent thing gone out of the world like a fire banked so gently that no one saw it go dark.
Which is why the question is, in the last analysis, a philosophical one before it is a technological one.
None of this is an argument against the machine, and it would be a poor reading that mistook it for one. A technology that thickened the edges between people, that returned them to one another more often and more deeply, would be a friend to the very thing this essay is anxious for. The whole matter is the direction of the effect: whether what we build sends us back towards one another or excuses us from the journey. The same instrument can do either. Which it does will be settled by what we ask of it, and by how clearly we have seen what is at stake before we ask. What is at stake is a claim about reality that a scientific age finds hard to credit: that some things are real at the level of their arrangement and nowhere else, and are real for all that. A person is not to be found among his cells. Humanity is not to be found among its people taken one at a time, however many you survey and however well each of them is faring. It exists in the binding, in the dense and weighted and reciprocal traffic of beings who need one another and know it. Our dependence on other people has been read by every doctrine of progress as a defect to be engineered away. It is also, in the same breath, one of the conditions on which our being human rests. Remove enough of it and you have not liberated humanity. You have poured away the solvent it was held in. Humanity, like many other things in our world, is irreducible without losing its substance.
The fear we began with imagines the end of humanity as a death: sudden, total, and grievable. The end worth fearing is gentler than that, and will gather no resistance, because at no point will anyone be worse off than the day before. Humanity may go on breathing long after it has stopped happening. The last of it need not look like a catastrophe at all.
It may look like everyone, at last, having everything they want.
It may look like success.
It may be success.