One appeal of the big city has long been the ability it affords us to disappear into the crowd, but a fact of life in the 21st-century metropolis is that privacy is nowhere to be found. Between establishments that don’t take cash, apartments that use facial recognition for entry, cameras on kiosks on every corner, and, starting this spring, new tap cards to access the subway, anonymity is over. Serendipity — the happy accidents and strange exchanges of a metropolis always beyond our grasp — may be on its way out, too. While our phones invite companies to follow us around with relentless precision, we walk the streets heeding the suggestions of maps and apps, following a collective bliss dependent on the information shared by — well, whom exactly? A seamless and predictable urban experience is the promise of the smart city, but what does all this artificial intelligence add up to? Artist Brian House sent up the algorithmic view of the city in a seven-day performance “following” the trajectories of a “plausible yet non-existent New Yorker,” composed from the data of 1,000 anonymous residents. Here, he recounts his experiment, which reveals, and argues for, the persistence of unpredictability.
Half a century ago, the Architectural League invited Vito Acconci to participate in an “exhibition” on the streets of NYC. What he came up with was Following Piece (1969):
Each day I pick out, at random, a person walking in the street.
I follow a different person everyday; I keep following until that person enters a private place (home, office, etc.) where I can’t get in.
Over the course of the month, Acconci followed 23 individuals for durations ranging from a few minutes to eight hours. Beyond a series of staged photographs taken after the fact, his documentation consists only of affectless descriptors, a list of incidents such as “11:10AM … Man in brown jacket; he walks south on Bleecker.”
Today, Following Piece can be readily understood as a kind of algorithm, one that is also performed by the countless apps, cameras, and sensors that keep tabs on us as we move around the city every day. In fact, Acconci described himself as an automaton, writing in his notes that once he determined the formula for the piece, “I don’t have to control myself … I am almost not an ‘I’ anymore; I put myself in the service of this scheme.”
Such surrender to algorithmic logic disregards the unspoken dynamics that otherwise sustain urban spaces. For one, singling out a person to follow is a creepy thing to do; rather than respecting a personal right, what is private becomes only where a follower “can’t get in.” Just as disturbing is what becomes of the public if people are just paths. There’s no shared sense of urban life in a composite of trajectories, nothing that speaks to the experience, for instance, that makes one a New Yorker.
In these respects, Following Piece in many ways anticipates the recent advances in artificial intelligence (AI) that promise to revolutionize urban life via algorithms. From self-driving cars to dynamic planning tools to predictive policing (see also), “smart cities” are all the rage for the city of the future. Yet AI works by making predictions about the world based on the past — that is, based on the data that are used to train it. So it is also, in essence, a “follower.” Despite the energy spent by urbanists, artists, and technologists alike fetishizing AI’s supposed creativity, AI cannot imagine that which exceeds our own capacity to measure. And the collective imagination that makes a city is one of those things.
Further, AI even makes it difficult to tell when data from multiple people are being used, and not just on the streets. Take WaveNet, a speech synthesizer that currently provides the voice of Google Assistant and the new Duplex service. When we hear WaveNet, we hear it — and respond to it — as if it were an embodied individual. The “ums,” inhales, and lip smacks that it makes all suggest that it has personal quirks just as we do, and that we can empathize with it accordingly. But, in actuality, WaveNet is a chorus: it is only convincing because it has been trained on the recorded voices of a whole lot of talking humans. That their identities are obfuscated by the process demonstrates how AI is prone to speak over its composite origin.[1]
So an urban AI might track, collate, synthesize, simulate, and optimize the lives of people in a city. But it forever remains a follower, and so the result does not exceed the sum of its parts. This is the profound difference between how an algorithm gathers up humans by the numbers and how people, in their everyday lives, together shape their public spaces.
As an artist, I wanted to gain a material understanding of the tension between how AI combines individuals and what it is to encounter each other in the city. How could I program something myself that demonstrated automated logics taken to their extreme? Luckily, I had access to a unique dataset drawn from urban life.
During my time at the New York Times Research and Development Lab, I’d worked on a project called OpenPaths.[2] My colleagues and I had been bothered by the fact that so many corporations keep tabs on the physical location of our phones as we move around the city, while we don’t have access to those data ourselves. So we built a free tool that anyone could use to securely self-track their location. The platform also allowed participants to anonymously contribute their data to experimental projects by journalists, activists, academics, and artists, with the idea that there might be more compelling uses for these data than the predatory advertising for which they are typically employed.
I made my own request of OpenPaths participants, and I trained an AI with their data — millions of geolocation points that marked the daily paths of 1,000 residents of New York City. The training process took some doing. Getting an LSTM recursive neural network to work with the data required a lot of trial and error, and I had to rent a virtual computer from Amazon to handle the computational intensity (let no one suggest that an AI just learns by itself). But eventually, it worked.
This meant that I could now give the AI a time and a location, and it would “predict” the next location. This prediction would be based on the data, some amalgamated statistical logic that says that a New Yorker — not any New Yorker in particular, nor even an average of those New Yorkers who happened to be represented in the dataset, but some plausible yet non-existent New Yorker — would move around the city in this way. Accumulate enough predictions by feeding them right back into the model, and the wisdom of the crowd becomes the daily rhythms of this hypothetical individual.
But watching a simulation on my laptop screen made it too easy to imagine that this path actually was indicative of a life, a singular life, and not a distilled trace of real collective histories that were still unfolding. Taking a cue from Acconci, I decided to follow the AI — by foot, on the streets. But if his piece reduced a person to a path, I hoped to make a path back into a crowd, as an act of following might fill in something of what the AI had left out. And although I’d have no way of knowing if the people I encountered had contributed to the dataset, perhaps I’d find something of them that remained.
I set out on a Wednesday afternoon, and I kept going for one full week. I had the AI on my phone so I could see where to go and how long to stay there. If I wasn’t on the move, I allowed myself the option to take a break to eat, relieve myself, or sleep, as long as I returned to the same spot to pick up the path again.
I followed the AI to places familiar to me from my life in the city and to places with which I was entirely unacquainted. Most days I commuted to “work,” whether uptown, downtown, or midtown; I surveyed various bars and restaurants in Brooklyn and Long Island City; I went to a doctor’s office, and even to the cemetery. Days ended at apartment complexes and row houses across the city. Each day was a different experience because I’d programmed the AI to forget what it had done the day before.
Upon arriving somewhere, I attempted to engage intuitively with the place. If it was a restaurant, I ate, or if it was a store, I shopped. There were limits, however. I frequently encountered thresholds that I did not have the means to cross — maybe I didn’t have a key, or maybe I just couldn’t explain my purpose for being there. Of course, these contours of access were defined in relation to my being white, male, and able-bodied, however universal the directions given by my app might seem. That I spent a lot of time in downtown Manhattan and north Brooklyn was the result of an original dataset that was hardly representative of the city’s population.
The week was exhausting, boring, and sometimes revelatory in the small discoveries that make this city perpetually unpredictable. A relatable frustration overheard, a cascade of shadows across a facade, a spontaneous but friendly choreography between strangers. As estranged by the algorithm as we were, I felt a sense of intimacy distributed among all the people behind the data who must have had similar experiences in these same places.[3] Taking photos was an attempt to relocate the data in our shared serendipity.
AI is already us, but it is still something less than a “we.” We have an invaluable freedom to be unpredicted and unpredictable, to be a public, not a composite, and neither to be followed nor followers. We should not automate it away. To title my strange performance, I lifted a lyric from David Byrne — Everything That Happens Will Happen Today. From one perspective, the phrase echoes the totalizing impulse behind data collection, and how AI prescribes a future that follows the past. And yet, read with a wink, it’s everything else and that which has yet to come.
I take the shuttle to Franklin Ave. and then the C to Manhattan.
10:20AM @ 160 Front St, Manhattan
Office. I sit on the doorstep of the building, which is closed for construction. I watch the construction that’s also happening across the street.
Two and a half hours later, I walk downtown on Water.
1:06PM @ 66 Water St, Manhattan
I eat french fries and a hot dog from the food truck.
“No way it’s worth that much.” / “Same thing happened last summer.” / “Well, follow up on it and tell me.”
Guys waiting for a smoothie. The one talking punctuates his points with physical contact.
A barber shop is in the shadow of the building. Some final snips, the handheld mirror held up for approval … looks good.
49 minutes later, I walk uptown on Pearl.
2:04PM @ 29 Gold St, Manhattan
Courier depot. When someone enters the building, I get a glimpse of a stocked fridge open inside.
51 minutes later, I walk uptown on William.
3:07PM @ 80 Broad St, Manhattan
Offices. I walk in through the revolving doors. The woman at the desk is very nice, she’s trying to find a listing for me. I check out the digital art — she tries to explain the best way to make a video of it. Afterwards, I stand in front of the building with the smokers.
“In order for them to use the software, you have to show them how it’s good for their specific vertical.”
