“I’m a student at the University of Chicago,” Venkatesh sputtered, sticking to his survey script, “and I am administering—”

“Fuck you, nigger, what are you doing in our stairwell?”

There was an ongoing gang war in Chicago. Things had been violent lately, with shootings nearly every day. This gang, a branch of the Black Gangster Disciple Nation, was plainly on edge. They didn’t know what to make of Venkatesh. He didn’t seem to be a member of a rival gang. But maybe he was some kind of spy? He certainly wasn’t a cop. He wasn’t black, wasn’t white. He wasn’t exactly threatening—he was armed only with his clipboard—but he didn’t seem quite harmless either. Thanks to his three months trailing the Grateful Dead, he still looked, as he would later put it, “like a genuine freak, with hair down to my ass.”

The gang members started arguing over what should be done with Venkatesh. Let him go? But if he did tell the rival gang about this stairwell hangout, they’d be susceptible to a surprise attack. One jittery kid kept wagging something back and forth in his hands—in the dimming light, Venkatesh eventually realized it was a gun—and muttering, “Let me have him, let me have him.” Venkatesh was very, very scared.

The crowd grew, bigger and louder. Then an older gang member appeared. He snatched the clipboard from Venkatesh’s hands and, when he saw that it was a written questionnaire, looked puzzled.

“I can’t read any of this shit,” he said.

“That’s because you can’t read,” said one of the teenagers, and everyone laughed at the older gangster.

He told Venkatesh to go ahead and ask him a question from the survey. Venkatesh led with the how-does-it-feel-to-be-black-and-poor question. It was met with a round of guffaws, some angrier than others. As Venkatesh would later tell his university colleagues, he realized that the multiple-choice answers A through E were insufficient. In reality, he now knew, the answers should have looked like this:

1. Very bad

2. Bad

3. Neither bad nor good

4. Somewhat good

5. Very good

6. Fuck you

Just as things were looking their bleakest for Venkatesh, another man appeared. This was J.T., the gang’s leader. J.T. wanted to know what was going on. Then he told Venkatesh to read him the survey question. He listened but then said he couldn’t answer the question because he wasn’t black.

“Well then,” Venkatesh said, “how does it feel to be African American and poor?”

“I ain’t no African American either, you idiot. I’m a nigger.” J.T. then administered a lively though not unfriendly taxonomical lesson in “nigger” versus “African American” versus “black.” When he was through, there was an awkward silence. Still nobody seemed to know what to do with Venkatesh. J.T., who was in his late twenties, had cooled down his subordinates, but he didn’t seem to want to interfere directly with their catch. Darkness fell and J.T. left. “People don’t come out of here alive,” the jittery teenager with the gun told Venkatesh. “You know that, don’t you?”

As night deepened, his captors eased up. They gave Venkatesh one of their beers, and then another and another. When he had to pee, he went where they went—on the stairwell landing one floor up. J.T. stopped by a few times during the night but didn’t have much to say. Daybreak came and then noon. Venkatesh would occasionally try to discuss his survey, but the young crack dealers just laughed and told him how stupid his questions were. Finally, nearly twenty-four hours after Venkatesh stumbled upon them, they set him free.

He went home and took a shower. He was relieved but he was also curious. It struck Venkatesh that most people, including himself, had never given much thought to the daily life of ghetto criminals. He was now eager to learn how the Black Disciples worked, from top to bottom.

After a few hours, he decided to walk back to the housing project. By now he had thought of some better questions to ask.

Having seen firsthand that the conventional method of data gathering was in this case absurd, Venkatesh vowed to scrap his questionnaire and embed himself with the gang. He tracked down J.T. and sketched out his proposal. J.T. thought Venkatesh was crazy, literally—a university student wanting to cozy up to a crack gang? But he also admired what Venkatesh was after. As it happened, J.T. was a college graduate himself, a business major. After college, he had taken a job in the Loop, working in the marketing department of a company that sold office equipment. But he felt so out of place there—like a white man working at Afro Sheen headquarters, he liked to say—that he quit. Still, he never forgot what he learned. He knew the importance of collecting data and finding new markets; he was always on the lookout for better management strategies. It was no coincidence, in other words, that J.T. was the leader of this crack gang. He was bred to be a boss.

After some wrangling, J.T. promised Venkatesh unfettered access to the gang’s operations as long as J.T. retained veto power over any information that, if published, might prove harmful.

When the yellow-gray buildings on the lakefront were demolished, shortly after Venkatesh’s first visit, the gang relocated to another housing project even deeper in Chicago’s south side. For the next six years, Venkatesh practically lived there. Under J.T.’s protection he watched the gang members up close, at work and at home. He asked endless questions. Sometimes the gangsters were annoyed by his curiosity; more often they took advantage of his willingness to listen. “It’s a war out here, man,” one dealer told him. “I mean, every day people struggling to survive, so you know, we just do what we can. We ain’t got no choice, and if that means getting killed, well shit, it’s what niggers do around here to feed their family.”

Venkatesh would move from one family to the next, washing their dinner dishes and sleeping on the floor. He bought toys for their children; he once watched a woman use her baby’s bib to sop up the blood of a teenaged drug dealer who was shot to death in front of Venkatesh. William Julius Wilson, back at the U. of C., was having regular nightmares on Venkatesh’s behalf.

Over the years the gang endured bloody turf wars and, eventually, a federal indictment. A member named Booty, who was one rank beneath J.T., came to Venkatesh with a story. Booty was being blamed by the rest of the gang for bringing about the indictment, he told Venkatesh, and therefore suspected that he would soon be killed. (He was right.) But first Booty wanted to do a little atoning. For all the gang’s talk about how crack dealing didn’t do any harm—they even liked to brag that it kept black money in the black community—Booty was feeling guilty. He wanted to leave behind something that might somehow benefit the next generation. He handed Venkatesh a stack of well-worn spiral notebooks—blue and black, the gang’s colors. They represented a complete record of four years’ worth of the gang’s financial transactions. At J.T.’s direction, the ledgers had been rigorously compiled: sales, wages, dues, even the death benefits paid out to the families of murdered members.

At first Venkatesh didn’t even want the notebooks. What if the Feds found out he had them—perhaps he’d be indicted too? Besides, what was he supposed to do with the data? Despite his math background, he had long ago stopped thinking in numbers.

Upon completing his graduate work at the University of Chicago, Venkatesh was awarded a three-year stay at Harvard’s Society of Fellows. Its environment of sharp thinking and bonhomie—the walnut paneling, the sherry cart once owned by Oliver Wendell Holmes—delighted Venkatesh. He went so far as to become the society’s wine steward. And yet he regularly left Cambridge, returning again and again to the crack gang in Chicago. This street-level research made Venkatesh something of an anomaly. Most of the other young Fellows were dyed-in-the-tweed intellectuals who liked to pun in Greek.