The Numbers Game

Demography experts say the changes made in the National Jewish Population Survey 2000 methodology may make it impossible to compare 1990s reality to today’s.
Several potentially important alterations were made to the interview questions, they said, and there are “methodological consequences related to the way people are interviewed,” said Egon Mayer, chairman of the sociology department at Brooklyn College and a member of the NJPS 2000 technical advisory committee.
The 1990 study questions were part of an omnibus survey in which respondents were asked about anything from beer preference to voting patterns.
This time, the NJPS was a focused study conducted over one year and devoted solely to questions about Jewish identity and behavior.
Some, including Barry Kosmin, who directed the 1990 NJPS and now works as director of the Institute for Jewish Policy Research in London, say that embedding Jewish identity questions into an omnibus survey is more likely to elicit truthful answers than in a “focused” study.
“People know that the Jewish federations are behind the survey and don’t want to be found because they don’t want to get hit up” for contributions, Kosmin said. “People will say ‘I’m not Jewish and neither is my father or grandfather, alav hashalom,” he said, using the Hebrew phrase “peace be upon him,” said when someone has died.
Trimming the “phenomenal cost of a boutique study,” like the $6 million NJPS 2000 (the 1990 study, by comparison, cost about $400,000), led researchers to aim more of the random-digit-dialed phone calls toward heavily populated Jewish areas, with the results then weighted to account for less-populated areas.
But that, acknowledges NJPS 2000 director Jim Schwartz, relies on some “reasonable assumptions” about how many Jews live in a particular area.
The result, said Kosmin, is that people who are not in those areas go uncounted.
The sample now, with roughly 4,700 Jewish respondents, is nearly twice as large as it was in 1990.
But UJC researchers had trouble finding people willing to respond to the entire interview, which on average took 40 minutes, and ended up offering $25 incentive payments to those who did.
“Once you start paying people, there has to be a psychological effect leading people to tell you the answers you want to hear,” Kosmin said.
Moreover, the new survey interviewed only the Jewish members of households with Jews in them, while the 1990 study received information about the Jewish behavior in the home from non-Jewish members, like gentiles married to Jews.
Asking to speak with the Jew in the household when a gentile partner was already on the phone “could have some effect on the kinds of interviews that got completed and those that did not,” Mayer said.
It could be perceived as a slight bias against the non-Jew and in the long run could eliminate a percentage of the intermarried population” because the Jew proved too hard to reach, Mayer said.
In addition, this time there was also more of a “conscious concern” about taking people who really aren’t Jewish, but at first say they are, out of the sample, said Jeff Scheckner, a researcher who was involved in 1990 and, until last summer, on the NJPS 2000.
For example, some people say they are Jewish but when interviewed further explain their response by adding “ ‘because Jesus was a Jew’ or ‘I’m Mormon and we have a kinship with the Jews,’ ” said Scheckner.
“Then there were other cases of people who said ‘my mother was Jewish but we were raised by someone else.’ It was really a tough call sometimes to determine who was and who wasn’t,” Jewish, Scheckner said, but critical in order to determine both the number of Jews and the intermarriage rate.
Each of these changes potentially alters the findings, the experts say, making it difficult to compare findings between the 1990 and 2000 surveys.
Then again, any population study is going to have its flaws, Kosmin said.
“The fact is that the NJPS provides information from a fraction of a community at one point in time — a photograph of 5,000 people. But the fact is that there is a movie of millions playing on the screen,” he said.
Trying to capture all their stories in a population study, Kosmin said, “is a very difficult task to do, and is open to interpretation.”