New survey data about the Shoah is always welcome. The Jewish Week article about a new survey concerning knowledge of the Shoah provides information about many interesting topics (“Shoah Knowledge Lacking in U.S., New Survey Finds,” April 13). However, there is a major omission in the article, which appears to reflect a serious limitation in the research itself.
An essential issue for every survey research project is the nature and quality of the methodology. Research ethics require that the presentation of research results be transparent and provide the reader with a clear statement describing the research methodology. Without such information, the reader has no basis for evaluating the data quality.
The Claims Conference website (claimscon.org/study) contains some information about the study not available in The Jewish Week article, including a link to a PDF about the study prepared by the research firm. No additional information about the study appears on the research firm’s website (schoenconsulting.com).
The Jewish Week reported that the data was collected through interviews with 1,350 people. But it did not say whether these were personal, telephone, online or some other data collection method.
Readers who want to know if survey results are reliable need to know if the sample selected is representative of a particular population. The Schoen Consulting PDF does not provide any additional information about the sample selection method.
The Schoen Consulting PDF indicates that this study was with a sample of adults age 18 and older. It does not indicate if the interviews were conducted in all 48 contiguous states or some other geography. [The data was collected with a random and demographically representative sample of the adult population in the United States.]
The Jewish Week reported that the interviews were conducted in February 2018. In fact, they were conducted from Feb. 23-27, not during the whole month. The article also did not list the survey’s margin of error, which was plus/minus 3 percent.
The only way to reliably discuss changes in attitudes or behavior over time is by comparing results from two or more surveys that use representative samples. Yet the article compares some of the new study results with a 2005 AJC poll, without confirming that either survey used a representative sample.
The New York Jewish Week brings you the stories behind the headlines, keeping you connected to Jewish life in New York. Help sustain the reporting you trust by donating today.
