Another problem historians often encounter has to do with sampling. Statisticians have developed elaborate ways to measure the reliability of samples, and to establish the likelihood that a particular sample appropriately embodies the qualities of the larger population from which it was selected. For a host of reasons, the kind of sample statisticians prefer to work with is a random sample, and this kind of sample can be hard for historians to draw. We often have only the partial remains of a larger body of data to work with—say one month’s payroll of a business, two plantations’ records of slave births over a given decade, or one neighborhood’s building survey before a major fire. Statisticians call this kind of sample a "sample of convenience," but samples of convenience are frequently samples of necessity for historians. Again, we cannot go back and take a new sample of the entire body of data if the surviving records are incomplete.

Yet if the circumstances are favorable, you may want to draw a random sample from a larger population. When dealing with data sets of several hundred records, the advantage of sampling is great: you can save time, energy, and perhaps even your sanity without sacrificing statistical validity. Be sure to note, however, that a random sample is not the same thing as a "systematic sample." Going through a list and selecting every "nth" record is not a random process. To take a random sample, you should use a random number table (commonly found in the back of statistics textbooks) or a random number generator (included in various computer programs) to make sure that the selection process is truly unbiased. Once you have a random sample, you can apply a host of popular statistical tests that are not appropriate for other kinds of samples.

What’s the big deal about sampling? You can’t pick up a newspaper or magazine without reading the results of an opinion poll these days, and they’re all based on random sampling, right? In 1936, that wasn’t so clear.

The magazine Literary Digest had correctly predicted the outcomes of the 1916, 1920, 1924, 1928, and 1932 elections by conducting polls. In 1936, the magazine mailed 10,000,000 ballots and more than 2,000,000 were returned. On the basis of this method, the magazine believed it had asked one fourth of the nation’s voters which candidate they intended to vote for. Literary Digest’s prediction? That Republican presidential candidate Alfred Landon would win 57 percent of the popular vote and 370 electoral votes. But George Gallup, founder of the new American Institute of Public Opinion, predicted the opposite—that Roosevelt would win 56 percent of the popular vote and at least 315 electoral votes.

Gallup, of course, was right. How had the Literary Digest blundered so disastrously? The problem lay in both their response rate and their original sample. Those angry at Roosevelt were much more motivated to send back the Digest ballots. In addition, the ten million person "sample" was drawn from owners of automobiles and people listed in phone books—groups who were more affluent and thus less likely to vote for Roosevelt.

Gallup employed a more sophisticated method of random sampling than Literary Digest, using areas drawn from existing election precincts and selecting a systematic sample of households. Within precincts, Gallup randomly chose a starting point then selected each nth household from which to interview one adult. He made sure to interview an equal number of men and women, to correct for variability in household size, and to ask respondents about how likely they were to vote in the upcoming election, so that he could project and account for voter turnout in his final polling results.