intrust

Censi (or censuses if you will) are a big deal. They help to determine the tax base and the make-up of political constituencies and they underlay the decisions of government. At the end of the 19^th century the US Census Bureau, despite being the single largest employer in the land at the time¹, was having a rough time keeping up with the rapidly expanding population, swelled by millions of immigrants. By federal law the census was to be taken every ten years, and since the 1880 census had taken eight years to complete, it was feared that the 1890 census would not be finished before the 1900 count was due to begin.

Herman Hollerith put together one of three entries in a Census Bureau contest, staged in order to find a way to speed up the process of tabulating census records. He had knowledge of the Jacquard loom invented almost a hundred years earlier, and during a stint of working for the railroads he had observed the use of what was called a "punch photograph": Conductors, in order to discourage free riders, would punch notches in the edges of a passenger’s ticket denoting their height, colour of eyes and hair, etc. Hollerith amalgamated these technologies into a set of machines that were able to tabulate the collected information on 62 million individuals in a matter of months. The punch cards designed by Hollerith, which were purposely the size of the US dollar, were still in use up into the 1970s and the company which he founded eventually became IBM

Herman Hollerith rationalized the US census with the help of tabulating machines and a warehouse full of punch cards. But there wasn’t a lot of room on the punch cards for the information desired – room for W3 has traditionally come at a premium. Hollerith’s machines calculated statistics that were gathered by an army of census workers called enumerators, part time workers who roamed the country gathering facts about the populace. The enumerator carried scorecards, known as a schedules, on which to notate relevant figures.

Any enumerator is going to see and experience all sorts of things in her work, which she might reflect upon, but which will have no appropriate notch on her scorecard, and for that matter, be of no interest to her employer. In the end, census figures are about the generalities of a populace – not an individual’s personal details. But we may note that any attempt to record even a fraction of the W2 observations of one single enumerator, if so desired, would easily outweigh, in terms of data storage, the relevant statistics of the entire populace. So the filtering of excess W3 begins at the source of its collection.

// posted by Greg FitzPatrick @ 14:37