6.16 Public health surveillance
Oxford Textbook of Public Health
Public health surveillance
Ruth L. Berkelman, Donna F. Stroup, and James W. Buehler
Purposes of surveillance
Establishing a surveillance system
Public health surveillance in developing countries: special considerations
Surveillance system objectives
Constituents of surveillance systems
Nature of public health programmes
Health problems under surveillance
The flow of information
Data management and dissemination
Analysis of surveillance data
Dissemination of data
Evaluation of surveillance systems
Public health surveillance is the epidemiological foundation for modern public health. Surveillance data resulting from the continuous monitoring of the occurrence of a disease or condition, such as AIDS or lead poisoning, underlie what public health actions are taken and reflect whether these actions are effective. The term ‘surveillance’ is derived from the French word meaning ‘to watch over’ and, as applied to public health, means the close monitoring of the occurrence of selected health conditions in the population. Although surveillance methods were originally developed as part of efforts to control infectious diseases, basic concepts of surveillance have been applied to all areas of public health. Public health surveillance has been expanded to include not only information on diseases, injuries, and other conditions but also information such as the prevalence of risk factors, both personal and environmental.
In 1963 Langmuir defined disease surveillance as ‘the continued watchfulness over the distribution and trends of incidence through the systematic collection, consolidation, and evaluation of morbidity and mortality reports and other relevant data’ together with timely and regular dissemination to those who ‘need to know’ (Langmuir 1963). In 1968 the 21st World Health Assembly described surveillance as the systematic collection and use of epidemiological information for the planning, implementation, and assessment of disease control; in short, surveillance implied ‘information for action’ (WHO 1968). The role and concept of public health surveillance continue to evolve as the scope of surveillance broadens and as increasingly sophisticated methods are applied (Thacker et al. 1989; Thacker and Stroup 1994).
Surveillance should begin when there exists or is likely to occur a public health problem for which programmes for prevention and control of a health event have been or may need to be initiated. A critical component of the definition of surveillance is that surveillance systems include the ongoing collection, analysis, and use of health data. Thus, health information systems (for example, registration of births and deaths, routine abstraction of hospital records, health surveys in a population) that are general and not linked to specific prevention and control programmes do not, by themselves, constitute surveillance. However, data collected from ongoing health information systems may be useful for surveillance when systematically analysed and applied on a timely basis.
The idea of collecting data, analysing them, and considering a reasonable response stems from Hippocrates (Eylenbosch and Noah 1988). When writing on disease occurrence, Hippocrates made a distinction between the steady state, the endemic state, and the abrupt change in incidence—the epidemic. Possibly, the first public health action that can be attributed to surveillance occurred in the 1300s when public health authorities in a port near the Republic of Venice prevented passengers from coming ashore during the time of epidemic bubonic plague in Europe. The first Bill of Mortality was issued in London in 1532 as a consequence of fear of a plague epidemic. John Graunt’s treatise Natural and Political Observations on the Bills of Mortality published in 1662 is generally recognized as one of the first documents to describe use of numerical methods for monitoring public health. In 1776 Johann Peter Frank advocated a more extensive monitoring of health in Germany that would support public health efforts related to the health of schoolchildren, prevention of injuries, maternal and child health, and public water and sewage disposal.
William Farr is recognized as the founder of the modern concept of surveillance. As Superintendent of the Statistical Department of the General Registrar’s Office in Great Britain from 1839 to 1879, he collected, analysed, and interpreted vital statistics and disseminated the information in weekly, quarterly, and annual reports. He did not stop with publication of official reports, but regularly contributed papers to medical journals and even used the public press to achieve effective action (Langmuir 1976). Thus, he took the responsibility of seeing that action was taken on the basis of his analyses.
In the nineteenth century, Farr’s efforts at health monitoring were extended by Edwin Chadwick, who investigated the relationship between environmental conditions and disease. Chadwick was followed by Louis Rene Villerme, who analysed the relation between poverty and mortality in Paris. In the United States, Lemuel Shattuck also published data that related deaths, infant and maternal mortality, and infectious diseases to living conditions. He further recommended standardized nomenclature for cause of disease and death, and the collection of health data that included sex, age, locality, and other demographic factors. The first international list of causes of death was developed in 1893 (Eylenbosch and Noah 1988).
Increasingly, elements of surveillance were applied to aid in detecting epidemics and in preventing and controlling infectious diseases. In 1899 the United Kingdom began compulsory notification of selected infectious diseases. National morbidity data collection on plague, smallpox, and yellow fever was initiated in 1878 in the United States, and by 1925 all states were reporting weekly to the United States Public Health Service on the occurrence of selected diseases. In a public health context, the term surveillance was increasingly applied to programmes of reporting selected infectious diseases in a population, with less emphasis on its application to quarantine of individuals (Langmuir 1963; Thacker and Gregg 1996).
Similar reporting activities were occurring in Europe at about the same time. In 1907 the Office International d’Hygiene Publique, predominantly composed of European member states, was created (WHO 1958). The office was to disseminate information in a monthly bulletin on the occurrence of selected diseases, most notably cholera, plague, and yellow fever. In the succeeding decades, other diseases were recommended for surveillance in step with the International Sanitary Regulations. However, many of the morbidity and mortality reporting systems were not systematic and were still largely developed for long-term archival functions.
Since the early 1950s, the critical importance of surveillance to public health efforts has been demonstrated frequently. In 1955 acute poliomyelitis among recipients of the poliomyelitis vaccine in the United States threatened national vaccination programmes that had just begun. In collaboration with state health departments, the American Centers for Disease Control (CDC) developed an intensive national surveillance system, and at one point a daily report was being issued regarding poliomyelitis cases. The surveillance data assisted epidemiologists in demonstrating that the problem was limited to a single manufacturer of the vaccine and allowed the vaccination programme to continue with a resulting dramatic decline in cases of acute poliomyelitis in the United States in successive years (Langmuir 1963). During the worldwide malaria control programme, surveillance was used to determine areas of continued transmission and to focus spraying efforts, as well as to document those areas without malaria (Raska 1966). With the subsequent decline in malaria control efforts, surveillance data have documented the re-emergence of malaria in many areas of the world. In so doing, these data have contributed to renewed interest in malaria control at the end of the twentieth century.
Surveillance was also the foundation for the successful global campaign to eradicate smallpox. When the campaign began in 1967, efforts were focused on achieving a high vaccination level in countries with endemic smallpox; however, it was soon evident that a programme based on surveillance to target vaccinations in limited areas would be more efficient. Smallpox reporting sources, usually medical facilities, were contacted on a routine basis, and thus a reporting network was firmly established in most countries. In addition, other reporting sources were often established, including markets, schools, police, agricultural extension workers, and others. In 1973, as the goal of eradication neared, a systematic house-to-house search for cases was established in India and subsequently used widely in Pakistan and Bangladesh (Henderson 1976). Well-designed surveillance systems for data collection, tabulation, and routine feedback were vital to the success of the programme.
In 1981, shortly after the disease later named AIDS was recognized, national surveillance was begun in the United States and other countries. Even before the aetiological agent, HIV, was identified, surveillance data contributed to identifying modes of transmission, population groups at risk for infection, and, equally important, population groups not at risk for infection. These data have been instrumental in directing public health resources to programmes, preventing further spread of HIV, and averting widespread public hysteria (Jaffe et al. 1983).
The need for a strong infrastructure for surveillance systems is currently being re-emphasized not only as countries face the emergence and re-emergence of infectious diseases (Berkelman et al. 1994; Heymann and Rodier 1998) but also as a result of the increasing threat of biological terrorism (Henderson 1999). Plans for detecting terrorist events include strengthening current surveillance systems as well as establishing new ones, such as surveillance of emergency calls for medical assistance, and admissions of patients to intensive care units for respiratory conditions.
The potential usefulness of surveillance as a public health tool to address problems beyond infectious disease was emphasized in 1968 when the 21st World Health Assembly recommended the application of surveillance principles to a wider scope of problems, including cancer, atherosclerosis, and social problems such as drug addiction (WHO 1968). Many of the principles of surveillance traditionally applied to acute infectious diseases have also been applied to chronic diseases and conditions, although some differences in surveillance techniques have been observed (Thacker et al. 1995) (Table 1). Even though chronic diseases may have long latency periods, trends in their incidence may change relatively quickly, and surveillance can play a key role in detecting these changes when effective interventions are applied (Berkelman and Buehler 1990).
Table 1 Acute and chronic disease surveillance
In addition to the increased scope of health problems under surveillance, the methods of surveillance have expanded from general disease notification systems to include survey techniques, sentinel health-provider systems, and other approaches to data collection (Thacker and Berkelman 1988; Thacker and Stroup 1994). Beyond disease notification, an ideal surveillance system would provide analyses of risk factors for disease and injury.
The assimilation of computers into the workplace has made possible more efficient data collection as well as more rapid and sophisticated analyses (Dean et al. 1998). In the United States, all state health departments are linked to the CDC by computer for the routine collection and dissemination of selected data on notifiable health conditions (CDC 1991). In developing countries, computers are increasingly being used with epidemiological programmes for analysis and mapping (Frerichs 1991; Dean et al. 1993). Geographical information systems are also in widespread use.
The explosive development of technology will include the development of high-capacity storage devices, expansion of the capabilities of the internet, use of local- and wide-area networks for entry of surveillance data at multiple computers simultaneously, and development of new programming tools, video and computer integration, and voice and pen input. Integration of systems, including data standards, is needed to allow maximal use of these advancements (Morris et al. 1996).
Purposes of surveillance
A surveillance system should be designed to meet the needs of a prevention and control programme (Table 2). These needs usually include a description of the temporal and geographical trends in the occurrence of a health event in a particular population. Most importantly, surveillance systems should identify changes in disease occurrence. The data should be useful for substantiating patterns of both endemic and epidemic disease.
Table 2 Purposes of public health surveillance
The role of surveillance in guiding public health programmes is illustrated by the first major national disease control activity initiated by the CDC—the Malaria Eradication Programme. Surveys in the mid-1930s had established malaria to be an endemic problem deeply rooted in the south-eastern part of the United States. An extensive chlorophenothane (DDT) spraying programme was launched after the Second World War, and surveillance was instituted in 1947. Data from this surveillance system established rapidly that endemic malaria had essentially disappeared, probably even before the DDT programme was under way (Langmuir 1963). In this case, surveillance was used as the basis for dismantling a public health programme and redirecting public health resources to problems of higher priority.
However, the need for surveillance may continue for a disease even when prevention and control programmes are cut back, particularly for infectious diseases such as tuberculosis, dengue fever, or malaria whose incidence may change quickly. Generally, the more quickly re-emergence of a disease is detected, the more quickly and efficiently it can be controlled (CDC 1992a; PAHO 1994).
Surveillance data are also useful for evaluating the effectiveness of prevention and control programmes (CDC 1992b) and of regulations or laws modified or initiated to address public health concerns (for example safety of food and water, alcohol-related motor vehicle injuries). Monitoring of changes in the incidence of the disease or condition is necessary, and monitoring associated risk factors (food and water sanitation, self-reports of drinking and driving) may also be useful. The rapid decline in morbidity from many infectious diseases in certain populations has been related directly to vaccination campaigns that were conducted as a result of surveillance data on disease incidence. Direct correlation of a single intervention to a specific disease outcome may be difficult when the aetiology of the disease is multifactorial, but the impact of an intervention on disease outcome, including its incidence and severity, remains the ultimate test of policy.
Assessment of the burden of disease, including its incidence (that is, the number of people newly affected each year) and its current and projected prevalence (that is, the number of people affected by the disease at any point in time), is essential to planning public health programmes. For example, in the 1980s, surveillance for AIDS was critical to the forecasting of the future impact of that disease in the United States (Gail and Brookmeyer 1988; Institute of Medicine 1988). With the ageing of the population, projections of disease prevalence in the elderly are being emphasized (Murray and Lopez 1997).
Surveillance may be initiated to identify risk factors associated with disease and to suggest hypotheses for further investigation; cases identified through surveillance are sometimes used in case–control studies, as in the early studies of toxic shock syndrome and the AIDS epidemic (Shands et al. 1980; Jaffe et al. 1983). Effective preventive actions were formulated based on such research even before the aetiological agents, a toxigenic strain of Staphylococcus aureus and HIV, were discovered.
Establishing a surveillance system
Establishing a surveillance system requires a statement of objectives, definition of the disease or condition under surveillance, and implementation of procedures for collecting, interpreting, and disseminating information. Surveillance systems can be considered as information loops, or cycles, that involve health-care providers, public health agencies, and the public. A weakness in any part of the loop or information chain weakens the entire surveillance process. For example, if public health measures mandate that infectious conditions such as cryptosporidiosis must be reported, the surveillance system will be successful only if laboratories have the capacity to diagnose the infection (Berkelman 1994). Likewise, if the diagnosis on a hospital discharge record is coded incorrectly, surveillance data based on hospital discharge records will reflect the inaccuracies.
The information cycle begins when cases occur and is completed when information about these cases is made available and is used for prevention and control. This process may involve multiple cycles, ranging from the local response to individual cases to the development of national policies based on information aggregated from many cases. Essential to the completion of the surveillance cycle is the return of information to those who ‘need to know’ (Langmuir 1963), and thus attention must be directed not only to procedures for collecting data but also to procedures for ensuring that useful information is returned to constituents.
The likelihood that effective interventions can be found, can prevent the occurrence of disease, or can alleviate the course of existing disease is an important consideration in determining whether a surveillance system will be useful. However, even in the absence of a currently effective intervention, surveillance data can indicate the need for legislation (for example, the use of surveillance data on traumatic head injuries to influence legislation regarding mandatory use of bicycle helmets in children) or the need for more resource allocation if a problem poses an increasing health threat (for example, development of a pneumococcal vaccine for children aged under 2 years old becomes more urgent as the incidence of multidrug-resistant pneumococci increases). A surveillance programme is less likely to succeed if no clearly effective control or prevention measures are defined.
Health priorities for surveillance must be continually evaluated as new infections emerge (for example, Escherichia coli 0157:H7), the population is exposed to new hazards (for example, new consumer products, environmental contamination), and other health conditions change. Surveillance for both the disease or condition, as well as for associated risk factors and prevention services, should be considered.
Public health surveillance in developing countries: special considerations
Many aspects of public health surveillance are similar for developed and developing countries; these include the steps of system design and the principle of collecting information for action. However, existing data sources and resources for targeted data collection are generally far more limited in developing countries than in developed countries; scarce resources have led to relatively inexpensive and innovative surveillance techniques in many developing countries.
Tracking disease trends (particularly infectious diseases) has been the main reason surveillance systems have been instituted in developing as well as developed countries (Langmuir 1963). Many infectious diseases and other conditions of public health import—diarrhoea, malaria, pneumonia, and malnutrition—occur in settings with only rudimentary health care and few laboratory resources. Diseases are often empirically treated, and lack of definitive diagnosis may hinder surveillance and response efforts. Resources for surveillance that contribute to passive disease notification may result in inadequate data to meet surveillance or other health objectives (Sandiford et al. 1992).
Successful surveillance systems have been developed and maintained for targeted conditions; their success is partly dependent on features that include low budget, simplicity of reporting procedures, personal rapport with people in the network, regular feedback, and visible intervention consequent upon reporting specific conditions. For example, surveillance for childhood vaccine-preventable diseases has been conducted in Vellore, India, for more than a decade; every hospital (both government and private) is enrolled and participates in the system (John et al. 1998).
Programmes to eradicate polio (and previously smallpox) make extensive use of surveillance to monitor the progress toward reaching their goals (CDC 1999a). Eradication programmes must rely on targeted surveillance, which becomes more important (and expensive) as the target disease approaches eradication (Henderson 1998). The resources for targeted surveillance activities, including laboratory support, for an eradication or other specific programme may come from other countries, and may include both public and private donors.
In many developing countries, the process of linking surveillance to objectives highlights the need for mortality data and the absence of vital registration. The most basic health statistics are limited in many developing countries, with death registration inadequate or non-existent. Use of the verbal autopsy, which uses a caretaker interview to determine the cause of death, may assist in following mortality patterns in places without routine death registration (Kaufman et al. 1997; Fantahun 1998; Tollman et al. 1999). Sensitivity in establishing an accurate cause of death may be lower for some acute febrile conditions such as malaria than for conditions such as maternal causes, injuries, tuberculosis, and AIDS (Chandramohan et al. 1998), and different techniques in conducting verbal autopsies may result in quite different sensitivities for specific conditions (Quigley et al. 1996).
Other sources of health information may include United Nations International Children’s Emergency Fund (UNICEF), the World Health Organization (WHO), international conferences, non-governmental organizations, and population laboratories (for example, the International Center for Diarrheal Disease Research, Bangladesh). Although health problems are similar in many low-resource settings, relying on data from other countries can create major problems when there are geographical differences in the incidence of the condition. In addition, the health impact associated with certain conditions such as hepatitis B, rotavirus, or malaria may be significantly different in different regions and countries.
The design of surveillance systems must consider such issues as resources available, security, geography, population dispersion and mobility, type of health system, and literacy. Problems (more common but not unique to developing countries) may include limited personnel available for public health, multiple vertical systems, lack of laboratory capacity, and infrastructure and communications constraints (for example, lack of equipment, supplies, or electrical power).
Solutions to address the lack of personnel for public health and prevention have included voluntary systems (using community health workers, traditional birth attendants, or village volunteers). More familiar solutions are public health training programmes designed to meet human resource gaps (Adams and Hirschfeld 1998). Concerns about the cost-effectiveness of short-term training and the lack of applicability of long-term training to academic graduates have resulted in programmes targeting the specific needs of public health agencies. Examples include the Field Epidemiology Training Programs and the Public Health Schools Without Walls (Music and Schultz 1990; Cardenas et al. 1998), training programmes that have been initiated in both developed and developing countries.
Concurrently, the increase in availability of computers to analyse and transmit surveillance data and the decrease in the cost of such technology offers increased opportunity for surveillance. Epi Info is a public health computer program designed to assist data management and analysis that is available free and over the internet (Dean 1999); this tool has been used successfully in both developed and developing countries. This program is available in seven languages (English, French, Spanish, Arabic, Russian, Chinese, and Serbo-Croat), and manuals or portions of it have been translated into these languages and Italian, Portuguese, German, Norwegian, Hungarian, Czech, Polish, Romanian, Indonesian, and Farsi. Epi Info and other information systems should be seen as tools to be used to provide data to policy-makers and others to inform decisions which improve health.
Surveillance system objectives
Defining the objectives of a surveillance system depends on what information is needed, who needs it, and how it will be used. Implementing a system will require a balance of competing interests, and a clear statement of objectives will provide a framework for subsequent decisions. For example, the desire to collect detailed information about cases may compete with needs to assess the number of cases rapidly. Thus, if the primary objective is to obtain rapid case counts, less information would be collected about each case to avoid delays in reporting. The objectives of a surveillance system will be shaped by its target population and its constituents, the nature of prevention and control programmes, and the health problem under surveillance.
A surveillance system seeks to identify health events within a specified population. This population may be defined on the basis of where people live, work, attend school, or use health-care services. Alternatively, the population may be defined on the basis of where health events occur. For example, a surveillance system that monitors newborn health as a measure of prenatal care services would focus on deliveries to women who live within a community and not on women who live elsewhere but deliver in the community’s hospitals. In contrast, surveillance of traffic injuries aimed at identifying roadway hazards could include all injuries that occur in a community, regardless of whether affected people are community residents.
Constituents of surveillance systems
Surveillance systems are likely to have many constituents, including health-care providers, public health professionals, researchers at academic health centres, politicians, media reporters, the public, and others with diverse perspectives and uses for surveillance data. Because these diverse needs cannot always be satisfied, the primary or most important constituents should be identified.
Nature of public health programmes
The objectives of surveillance systems will be shaped by the objectives and capabilities of the public health programmes they serve. For example, a programme to eradicate an infectious disease requires intensive surveillance in the final stages of the campaign that emphasizes identification of all people with the disease (Hinman and Hopkins 1998). This strategy was used in the smallpox eradication programme and has also been employed in the poliomyelitis eradication programmes in the Americas (Foege et al. 1975; Biellik et al. 1992). In contrast, an educational programme to influence behaviour may depend on a surveillance system that describes the practices of a sample of people in a community (Remington et al. 1988).
Health problems under surveillance
It is necessary to decide exactly what disease or health problem will be under surveillance, using such criteria as the magnitude of the public health problem (or potential magnitude) as well as the capacity to prevent or control the disease or condition through public health actions. Surveillance may frequently be conducted for any of several points along a spectrum, ranging from exposure to an adverse outcome. It is important to consider which manifestation(s) or stage(s) of a disease should be under surveillance. For example, manifestations of ischaemic heart disease include abnormal diagnostic tests in the absence of symptoms, angina pectoris, acute myocardial infarction, and (sudden) death. If the goal of surveillance is to assess the burden of the disease on health-care systems, a broad definition that encompasses various manifestations may be appropriate. If the purpose is to monitor trends in the disease, a more limited and severe manifestation, such as myocardial infarction, may be the appropriate target for surveillance. If resources are limited, surveillance based on analyses of death certificates may be most feasible; however, interpretation of trends may be complicated by independent trends in the occurrence of the disease, advances in treatment, or changes in coding vital records. Alternatively, attention may be focused on risk factors for cardiovascular disease, such as hypertension, smoking, cholesterol levels, and physical activity (Arnett et al. 1998; McQueen 1999). For surveillance of infections carried by animals or arthropods (such as rabies and encephalitis), surveillance of infection in the reservoir host may be as important as surveillance in the human population.
The case definition is fundamental to any surveillance system because it is the formal answer to the question of what manifestations of a disease or condition are under surveillance (CDC 1997) (see http://www.cdc.gov/epo/dphsi/casedef). It is both a criterion for determining who is counted and a guide to local health departments for case investigations and follow-up. It ensures that the same measure is used across geographical areas. The case definition must be sufficiently inclusive (sensitive) to identify people who require public health attention but sufficiently exclusive (specific) to avoid unnecessary diversion of that attention. In addition, the case definition must be usable by all people on whom the system depends for case reporting. There is no ideal case definition for any particular disease or condition. The following are two possible case definitions that could be used in conducting surveillance for hepatitis A.
Illness characterized by jaundice, elevated liver enzymes, and serological detection of immunoglobulin M antibodies against hepatitis A. This definition presumes that affected people will have access to health-care services, including diagnostic testing. This approach would exclude people who have an epidemiological and clinical picture consistent with hepatitis A but who lack serological testing. The definition also does not include people with asymptomatic hepatitis A infection. An alternative may be to use a definition that includes all people with a positive test for immunoglobulin M antibodies against the hepatitis A virus. To accommodate these possibilities, the case definition may be subdivided to allow for symptomatic versus asymptomatic cases or for gradations based on certainty of diagnosis (confirmed case, presumptive case, possible case, and so on).
Illness characterized by yellow eyes. This definition is simple and may be appropriate in a setting where hepatitis A transmission has been documented and where there is limited access to diagnostic services, where field staff have little formal training, and where there is an emergent need to assess a population rapidly (for example, a common-source outbreak). With this definition, hepatitis case counts may include some people with jaundice caused by other conditions, but the lack of specificity may not substantially affect the usefulness of the overall information.
Neither of these two definitions or possible variations is inherently better than the other, and each may be appropriate in a given setting. The first definition is more specific but may lack utility in circumstances in which laboratory testing is limited. The second definition would not be appropriate for a population in which the majority of people are immune to hepatitis A, either because of past infection or vaccination.
Thus, the definition must be geared to the circumstances of each surveillance system. The definition must also remain current as conditions change. During the course of the poliomyelitis eradication campaign, different definitions have been used, with a less specific definition needed when cases of poliomyelitis are common. Following successful vaccination campaigns and a large reduction in disease incidence, case definitions for poliomyelitis require more specificity, which in turn requires laboratory confirmation of cases of acute flaccid paralysis (Andrus et al. 1992).
A similar range of possible case definitions also exists for surveillance systems that focus on adverse health exposures rather than disease outcomes. For example, in a surveillance system that addresses occupational hazards, exposure to a harmful substance may be monitored by self-report of workers, by company log-books of manufacturing procedures, or by routine measurement of substances in the work environment, on workers’ clothing, or in specimens collected from workers. Each of these possible case definitions would require different levels of co-operation from the company or workers, and each may be subject to unique limitations that could bias surveillance.
The flow of information
Although surveillance systems for chronic diseases and other conditions frequently rely on multiple data sources that may be collected primarily for purposes other than public health, many surveillance systems depend on data acquired explicitly for surveillance purposes and rely on a sequential flow of these data through the full surveillance cycle. Each facet of this process should be carefully planned, as described below.
People responsible for reporting cases may be all health-care providers in a defined area, selected providers, or people at specific institutions (for example clinics, health-care organizations, laboratories, hospitals, schools, factories, and so on). In addition to communicating case reports, reporters may be responsible for collecting specimens needed by public health agencies for laboratory confirmation or application of molecular epidemiological techniques (for example, determining whether a case of poliomyelitis has been caused by a vaccine strain or a wild-type poliomyelitis virus).
Data collection instruments
The desire to collect detailed information must be tempered by the need to limit data to items that can be reliably and consistently collected over the long term. Forms or other data collection instruments that are too detailed and too complicated will not be welcomed by those on whom the surveillance system must depend. This is independent of whether the forms are computerized, although computerization may make the process more acceptable to reporters. In addition, computerized systems that exist for other purposes (for example patient records) may permit more detailed collection of data without additional burden to the reporter.
In addition to the widespread use of computers in surveillance (Koo and Wetterhall 1996), standards for exchanging information are critical to the future utility of all public health surveillance and information systems. Many coding systems currently used in public health are not compatible with the needs of other organizations. In 1996, the American Congress passed the Health Insurance Portability and Accountability Act to encourage the development of standards for data related to health care (Chute et al. 1998). The passage of this legislation has increased the level of activity related to integration of clinical information in the United States.
Surveillance systems provide data on a regular basis, ranging from daily to annually. Whatever periodicity is used should be specified and adhered to by participants in all phases of the surveillance loop; reporting should occur even when the number of reported cases is zero. To contain an outbreak of meningococcal meningitis, the health department must receive reports of cases quickly (that is, within 24 h) so that necessary control measures may be taken immediately. In contrast, a breast cancer registry evaluating the effectiveness of targeted screening services for breast cancer may collect and analyse data on a quarterly basis or even less frequently. To achieve the objective of the surveillance system, the appropriate timing should be considered as the system is established. With increasing computerization and Internet use, reporting at the time of case identification is becoming a reality.
Aggregation of data
Surveillance data may be in the form of individual patient records or aggregate counts and tabulations. For example, there may be a need at the local level to maintain records on individual people to direct follow-up services. In addition, individual data permit more flexibility of analysis than aggregate data, and computerization has facilitated transfer of case-specific data more easily to a central level.
The mode of data transmission will depend on both the need for timeliness and communications resources. In many health agencies in developed and developing countries, computers are still unavailable and reliance on postage of forms and facsimile transmission is needed. During the 1980s, computers were introduced and increasingly used in surveillance systems, facilitating transmission of data in electronic formats and Internet communications (Valleron et al. 1986; CDC 1991; Koo and Wetterhall 1996).
Computerization has also been helpful in transmitting molecular data on isolates of certain pathogens, such as the pulsed field gel electrophoresis patterns of E. coli 0157:H7 through Pulse-Net (http://www.cdc.gov/ncidod/dbmd/pulsenet/pulsenet.htm). In addition, computerization may facilitate and enhance regular and personal contact among public health officials, health-care providers, and others who participate in such activities as a closed electronic mail system. The Emerging Infections Network, in which hundreds of infectious disease practitioners participate in the United States, uses an electronic mail conference for on-line discussion when new insights into disease occurrence are needed and allows for close communication between public health officials and health-care providers (Executive Committee of the Emerging Infections Network 1997).
Data management and dissemination
The following issues in data management and dissemination should be considered in planning for storage, analysis, and dissemination of surveillance information.
Surveillance data often need to be updated. Information that was initially unattainable may become available, follow-up investigations yield supplemental information, people initially classified as meeting or not meeting a case definition may be reclassified, errors in reporting may be identified and corrected, and duplicate case reports may be recognized and culled. One approach to handling these and other changes is to maintain both provisional and final records, including separate publications for provisional and final data. When analysing trends, it is often useful to compare provisional data in one period of time with provisional data from another point in time, since bias in preliminary data may change when data are updated. Provisional reports may satisfy immediate information needs, whereas final and more delayed reports can accommodate corrections and updates to a reasonable limit and can serve an archival function. Computerization facilitates record updates.
Selecting measures for time and place
A case report may include dates, such as those of the onset of disease, the diagnosis, the report to local health authorities, and the report to regional or national health authorities. Analyses of surveillance data may be based on the date of any of these events. However, if there are, for example, long delays between dates of diagnosis and report, analyses of trends based on date of diagnosis will be unreliable for the most recent periods. Similarly, surveillance data may be tabulated on the basis of the site of occurrence of the health event, the site of diagnosis, or the residence of people reported. The selection of these measures for time and place may also differ for provisional and final surveillance reports. As with other statistical methods, the availability of software aids spatial analysis methods (Dean et al. 1991; Biomedware 1994).
Preventing inappropriate disclosure of surveillance data is essential both to the privacy of people with reported cases of disease and to the trust of participants in the surveillance system. The protection of confidentiality begins with limiting data collection and transmission to a minimum and includes ensuring the physical security of surveillance records, the discretion of surveillance staff, and legal safeguards (Federal Committee on Statistics 1994). To elicit public health surveillance information from the public and from health-care providers, strong laws that assure a careful procedure for maintaining and reporting data are frequently necessary to ensure the privacy of personal information (Gostin and Hadley 1998). Privacy regulations in the United States are currently based on a patchwork of state and local legislation and may lack adequate protection for electronic health information. Recent bills introduced in the American Congress include definitions of protected information and descriptions of disclosures that may occur with or without consent. Sometimes forgotten in the discussion of health information privacy is the concept that use of electronic information systems can often improve the security of data.
Physical protection of records is accomplished by rules of conduct for people involved in the design, development, operation, or maintenance of any surveillance system. For example, confidential records should be kept locked up at all times when not in use. When confidential records are in use, they must be kept out of the sight of people not authorized to work with the records. Except as needed for operational purposes, copies of confidential records should not be made. When confidential surveillance records are in the possession of other agencies, provision should be made for their protection.
Provision of data containing identifiers of individuals or establishments should be held to the minimum number deemed essential to perform public health functions. Categories should be sufficiently broad to avoid inadvertent identification of an individual person or institution. In particular, release of information for small geographical areas must be carefully considered to protect confidentiality (Committee on National Statistics 1993).
Initiating and maintaining participation
Public health agencies depend on the ongoing co-operation of others to identify and report cases in most surveillance systems. Whether reporting is required by law, is voluntary, or is financially rewarded, most reporting still takes time and effort (this may change as information systems play a larger role). Many approaches require contact of public health professionals with the reporting sources; dissemination of reports that document the usefulness of surveillance data are likely to be a key to initiating and maintaining participation in the system. In addition to professional meetings and other personal contacts, electronic mail affords an excellent route of informal communication between public health professionals and health-care providers.
For certain diseases, reporting is often required by law. Although legal mandates may not guarantee reporting, they establish the authority under which health agencies conduct surveillance. In addition, reporting laws and regulations may identify not only those who are required to report cases but also those who may report cases without fear of liability for violation of privacy. Statutes may also protect health agencies from forced disclosure of the identity of people with particular diseases.
If the surveillance loop of data collection, analysis, interpretation, and feedback is to function as a continuous process, an organizational structure is required. Such a structure depends on the resources available, including the number of personnel and their level of training, the technology available for communication and data management (for example computers), and financial constraints, as well as the number and type of diseases, health conditions, or risk factors under surveillance. In one example of a simple form of reporting, the organizational structure requires health-care providers to report a single disease or health event on a regular basis to a co-ordinating public health authority. A more complex form would include a network of reporting units dealing concurrently with problems related to many diseases. In any case, the structure must allow data to be gathered from various sources and evaluated by epidemiologists in time for appropriate action to be taken. These data must be routinely disseminated to a targeted audience and reproduction and telecommunication equipment are minimal requirements.
The structure should provide support for training of key personnel in surveillance through seminars, distance-based learning, or other venues that give field and central staff the opportunity to review procedures and to resolve operational problems. Appropriate technical support, such as provision of diagnostic reagents, laboratory space, and computer equipment, must also be ensured. Finally, the need for regular evaluation of the surveillance systems should be recognized.
Delegating tasks to international, national, regional, and local health authorities should depend on information needs and resources at each level. Particular attention should be directed to the local level because primary responsibility for information collection and public health responsibilities are usually local. Central agencies are responsible for guiding, as well as co-ordinating, data collection procedures; they ensure that surveillance data are collected using standardized methodology such that the data from one geographical area can be reliably compared with data from another area and such that the data can be aggregated into regional or national summaries. Also, because many monitoring efforts for non-infectious conditions (for example, traffic injuries, water pollution, and so on) are often dealt with by governmental agencies other than public health agencies (for example, police authorities, environmental protection agencies), there needs to be effective co-ordination between health authorities and other appropriate authorities; for this purpose, procedures may need to be established to ensure the necessary communication.
Public health surveillance data are collected in many ways, depending on the nature of the health event under surveillance, potential methods for identifying the disease, the population involved, the resources available, and the goals of the programme. Some surveillance systems may rely on a single source of data with alternate data sources being used periodically to evaluate or to enhance the completeness of routine surveillance data.
Notifiable disease reporting is the surveillance approach traditionally used by public health programmes. A system of notification is based on laws or regulations by health authorities that require reporting of selected diseases or conditions, usually infectious, to the health department to support and direct prevention and control programmes (Rousch et al. 1999). Notification reporting may be instituted at many levels (local, national, and international). Ultimately, under a system of notification, the reporting will be most useful and most accurate for diseases if surveillance is supported and emphasized at a local level. People or institutions with responsibility for reporting to the public health authority often include doctors, other health-care providers, coroners and medical examiners, laboratories, and hospitals. Historically, doctors and other health-care workers such as infection control practitioners have been most important to systems of notification. Reliance on laboratory reporting and on computerized records collected primarily for other reasons is increasing.
In any country, the extent of notification activities depends on the availability of facilities and resources—trained staff, laboratory and other equipment, epidemiological services, and liaison with health-care providers and other key reporters—as well as the health priority of the disease and method of diagnosis (Berkelman et al. 1994). Reports are often initiated by health-care providers or other reporting source; for some diseases for which more complete reporting is sought, public health professionals may contact major reporting sources and/or review laboratory or other relevant records to ensure that cases are ascertained. These systems of reporting have been described as passive and active respectively, but the distinctions are not always clear. Data for many surveillance programmes represent a mixture of both reports elicited by public health professionals contacting health-care providers or reviewing records and reports submitted by health-care providers to public health officials without direct solicitation. Computerization of patient and laboratory records should facilitate reporting.
Reporting is generally incomplete for most notifiable diseases (Hinman 1977; Vogt et al. 1983). If people are asymptomatic or have only mild symptoms, they will not usually seek health care. Patients and doctors may conceal diseases that carry a social stigma, such as sexually transmitted diseases. Health-care providers may also fail to report because they may be unaware of regulations or because they may treat the symptoms without a complete laboratory investigation. Completeness of reporting may also be significantly influenced by factors such as medical community interest and publicity; the most important is probably the intensity of surveillance efforts, which is closely linked to availability of resources (Davis and Vergeront 1982; Buehler et al. 1992). Many incomplete data may serve their purpose, however. Epidemics, as well as general temporal and geographical trends, can be determined as long as the proportion of cases detected remains consistent over time and across geographical areas.
A comparison between cases of viral hepatitis reported by practitioners in private practice and cases reported in a population covered by an insurance plan in Israel demonstrated that, although completeness of reporting by the doctors was only 37 per cent, the distribution of reported cases by season and age was similar to that recorded in the insured population (Brachott and Mosley 1972). However, under-reporting may affect representativeness; a study of under-reporting of acute viral hepatitis in the United States demonstrated that homosexual men with hepatitis B and blood transfusion recipients with non-A non-B hepatitis were less likely to be reported than members of other risk groups (Alter et al. 1987). Thus, surveillance data acquired through reports initiated by health-care providers may not accurately reflect the risk for specific populations.
The potential for re-emergence of infectious diseases requires continued vigilance and capacity to respond, even though the control programme may not be a high public health priority (for example, plague). As new infectious agents are recognized, the need to expand the surveillance system to control these agents effectively has been recognized in many countries. Furthermore, as international travel and commerce facilitate the rapid spread of pathogens from one part of the globe to another, the need for improved international communicable disease surveillance has become apparent (Institute of Medicine 1992; Heymann and Rodier 1998).
Although infectious conditions have dominated the list of notifiable diseases in most countries, other diseases and conditions may also have to be reported. Adverse drug reactions, occupational injuries, poisonings, and specified malignancies, among others, may be required to be reported, particularly in developed countries (Faich et al. 1987; DeBock 1988; Freund et al. 1989; Koo and Wetterhall 1996).
In settings where the infrastructure does not exist to support accurate case reporting systems yet there is a need to assess the impact of a disease on morbidity, an alternative and potentially simple and rapid approach is to survey hospitals periodically for the number of admissions attributed to a particular condition (DeCock et al. 1989). Another crude but inexpensive surveillance system for diseases with high morbidity rates and for which notification may not be appropriate (for example, gastrointestinal illnesses or influenza) may be based on absenteeism from schools or industry, depending on the ages of the affected populations.
Health-care provider networks
Networks of health-care providers have been organized in recent years, primarily to gather information on selected health events. Most have been organized by practising doctors on a voluntary basis; in many European countries, these networks have formed firm relationships with both public health authorities and academic centres, and often form the basis for morbidity surveillance (Valleron et al. 1986).
The strengths of sentinel provider systems include the commitment of the participants, the possibility of collecting longitudinal data, the flexibility of the system to address a changing set of conditions, and the ability to gain information on all patient-provider encounters, regardless of severity of illness. The most severe limitation of this type of system is that the population served by these doctors may not be representative of the general population. In addition, the illness must be fairly common to provide representative incidence data from a small sample of doctor contacts.
A voluntary network of general practitioners in Belgium was initiated in 1978 (Stroobant et al. 1988). Practitioners were selected who were representative of Belgian general practitioners according to age and sex and who were geographically distributed to ensure coverage of the country. Participants report weekly and the results are sent to the participants on a quarterly basis. The list of health problems has included selected vaccine-preventable diseases, respiratory conditions, and suicide attempts, with some health problems such as mumps and measles reported continuously and others on a less frequent basis. A high level of participation has been documented, with the degree of form completion and continuity of reporting as criteria for assessment. The network has been evaluated in terms of its possible biases, such as non-participation of practitioners and difficulties in estimating the population at risk for the health problems under study; methods have been developed to reduce these biases (Lobet et al. 1987).
The British Paediatric Surveillance Unit (BPSU) (http://bpsu.rcpch.ac.uk/) was initiated in 1986 through the collaboration of several agencies: the British Paediatric Society, the Public Health Laboratory Service Communicable Disease Surveillance Centre, and the Department of Epidemiology at the University of London. The BPSU has also added collaborators from specialty groups such as orthopaedics, rheumatology, and dermatology. The BPSU has enabled paediatricians to participate in the surveillance of infections and infection-related conditions and in studies of uncommon disorders. It also provides a mechanism by which new diseases can be detected quickly and monitored. The reporting system involves the mailing of a monthly card which contains the disorders currently being surveyed. Examples of conditions under surveillance have included HIV infection and AIDS, insulin-dependent diabetes mellitus, acute flaccid paralysis, and new variant Creutzfeldt–Jakob disease in children.
In 1979 the Ministry of Public Health in China and the Chinese Academy of Preventive Medicine initiated a sentinel network of surveillance sites to address the need for more timely and representative data (Cheng 1992; Yang 1992). Data from county and provincial epidemic prevention stations are reported to the Chinese Academy of Preventive Medicine. The entire surveillance system consists of five components: the National Notifiable Diseases Reporting System, disease-specific surveillance systems for endemic areas, mortality statistics, natality statistics, and a sentinel surveillance system for approximately 35 infectious diseases, births, deaths, vaccinations, and risk factors. Data are analysed monthly, and a report is distributed to health officials in more than 3000 counties. The data have been used to develop prevention programmes (Zhang 1990), detect changes in infectious agents (Shen et al. 1984), and influence policy affecting communities (Yang et al. 1997).
Surveillance of routinely collected laboratory reports has been particularly useful for certain infectious conditions. For instance, in the United States, reporting from many public health laboratories is automated (Bean et al. 1992). In England and Wales nearly all microbiology laboratories report positive identifications of specified infections each week to the Communicable Disease Surveillance Centre. The advantages of the laboratory reporting system are its specificity, its flexibility in adding new diseases, its rapidity, and the amount of detail about the infectious agent that can be provided. Reports indicate trends or the appearance of rare infections originating from a common source that could not be identified by a single laboratory. One disadvantage is that the number of people from whom specimens are tested is usually not reported. In addition, the people tested may not be representative of the population at risk. For some infections, such as toxic shock syndrome, there is no laboratory test, and for many common illnesses a specimen may not be taken (for example influenza).
Nosocomial infection surveillance is often based on review of laboratory records by an infection control nurse or other designated staff (Brachman 1982). In 1970 in the United States, the National Nosocomial Infection Study was initiated to monitor the frequency and trends of nosocomial infection in American hospitals. Approximately 160 hospitals participate in what is now a voluntary national surveillance system, with microbiology studies reported on 90 per cent of infected patients (Gaynes et al. 1991). A network of laboratories of different medical centres around the world has been established to conduct surveillance of antibiotic resistance for various pathogens (Stelling and O’Brien 1997).
In addition, the use of molecular tools to enhance surveillance of pathogens is growing in many countries. PulseNet serves as a American network of public health laboratories that performs DNA ‘fingerprinting’ on bacteria that may be foodborne. The network permits rapid comparison of these ‘fingerprint’ patterns through an electronic database, for example, similar pulsed field gel electrophoresis patterns of E. coli 0157:H7 bacteria isolated from ill people suggest that the bacteria come from a common source, for example, a widely distributed contaminated food product (http://www.cdc.gov/ncidod/dbmd/pulsenet/pulsenet.htm).
Registries are comprehensive longitudinal listings of people with particular conditions. They often include detailed information about diagnostic classification, treatment, and outcome. Registries were initially established primarily for epidemiological research on individual diseases or conditions to develop aetiological hypotheses and to identify cases for further research (Weddell 1973). Registries have also been used to ensure the provision of appropriate care and to evaluate changing patterns of medical care; unlike other disease information systems, they cut across the different levels of severity of illness and may provide information over time about individual people. Recently, the value of registries for monitoring disease incidence and its distribution, as well as for evaluating the effectiveness of targeted screening programmes, has been more widely recognized.
To focus on selected diseases or conditions, registries often develop a constituency that promotes participation and reporting. Most registries rely on numerous sources of data for case detection including, but not limited to, hospitals, laboratories, and death records; few registries rely primarily on doctor notification. Public health professionals probably have the most experience with cancer registries and registries for congenital malformations.
Population-based cancer registries generally have relied on multiple sources of data, including most importantly clinical pathology laboratories and hospital diagnoses (Parkin 1988). Death certification is also important, and other records such as those from oncology or radiotherapy units are also useful where available. There has been increasing adherence to internationally recognized standards and the resulting data are used to compare the incidence of cancer in different geographical locations and distinct ethnic groups (Raymond 1997). In the United States, several population-based registries have been developed that conduct surveillance for cancer, and the national Co-ordinating Council for Cancer Surveillance was organized in 1995 to facilitate a collaborative approach among the involved organizations and to ensure maximal efficiency (Swan et al. 1998). In contrast, in many developing areas of the world, population-based surveillance systems are not feasible (Parkin 1986), but surveillance in selected institutions or laboratories may still be useful.
Surveillance for birth defects was first initiated in many parts of the world in response to the thalidomide tragedy; registries were established to provide reliable baseline rates for specific birth defects and to detect increases in the prevalence of birth defects as a means of rapidly identifying human teratogens (Kallen et al. 1984; Holtzman and Khoury 1986). The CDC has conducted birth defects surveillance in metropolitan Atlanta since 1967 by using multiple sources of ascertainment of all serious birth defects observed in stillborn and liveborn infants or recognized by signs and symptoms apparent in the first year of life (Edmonds et al. 1981; CDC 1993). The birth defects registry system in metropolitan Atlanta has been a valuable resource for monitoring rates of change of specific defects (Yen et al. 1992) and for conducting numerous genetic and epidemiological investigations of risk factors for birth defects (Mulinare et al. 1988; Erickson 1992). Moreover, the registry serves as a model for other state surveillance systems (Lynberg and Edmonds 1992). A total of 38 states are conducting or are planning birth defects surveillance activities (Erickson 1997). In addition to monitoring birth defect rates and serving as the basis for epidemiological studies, the data from these state registries are used to evaluate the effectiveness of prevention activities and to refer children for health services and early intervention programmes (Edmonds 1997).
Internationally, approximately 30 countries are now conducting birth defects surveillance and are members of international organizations such as the International Clearinghouse for Birth Defects Monitoring Systems (International Clearinghouse for Birth Defects Monitoring Systems 1998) and, for Europe, the EuroCAT (Lechat and Dolk 1993). The International Clearinghouse, for example, conducts a spectrum of surveillance activities that includes monitoring of selected conditions (for example, several birth defects, Down’s syndrome, multiple congenital anomalies) and the exchange of ‘rumours’, cluster information, and findings of still unpublished studies (International Clearinghouse for Birth Defects Monitoring Systems 1998). Additionally, the Clearinghouse promotes collaborative epidemiological studies and the development of new surveillance programmes worldwide.
Health information systems
Surveillance systems often depend on existing health data collection systems; these systems may be either integral to surveillance or serve as an adjunct to surveillance for specific diseases or conditions. Lack of accuracy and specificity in these existing data systems remains a concern, however, and most surveillance systems continue to need an additional data collection system to meet the needs of specific prevention and control programmes (Calle and Khoury 1991).
Data from medical claims records, death certificates, and other existing databases may not contain enough information to define public health priorities for reducing disease incidence. For example, an increase in deaths from cirrhosis, not otherwise specified, may be the result of an infectious agent, alcohol use, or other toxin. The occurrence of bladder cancer may or may not be related to a particular environmental exposure. These data are often most useful as an adjunct to surveillance systems designed more specifically for prevention and control programmes.
Mortality statistics serve as the most accessible source of data for comparisons of many health problems. In most developed countries, registration of deaths is compulsory and largely complete. Records include basic demographic information, the cause or causes of death, and other descriptive information about the circumstances of death. In other countries, registration may be conducted only in major cities, or not at all. Verbal autopsies may be used in areas without death registration (Kaufman et al. 1997; Chandramohan et al. 1998).
In all countries, the accuracy and specificity of many diagnoses are limited, and changes in the use of diagnostic categories and codes over time, together with variation in the quality of information, are limiting factors. For example, a study by the American National Cancer Institute revealed that seven countries in Europe and North America coded the underlying cause of death the same for only 53 per cent of a sample of 1246 death certificates sent to these countries (Percy and Dolman 1978). Despite these limitations, vital statistics, particularly mortality statistics, are used to support many surveillance activities.
Death certificates have been used in maternal mortality surveillance as a source of data to demonstrate progress towards reduction in maternal mortality in association with increased use of prenatal care and other factors. Analyses of death certificates in the United States have highlighted racial differences in mortality rates over time and differences in maternal mortality rates for women aged greater than 35 years. Because maternal mortality rates are often based on number of live births, this surveillance system also depends on birth certificate information (Kaunitz et al. 1984).
There is frequently a lengthy interval between death and collection and analysis of death certificates, which may make such vital statistics less useful for surveillance purposes when more current data are needed. However, summary vital data can be rapidly collected. For example, weekly reporting of deaths from 121 American cities to CDC has been integral to the surveillance of influenza epidemics in that country (Choi and Thacker 1981). In addition, automated systems for coding mortality information are both expanding and improving internationally (CDC 1999b).
Medical examiner and coroner reports
For a more detailed description of circumstances surrounding deaths (including autopsy reports, toxicology studies, and police reports), medical examiner and coroner records may be useful. In the United States, these reports are most representative of deaths caused by intentional and unintentional injuries and other unnatural causes. These records have been used for surveillance of such conditions as heat-wave-related mortality, sudden unexplained death syndrome in Southeast Asian refugees, and alcohol-related injuries (Jones et al. 1982; Berkelman et al. 1985; Parrish et al. 1987; Koo and Birkhead 1998). Systematic necropsy examinations have also been useful in ascertaining the contribution of tuberculosis to mortality of HIV-infected individuals in West Africa.
Medical care records
Hospital records and other medical care records may be a useful source of information on diagnoses, surgical procedures, and patient demographic characteristics. However, with increases in length and complexity of the medical record, retrieval of information has often been difficult and time-consuming. Although computerization of parts of these records has allowed their use for routine surveillance, a major limitation has often existed when identifiers are not recorded because repeat admissions and discharges by individual patients usually cannot be identified.
Hospital discharge records have been useful for surveillance of many medical care technologies, such as trends in the use of hysterectomies in the United States (particularly by geographical region), in the rate of coronary artery bypass graft procedures by sex and race, and in the assessment of outcome with carotid endarterectomies (Sattin et al. 1983; Thacker and Berkelman 1986; Caper 1987; McBean and Gornick 1994). More recently, hospital discharge record systems have been used as an alternative data source to evaluate surveillance data sets.
Relying exclusively on measurements of mortality from death certificates and morbidity from medical records can produce an underestimate of the impact of many diseases or conditions, such as malaria and arthritis, which may not result in contact with a health-care provider. A more complete estimate of the impact of disease has been attempted by including a measure of the loss of healthy life resulting from disability (Murray and Lopez 1997).
Insurance records and workers’ compensation claims
Insurance records and workers’ compensation claims have been useful for surveillance of injuries and illnesses in specific geographical locales. Because regulations governing completion and submission of forms differ both among and within juridictions, data derived from these systems cannot easily be compared. In addition, the use of medical claims data for surveillance may be limited by the accuracy of diagnostic recording as well as the problem of comparing different health systems (Pollack and Ringen 1992).
The severity of reported injury varies and is influenced by regulations influencing eligibility for workers’ compensation, and other legislation related to compensation and medical care, rehabilitation of those injured at work, and the degree of fear of job loss resulting from absence from work. Data from these systems generally provide an underestimate of the actual incidence or prevalence of the health condition under surveillance; the underestimate may be of a considerable degree.
In an evaluation of claims for workers’ compensation as an adjunct to an occupational lead surveillance system, the usefulness of claims was demonstrated: the likelihood that a company had a case of lead poisoning strongly correlated with the number of claims against the company (Seligman et al. 1986).
Surveys of health behaviour and doctor utilization
Household surveys of the general population, such as the National Health Interview Survey conducted in the United States (http://www.cdc.gov/nchs/nhis.htm) or the General Household Survey in England and Wales (Fraser et al. 1978; Twigg 1999), have provided information at the national level on personal health practices such as alcohol use and smoking, disabilities, and doctor encounters. In the People’s Republic of China, in addition to mandated information on acute infectious conditions, sentinel sites, known as disease surveillance points, are chosen through a statistical sample of provincial areas. These sites collect data on health events and medical encounters for the entire population within their jurisdiction (Cheng 1992; Yang 1992).
Although national estimates may be gained more efficiently from such surveys, local programmes may benefit from involvement in data collection and the flexibility to adapt data collection to their particular needs. Interview surveys conducted by telephone and in person can obtain personal health-related information with only minor differences in the reported prevalence of various health conditions between the two techniques. In developed countries, where most residences have telephones, telephone interviews have the advantages of lower cost and ease of supervising interviewers (Siegel et al. 1991).
Hazard and exposure surveillance
In addition to surveillance of health outcomes, two other types of surveillance are also used: hazard surveillance and exposure surveillance. Hazard surveillance has been defined as the ‘assessment of the occurrence of, distribution of, and the secular trends in levels of hazards (toxic chemical agents, physical agents, biomechanical stressors, as well as biological agents) responsible for disease and injury’ (Wegman 1992). Exposure surveillance is the monitoring of a population for the presence of an agent or the clinically non-apparent effects of an environmental hazard (for example, lead) or infectious agent (for example, HIV) in people within a population (Weniger et al. 1991). These types of surveillance may be complementary. Indeed, the optimal strategy for preventing or reducing the impact of a specific public health problem sometimes dictates the use of all three types of surveillance. For example, although hazard surveillance is an excellent measure available to detect potential health threats and to provide opportunities for primary prevention, exposure and outcome surveillance may provide valuable information for the evaluation of the effectiveness of hazard and exposure-reduction regulations.
Remote sensing and geographical information system technologies are now being used as an adjunct to disease surveillance (Barinaga 1993; Washino and Wood 1994) (see below). A major goal of this approach is the identification of environmental parameters that affect the patterns of disease risk and transmission.
In 1993 the southern Kerio Valley of Kenya experienced the first cases of yellow fever ever recorded in that country. The virus had recently infected the monkey population in the valley, and forest-dwelling mosquitoes were passing the virus from monkeys to people entering the forest. The human population in the area was vaccinated, but the question remained as to whether the inhabitants of the nearby cities also required vaccination. Satellite photographs are being used to determine whether continuous forest corridors to any of the cities exist through which the virus can travel from one monkey population to another.
Analysis of surveillance data
The uses of public health data often derive from a simple analysis of surveillance data according to the basic epidemiological parameters of time, place, and person. Analysis of data over time can reveal trends in disease or injury upon which public health actions or the need for such actions may be evaluated.
The characteristics of the people or groups who develop specific diseases or sustain specific injuries are important in understanding the disease or injury, identifying those at high risk, and targeting intervention efforts. For example, disparities in health (incidence or severity of disease) among members of different population groups highlight the need to identify cultural, economic, or social factors associated with these health problems (Hahn and Stroup 1994).
When combined with appropriate population information, morbidity or mortality rates can be calculated to compare risks of disease and the magnitude of various health problems. Often rates are examined in broad age groups that are selected to reflect the different sets of conditions affecting mortality rates in each group (Doll 1974). Proper analysis of surveillance data can also assist in determining aetiology, setting priorities, determining modes of transmission, risk factors associated with disease, and opportunities for prevention or control, detecting epidemics, monitoring long-term trends, making projections of future disease occurrence, and evaluating effectiveness of interventions.
Typically, public health surveillance data are completed, summarized, and reported over specified time intervals (for example weeks, months, or years). Methods applicable to such time series data can be used to separate true temporal trends in the underlying risk from the random fluctuations, or ‘noise’. The surveillance data should be plotted over the time during which they were collected. A clearer picture of this possible trend and of meaningful short-term patterns in these data are more evident if the random day-to-day variation in the number of cases is reduced. Smoothing potentially highlights meaningful patterns in collections of observed data by reducing the level of random noise (Devine and Parrish 1998). More advanced methods for analysis of surveillance data by time includes autoregressive time series techniques (Box and Jenkins 1976), generalized regression methods (Zeger 1988; Singh and Roberts 1992), and Bayesian modelling (Stroup and Thacker 1993). In addition, forecasts may be made by using regression and time series analyses, for example as for the surveillance of influenza (Fig. 1) (Serfling 1963; Choi and Thacker 1981; Lui and Kendal 1987; CDC 1999c), or analyses may combine information from several surveillance series (Newhouse et al. 1986; Stroup et al. 1988). In addition, modern methods (for example, Bayesian techniques) that incorporate information in addition to the data themselves, such as changes in a surveillance case definition or surveillance information from contiguous data (Stroup and Thacker 1993), can be useful when applied to public health surveillance data.
Fig. 1 Pneumonia and influenza mortality for 122 American cities, January 1995 to April 1999 (CDC 1999c).
The approach to the prevention and control of disease and injury is often determined by circumstances unique to ‘place’: the geographical distribution of the disease or of its causative exposures or risk-associated behaviour. The analysis of surveillance data by place has long used dot density maps. As with analysis by time, smoothing approaches can reduce the random noise in maps of surveillance data (Fig. 2) (CDC 1996; Devine and Parrish 1998). Timely and positionally accurate spatial or georeferenced information, in a digital format, is improving the ability to monitor disease occurrence, health inequalities, environmental exposures, and related health risks, and advance hypothesis generation about the associative causation of disease aetiologies and outcomes (Richards et al. 1999). For example, from the American Bureau of the Census national digital street and geographical boundary files, or Topologically Integrated Geographic Encoding and Referencing (TIGER) system, epidemiologists can translate or geocode street addresses into unique latitude and longitude locations. These locations then can be examined with computationally rigorous spatial statistical data analysis techniques using geographical information systems. New and continually expanding computational opportunities permit dynamic space–time modelling of georeferenced data on the extent, structure, and association of diseases and suspected covariates (Anselin 1998).
Fig. 2 Smoothed age-specific death rates (per 100 000 population) for age 70, colorectal cancer, white females from death certificates 1988 to 1992 and population data for 1990 (CDC 1996).
Compared with traditional methods of mapping, geographical information systems offers potentially substantive cost savings for local disease surveillance and prevention activities. For example, epidemiologists in local health departments have used geographical information systems to design automated early warning surveillance systems for newborns with a known residential potential for elevated exposures to nitrate-nitrogen in drinking water (Boria et al. 1999). Other applications have resulted in identifying locations with elevated risks of Lyme disease (Glass et al. 1995) and rodent bite and infestation (Childs et al. 1998).
Epidemic detection and cluster analysis
Many epidemics are detected by astute health-care providers who note or suspect an increase in disease occurrence often before disease reports are received, assembled, and reviewed by health departments. The ongoing surveillance process between health practitioners and health departments increases the likelihood that providers will contact the health department when they suspect an outbreak or any unusual occurrence of disease.
Surveillance is most likely to detect epidemics in situations where cases, despite their aetiological link, are occurring over a wide geographic area (Cliff et al. 1992), over a relatively gradual period (Nobre and Stroup 1994), or among a well-defined subgroup with links among cases (for example, epidemiological links or similar molecular patterns of isolates) that would not be apparent to individual practitioners.
For example, laboratory-based surveillance of Salmonella serotypes has identified outbreaks in which unusual serotypes and/or antimicrobial patterns identify an outbreak of diarrhoeal disease that might otherwise have gone undetected, such as the outbreak of drug-resistant Salmonella newport in a large geographical area of the United States, which originated from animals fed antimicrobial agents (Holmberg et al. 1984). In addition, a computer algorithm based on application of cumulative sums has been developed to identify outbreaks of Salmonella (Hutwagner et al. 1997).
In another example, age-adjusted oesophageal cancer mortality rates in white men and women in the United States have remained fairly steady during the period between 1950 and 1980 but nearly doubled for African-Americans during the same period. Oesophageal cancer has gradually become one of the most common malignancies in African-American men aged below 55 years, while still a relatively rare cancer in white men of a similar age (Blot and Fraumeni 1987).
A frequent concern of the analysis of surveillance data is whether an apparent cluster of health events in time is significant and unlikely to have occurred by chance alone. Although the application of cluster detection methods is growing (CDC 1990), many of these methods require further examination and their use in surveillance remains controversial (Hutwagner et al. 1997; Devine and Parrish 1998). Most of these methods involve a comparison of observed incidence with a historical baseline and may involve clustering in time (Wallenstein 1980; Gallus et al. 1986; Stroup et al. 1993), clustering in space (Cliff and Ord 1981), or clustering in both time and space (Klauber 1975), with the availability of statistical software aiding spatial and other analyses (Dean 1991; Biomedware 1994). For example, a computer algorithm based on application of cumulative sums has been developed and used to identify outbreaks of Salmonella. These methods of cluster detection may be most helpful when used as an exploratory analysis to advise more intensive investigations (Besag and Newell 1991).
Statistical limitations of surveillance data
Surveillance data have traditionally had specific characteristics that have made application of standard statistical techniques difficult. Firstly, reporting bias may produce data that are not representative of the population. For example, severe or otherwise noteworthy cases are more likely to be reported than minor illnesses. Rubella in a woman of child-bearing age is more likely to be reported than rubella in a man, and a patient in some clinics may be more likely to be reported than patients seen in other settings, particularly if the disease is socially stigmatizing. Health-care provider networks may have a biased sample of doctors (Lobet et al. 1987).
Secondly, under-reporting may be considerable, particularly in a voluntary system of notification. When another independent source of data is available (for example, hospital discharge, vital statistics), the total number of cases actually occurring can be estimated (Chandra-Sekar and Deming 1949; Cormack 1963) to determine, for example, the sensitivity of two systems for detecting vaccine-preventable diseases (Orenstein et al. 1986). In addition, specific information for each case may be incompletely reported (Buehler et al. 1989).
Provisional data increase the timeliness and hence may increase the usefulness of public health surveillance data to epidemiologists; however, provisional data may differ markedly from final data that have been confirmed. To enhance the usefulness of provisional data for recent periods, epidemiologists may compare these data retrospectively with confirmed data to estimate what final data for recent periods will eventually reveal (Thacker et al. 1989). A model can incorporate this consistent under-reporting to permit more accurate estimation of the final data from provisional data. In addition, when provisional data are used to examine temporal trends, current provisional data should be compared with historical provisional data rather than with final data to avoid bias.
Role of surveillance data in evaluation of community interventions
The ease with which trends in disease occurrence can be linked to interventions depends on both the disease and the intervention. The success of an immunization campaign can usually be easily inferred from surveillance data; however, such inferences become difficult when several factors contribute to a change in disease occurrence. Analyses are also difficult because of constraints such as migration and variable acceptance of interventions in the community. Programme evaluation may be improved by monitoring risk factors as well as various stages of morbidity. In addition, combining data from several communities with similar public health programmes will strengthen the assessment of programme effectiveness.
Mathematical models can be used to elucidate the complexities of evaluating community interventions (Stroup and Teutsch 1998). Such models have been used most extensively for infectious diseases. However, models for predicting the decline of mortality rates given changes in risk factors have also been developed for mortality due to cirrhosis using population changes in levels of consumption of alcohol (Skog 1984), for cardiovascular disease using changes in cigarette consumption in a population (Kullback and Cornfield 1976), and for blood lead levels given changes in legislation banning lead from petrol (Annest et al. 1983).
Linkage of surveillance data to other information sources
Given the complexity of establishing new data and information systems, there is increasing interest in the combination of existing databases for surveillance purposes. Linkage of datasets has facilitated calculation of rates, such as birthweight-specific death rates that can be calculated following linkage of birth and death certificates (McCarthy et al. 1980). In some countries, a unique number may be assigned to an individual at birth to serve as a reference number for any contact with health-care services (Paterson 1988). In other countries, a number may be assigned only for use at a single health-care facility or hospital. Techniques involved in data linkage are often complex and are based on matching records by comparison of key data fields (Newcombe 1988). When record systems are linked, the probability that the record linkage is correct must be determined, with the degree of certainty of a correct linkage depending on the comparisons of the individual identifiers such as name or initials, date or year of birth, sex, and race/ethnicity, and address. These methods of probabilistic record linkage are more likely than phonetic coding systems to identify people already reported though they often require a decision to be made on the part of the system operator as to whether the reported matches are valid. If all identifiers match exactly, the degree of assurance that the linkage is correct is high. The existence of few similarities argues against a correct linkage. Any linked set will normally contain a small number of pairs that should not have been linked and, conversely, will have missed a few pairs that should have been linked. An advantage of probabilistic data matching is that records may frequently be linked even when parts of the identifying information are either incomplete, miscoded, or mispelled.
Dissemination of data
Communication of surveillance data is an essential step in the surveillance chain. The purpose of the communication and the audience targeted must be defined. Appropriate feedback must be given to those providing the data to demonstrate their usefulness and to stimulate further reporting. People providing the data should be credited for their contributions and acknowledged for their provision of accurate and complete data. Public health professionals, policy-makers, or others who may be responsible for taking action or setting the direction of public health programmes in response to surveillance data must receive the information that they need from the surveillance system on a timely basis and in an appropriate format for their use. Web sites are increasingly used by agencies collecting surveillance data as with the CDC’s Morbidity and Mortality Weekly Report (http://www.cdc.gov/MMWR) and the WHO’s Weekly Epidemiological Record (http://www.who.int/wer).
The data must be provided on a regular basis, with the frequency of surveillance reports dependent on the nature of the surveillance system, the characteristics of the disease process (for example, surveillance reports on measles are required more frequently than reports on cancer), and the public health impact of the disease. For diseases and other health events requiring major policy decisions (for example removal of lead paint from older homes), it may be useful to provide frequent updates to remind policy-makers of the potential for prevention. In general, reports need not be issued at more frequent intervals than the data are collected from reporting sites. Provisional data should be accepted for dissemination, since rapid turn-around of data is usually more important than absolute accuracy and completeness; rarely have provisional data driven major public health decisions in directions different from those that would have been based on final data.
The format for dissemination varies with the target audience, but in any case, the design of the communications should be as creative as possible without losing essential information. A creative design will help to make the information stand apart from other documents and receive greater attention. Most policy-makers and clinicians would prefer to see the data interpreted using graphics accompanied by an abbreviated summary text; in particular, key decision-makers need simple easily understood graphics. The important role that graphs can play in visually decoding large quantities of data has been clearly demonstrated, with graphic displays giving the reader an understanding of large and complex datasets not conveyed easily in other ways (Tukey 1977; Tufte 1983; Pommerenke et al. 1994; Cleveland 1985; Dean et al. 1998). Computer graphics, in particular, have made the results of data analysis far more useful to private and public policy-makers in their planning and management of health-care resources. However, many epidemiologists and other scientists, including mathematicians projecting the future course of diseases, find the more detailed raw data in tabular format or on electronic media most useful. Comparison with previous years or previous periods (for example, experience of the last 12 months compared with experience of the previous 12 months) is often helpful.
Maps are useful in providing rapid insight into the geographical occurrence of diseases, and there is strong interest in computer mapping and graphic displays. Mapping both absolute counts of disease occurrence and rates of disease for more common conditions may be considered, particularly when geographical areas are quite variable in their population density.
Evaluation of surveillance systems
Surveillance systems should be periodically evaluated to ensure that important public health problems are under surveillance and that useful information for disease prevention and control is collected. An evaluation of a surveillance system should include a review of its objectives, a detailed description of its operation, an assessment of its performance, and recommendations (CDC 1988; Teutsch and Churchill 1994).
The performance of surveillance systems can be judged by using a series of attributes, including sensitivity, timeliness, representativeness, positive predictive value, acceptability, flexibility, simplicity, and costs. The importance of individual attributes will vary among systems, and efforts to improve on a system’s performance on one attribute may compete with efforts to improve its performance on another. Thus the evaluation of surveillance systems should not focus solely on the extent to which each attribute is achieved but rather on the attainment of the appropriate balance of attributes. The ultimate impact of improvements in surveillance should be assessed in terms of improvements in health (Thacker et al. 1986).
The sensitivity of a surveillance system can be judged by its completeness of case reporting. If all people with the condition under surveillance in the target population are detected by a surveillance system, then its sensitivity is 100 per cent. Sensitivity of surveillance systems can be measured by comparing routinely collected case reports with data obtained by special case-finding methods. For example, the sensitivity of AIDS surveillance has been assessed through detailed review of death certificates and various hospital records, such as laboratory data, patient log-books, and computerized discharge diagnoses (Chamberland et al. 1985; Rosenblum et al. 1992).
Timeliness refers to the entire surveillance cycle, ranging from how quickly cases are reported to the distribution of surveillance reports. The assessment of timeliness is dependent on the condition (for example, the assesment of timeliness would be quite different for meningitis than for cancer). Electronic communication has the capacity to improve timeliness significantly.
Representativeness is a measure of how well reported cases in a population reflect all cases that actually occurred in the population. This comparison often requires independent surveillance, which ascertains as many cases as possible in the population for a given time period (Klaucke 1994). Surveillance reporting is rarely complete, and cases that are reported may differ from unreported cases in terms of demographic characteristics, site or use of health-care services, or risk exposures (Alter et al. 1987). Representativeness of surveillance data is also affected by the quality of descriptive data that accompany case reports. Incomplete or incorrect data on surveillance forms limit representativeness.
Positive predictive value
People with reported cases of disease may not actually have the disease in question. This may reflect incorrect diagnoses (false positives), a lack of specificity in the case definition, or errors in the interpretation of the case definition. If all people reported as cases had the disease in question, then the positive predictive value would be 100 per cent. Positive predictive value depends both on the specificity of diagnostic tests and the case definition and on the prevalence of the condition under surveillance. Evaluation of the positive predictive value is difficult and requires a careful review of cases detected through routine methods. For example, hospital-based stroke surveillance based on readily available admission diagnoses was found to include a substantial proportion of people without cerebrovascular disease when more stringent diagnostic criteria were applied (Barker et al. 1984).
The concept of positive predictive value can be extended to the detection of epidemics in a surveillance system. If change in disease occurrence is used as an indicator to trigger investigations, then a high frequency of ‘false alarms’ would indicate a low predictive value for epidemic detection.
Surveillance systems depend on the co-operation of many people over a long period. If procedures are easy to follow and useful information is returned to participants, then acceptability is likely to remain high. Other factors, including protection of confidentiality of individual cases, may also be critical to the acceptability of surveillance systems.
The circumstances under which surveillance systems operate are subject to change, ranging from logistical constraints to information needs; surveillance systems should have sufficient flexibility to accommodate these changes. For example, surveillance for AIDS has been ongoing during a period of rapid evolution in the understanding of the disease, during the introduction of a diagnostic test for HIV infection, and during changing diagnostic and treatment practices as a once rare disease has become more common. Surveillance for AIDS has been flexible as demonstrated by revisions to the case definition in the 1980s and early 1990s (CDC 1992c).
Simplicity is desirable throughout the entire cycle in surveillance systems and is closely tied to other attributes such as acceptability, flexibility, and costs.
Surveillance systems incur costs in time, equipment, and supplies, which may be difficult to judge relative to their public health value. Evaluation of the costs and benefits of aggressive versus less aggressive case-finding methods in surveillance of selected notifiable diseases has yielded different conclusions that vary according to specific local circumstances (Hinds et al. 1985; Vogt et al. 1986). A description of the time requirements and costs of a surveillance system is useful for its evaluation (Hinds et al. 1985; Vogt et al. 1986; Stroobant et al. 1988; Morris et al. 1996).
The evaluation of a surveillance system should conclude with an assessment of its structure and usefulness, considering its mix of attributes in relation to its objectives. Recommendations should state whether the system should be continued and what specific changes, if any, should be made.
Public health surveillance has historically galvanized prevention and control programmes ranging from smallpox eradication and immunization campaigns for childhood diseases to programmes to prevent HIV infection and AIDS. Surveillance has also taken on increased visibility and importance in evaluating and directing prevention and control efforts for non-infectious diseases and conditions. Surveillance systems represent information loops, with data flowing from local to central agencies and back. Surveillance provides a stimulus to keep prevention and control activities moving rapidly and in the right direction, guiding the response to individual cases as well as public policy.
Effective public health interventions depend upon a continuing and reliable source of information. The data must be timely and representative of the population; they must be analysed and interpreted with feedback to the reporters and dissemination to those formulating and implementing public health policy. Resources necessary for the maintenance of the surveillance systems and for their regular evaluation should be allocated, balancing needs for data to direct prevention activities with needs for resources to implement those activities.
Adams, O.B. and Hirschfeld, M. (1998). Human resources for health. World Health Statistics Quarterly, 51, 28–32.
Alter, M.J., Mares, A., Hadler, S.C., and Maynard, J.E. (1987). The effect of underreporting on the apparent incidence and epidemiology of acute viral hepatitis. American Journal of Epidemiology, 125, 133–9.
Andrus, J.K., de Quadros, C., Oliver, J.M., and Hull, H.F. (1992). Screening of cases of acute flaccid paralysis for poliomyelitis eradication: ways to improve specificity. Bulletin of the World Health Organization, 70, 591–6.
Annest, J.L., Pirkle, J.L., Makuc, D., et al. (1983). Chronological trend in blood lead levels between 1976 and 1980. New England Journal of Medicine, 308, 1373–7.
Anselin, L. (1998). Exploratory spatial data analysis in a geocomputational environment. In Geocomputation, a primer (ed. P. Longley, S. Brooks, R. McDonnell, and M. MacMillan), pp. 77–94. Wiley, New York.
Arnett, D.K., Sprafka J.M., McGovern P.G., et al. (1998) Trends in cigarette smoking: the Minnesota Heart Survey, 1980 through 1992.American Journal of Public Health, 88, 1230–3.
Barinaga, M. (1993). Satellite data rocket disease control efforts into orbit. Science, 261, 31–2.
Barker, W.H., Feldt, K.S., and Feibel, J. (1984). Assessment of hospital admission surveillance of stroke in a metropolitan community. Journal of Chronic Diseases, 37, 609–15.
Bean, N.H., Martin, S.M., and Bradford, H. Jr (1992). PHLIS: an electronic system for reporting public health data from remote sites. American Journal of Public Health, 82, 1273–6.
Berkelman, R.L. (1994). Emerging infectious diseases in the United States, 1993. Journal of Infectious Diseases, 170, 272–7.
Berkelman, R.L. and Buehler, J.W. (1990). Public health surveillance of noninfectious chronic diseases: the potential to detect rapid change in disease burden. International Journal of Epidemiology, 19, 628–35.
Berkelman, R.L., Herndon, J.L., Callaway, J.L., et al. (1985). A surveillance system for alcohol- and drug-related fatal injuries. American Journal of Preventive Medicine, 1, 21–8.
Berkelman, R.L., Bryan, R.T., Osterholm, M.T., LeDuc, J.W., and Hughes, J.M. (1994). Infectious disease surveillance: a crumbling foundation. Science, 264, 368–70.
Besag, J. and Newell, J. (1991). The detection of clusters in rare diseases. Journal of the Royal Statistical Society, 154, 143–55.
Biellik, R.J., Bueno, H., Olive, J.-M., et al. (1992). Poliomyelitis case confirmation: characteristics for use by national eradication programmes. Bulletin of the World Health Organization, 70, 79–84.
Biomedware (1994). Statistical sortware for the clustering of health events. Biomedware, Ann Arbor, MI.
Blot, W.J. and Fraumeni, J.F., Jr (1987). Trends in oesophageal cancer mortality among US blacks and whites. American Journal of Public Health, 77, 296–8.
Boria, W., Berke, R., Clark, M., and Reisenweber, G.M. (1999). Public notification to families with newborns at risk of methemoglobinemia from drinking water exposure, Clymer, New York, 1996–1998. Journal of Public Health Management and Practice, 5, 37–8.
Box, G. and Jenkins, G. (1976). Time series analysis: forecasting and control. Holden-Day, Oakland, CA.
Brachman, P.S. (1982). Surveillance. In Bacterial infections of humans (ed. A.S. Evans and H.H. Feldman), pp. 49–61. Plenum Medical, New York.
Brachott, D. and Mosley, J.W. (1972). Viral hepatitis in Israel: the effect of canvassing physicians on notifications and the apparent epidemiological pattern. Bulletin of the World Health Organization, 46, 457–64.
Buehler, J.W., Stroup, D.F., Klaucke, D.N., et al. (1989). The reporting of race and ethnicity in the National Notifiable Diseases Surveillance System. Public Health Reports, 104, 457–65.
Buehler, J.W., Berkelman, R.L., and Stehr-Green, J.K. (1992). The completeness of AIDS surveillance. Journal of Acquired Immune Deficiency Syndrome, 5, 257–64.
Calle, E.E. and Khoury, M.J. (1991). Completeness of the discharge diagnoses as a measure of birth defects recorded in the hospital birth record. American Journal of Epidemiology, 134, 69–77.
Caper, P. (1987). The epidemiologic surveillance of medical care. American Journal of Public Health, 77, 669–70.
Cardenas, V., Sanchez, C., De la Hoz, F., et al. (1998) Colombia Field Epidemiology Training Program. American Journal of Public Health, 88, 1404–5.
CDC (Centers for Disease Control) (1988). Guidelines for evaluating surveillance systems. Morbidity and Mortality Weekly Report,37 (S-5).
CDC (Centers for Disease Control) (1990). Guidelines for investigating clusters of health events. Morbidity and Mortality Weekly Report, 39 (RR-11).
CDC (Centers for Disease Control) (1991). National electronic telecommunications system for surveillance—United States, 1990–1991.Morbidity and Mortality Weekly Report, 40, 502–3.
CDC (Centers for Disease Control) (1992a). Meeting the challenge of multidrug-resistant tuberculosis: summary of a conference. Morbidity and Mortality Weekly Report, 41 (RR-11), 49–57.
CDC (Centers for Disease Control) (1992b). A framework for assessing the effectiveness of disease and injury prevention. Morbidity and Mortality Weekly Report, 41 (RR-3).
CDC (Centers for Disease Control) (1992c). 1993 revised classification system for HIV infection and expanded surveillance case definition for AIDS among adolescents and adults. Morbidity and Mortality Weekly Report,41 (RR-17).
CDC (Centers for Disease Control) (1993). Surveillance for and comparison of birth defect prevalences in two geographic areas—United States, 1983–88. CDC Surveillance Summaries, 19 March 1993. Morbidity and Mortality Weekly Report, 42 (SS-1).
CDC (Centers for Disease Control) (1996).Atlas of United States Mortality. DHHS publication number 97-1015, US Department of Health and Human Services, Hyattsville, MD.
CDC (Centers for Disease Control) (1997). Case definitions for infectious conditions under public health surveillance. Morbidity and Mortality Weekly Report, 46 (RR-10).
CDC (Centers for Disease Control and Prevention) (1999a). Proceedings of the international effort on automating mortality statistics. US Department of Health and Human Services, Public Health Service, Washington, DC.
CDC (Centers for Disease Control and Prevention) (1999b). Progress toward global poliomyelitis eradication—1997–1998. Morbidity and Mortality Weekly Report, 48, 416–21.
CDC (Centers for Disease Control and Prevention) (1999c). Update: influenza activity—United States and worldwide, 1998–1999 season, and composition of the 1999–2000 influenza vaccine.Morbidity and Mortality Weekly Report, 48, 374–8.
Chamberland, M.E., Allen, J.R., Monroe, J.M., et al. (1985). Acquired immunodeficiency syndrome in New York City: evaluation of an active surveillance system. Journal of the American Medical Association, 254, 383–7.
Chandramohan, D., Maude, G.H., Rodrigues, L.D., and Hayes, R.J. (1998). Verbal autopsies for adult deaths:their development and validation in a multi-centre study. Tropical Medicine and International Health, 3, 436–46.
Chandra-Sekar, C. and Deming, W.E. (1949). On a method for estimating birth and death rates and the extent of registration. Journal of the American Statistical Association, 44, 101–15.
Cheng, C.M. (1992). Disease surveillance in China. In Proceedings of the 1992 International Symposium on Public Health Surveillance (ed. S. E Wetterhall). Morbidity and Mortality Weekly Report, 41 (Supplement), 111–22.
Childs, J.E., McLafferty, S.L., Ramses, A., et al. (1998). Epidemiology of rodent bites and prediction of rat infestation in New York City.Amercan Journal of Epidemiology, 148, 78–87.
Choi, K. and Thacker, S.B. (1981). An evaluation of influenza mortality surveillance, 1962–1979. I. Time series forecasts of expected pneumonia and influenza deaths. American Journal of Epidemiology, 113, 215–26.
Chute, C.G., Cohn S.P., Campbell, J.R., et al. (1998) A framework for comprehensive health terminology systems in the United States: development guidelines, criteria for selection and public policy implications. Journal of the American Medical Informatics Association,5, 503–10.
Cleveland, W.S. (1985). The elements of graphing data. Bell Telephone Laboratories, Murray Hill, NJ.
P>Cliff, A.D. and Ord, J.K. (1981). Spatial processes, models, and application. Pion, London.
Cliff, A.D., Haggett, P, and Stroup, D.F. (1992). The geographical structure of measles epidemics in the northeastern United States. American Journal of Epidemiology, 136, 592–602.
College of American Pathologists (1999). SNOMED RT and READ Codes to be combined in an international terminology of health. Joint development agreement by the College of American Pathologists and United Kingdom’s Secretary of State for Health. Press release, March 1999 (available at http://www.cap.org/html/public/snomedintl.html).
Committee on National Statistics, Commission on Behavioral and Social Sciences and Education, National Research Council, and Social Science Research Council (1993). Private lives and public policies: confidentiality and accessibility of government statistics. National Academy Press, Washington, DC.
Cormack, R.M. (1963). The statistics of capture–recapture. Ocean Marine Biology Annual Review, 6, 455–506.
Davis, J.P. and Vergeront, J.M. (1982). The effect of publicity on the reporting of toxic-shock syndrome in Wisconsin. Journal of Infectious Diseases, 145, 449–57.
Dean, A.G. (1999). Epi Info and Epi Map: Current status and plans for Epi Info 2000. Journal of Public Health Management and Practice, 5, 54–7.
Dean, A.G., Dean, J.A., Burton, A.H.,and Dicker, R.C. (1991). Epi Info: a general purpose microcomputer program for public health information systems. American Journal of Preventive Medicine, 7, 178–82.
Dean, A.G., Dean, J.A., Burton, J.H., et al. (1993). Epi Info, Version 6. A word processing, database and statistics program for epidemiology on microcomputers (computer program). Centers for Disease Control and Prevention, Atlanta, GA.
Dean, A.G., Shah, S.P., and Churchill, J.E. (1998). DoEpi. Computer-assisted instruction in epidemiology and computing and a framework for creating new exercises. American Journal of Preventive Medicine, 14, 367–71.
De Bock, A. (1988). Surveillance for accidents at work. In Surveillance in health and disease (ed. W.J. EyIenbosch and D. Noah), pp. 191–201. Oxford University Press, New York.
DeCock, K.M., Odehouri, K., Moreau, J., et al. (1989). Rapid emergence of AIDS in Abidjan, Ivory Coast. Lancet, 334, 408–11.
Devine, O. and Parrish, R.G. (1998) Monitoring the health of a population. In Statistics in public health (ed. D.S. Stroup and S.M. Teutsch), pp. 59–91. Oxford University Press, New York.
Doll, R. (1974). Surveillance and monitoring. International Journal of Epidemiology, 3, 305–14.
Edmonds, L.D. (1997). Birth defects surveillance at the state and local level.Teratology,56, 5–7.
Edmonds, L.D., Layde, P.M., James, L.M., et al. (1981). Congenital malformations surveillance: two American systems. International Journal of Epidemiology, 10, 247–52.
Erickson, J.D. (1992). Teratology data from the Atlanta birth defects case control study. Teratology, 43, 41–51.
Erickson, J.D. (1997). Introduction: birth defects surveillance in the United States.Teratology, 56, 1–4.
Executive Committee of the Infectious Diseases Society of America Emerging Infections Network (1997). Emerging Infections Network: a new venture for the Infectious Diseases Society of America. Clinical Infectious Diseases, 25, 34–6.
Eylenbosch, W.J. and Noah, N.D (ed.) (1988).Surveillance in health and disease. Oxford University Press.
Faich, G.A., Knapp, D., Dreis, M., and Turner, W. (1987). National adverse drug reaction surveillance: 1985. Journal of the American Medical Association, 257, 2068–70.
Fantahun, M. (1998) Patterns of childhood mortality in three districts of north Gondar Administrative Zone. A community-based study using the verbal autopsy method. Ethiopian Medical Journal, 36, 71–81.
Federal Committee on Statistics (1994). Private lives and public policies. US Department of Health and Human Services, Washington, DC.
Foege, W.H., Millar, J.D., and Henderson, D.A. (1975). Smallpox eradication in West and Central Africa. Bulletin of the World Heath Organization, 52, 209–22.
Fraser, P, Beral, V., and Chilvers, C. (1978). Monitoring disease in England and Wales: methods applicable to routine data-collecting systems. Journal of Epidemiology and Community Health, 32, 294–302.
Frerichs, R.R. (1991). Epidemiologic surveillance in developing countries. Annual Reviews in Public Health, 12, 80–257.
Freund, E., Seligman, P.J., Chorba, T.L., et al. (1989). Mandatory reporting of occupational diseases by clinicians. Journal of the American Medical Association, 262, 3041–4.
Gail, M.H. and Brookmeyer, R. (1988). Methods for projecting course of acquired immunodeficiency syndrome epidemic. Journal of the National Cancer Institute, 80, 900–11.
Gallus, G., Mandelli, C., Marchi, M., and Radaelli, G. (1986). On surveillance methods for congenital malformations. Statistics in Medicine, 5, 565–71.
Gaynes, R.P., Culver, D.H., Emori, T.G., et al. (1991). The national nosocomial infections surveillance system: plans for the 1990s and beyond. American Journal of Medicine, 91, 116S–20S.
Glass, G.E., Schwartz, B.S., Morgan, M.M., III, Johnson, D.T., Noy, P.M., and Israel, P. (1995). Environmental risk factors for Lyme disease identified with geographic information systems. American Journal of Public Health, 88, 1019–21.
Gostin, L.O. and Hadley, J. (1998). Health service research: public benefits, personal privacy, and proprietary interests. Annals of Internal Medicine, 129, 833–5.
Hahn, R.A. and Stroup, D.F. (1994). Race and ethnicity in public health surveillance: criteria for the scientific use of social categories. Public Health Reports, 109, 7–15.
Henderson, D.A. (1976). Surveillance of smallpox. International Journal of Epidemiology, 5, 19–28.
Henderson, D.A. (1998) Eradication: lessons learned from the past. Bulletin of the World Health Organization,76 (Supplement 2), 17–21.
Henderson, D.A. (1999). The looming threat of bioterrorism. Science, 283, 1279–82.
Heymann, D.L. and Rodier, G.R. (1998) Global surveillance of communicable diseases. Emerging Infectious Diseases, 4, 362–5.
Hinds, M.W., Skaggs, J.W., and Bergeisen, G.H. (1985). Benefit cost analysis of active surveillance of primary care physicians for hepatitis A.American Journal of Public Health, 75, 176–7.
Hinman, A.R. (1977). Analysis, interpretation, use and dissemination of surveillance information. Pan American Health Organization Bulletin, 11, 338–43.
Hinman, A.R. and Hopkins, D.R. (1998). Lessons from previous eradication programs. In The eradication of infectious diseases (ed. W.R. Dowdle and D.R. Hopkins), pp 19–32. Wiley, New York.
Holmberg, S.D., Osterholm, M.T., Senger, K.A., and Cohen, M.L. (1984). Drug-resistant Salmonella from animals fed antimicrobials. New England Journal of Medicine, 311, 617–22.
Holtzman, N.A. and Khoury, M.J. (1986). Monitoring for congenital malformations. Annual Review of Public Health, 7, 237–66.
Hutwagner, L.C., Maloney, E.K., Bean, N.H., Slutsker, L.K., and Martin, S.M. (1997) Using laboratory-based surveillance data for prevention: an algorithm for detecting Salmonella outbreaks. Emerging Infectious Diseases, 3 (http://www.cdc.gov/ncidod/eid/vol3no3/hutwagnr.htm).
Institute of Medicine (1988). Approaches to modeling disease spread and impact. Report of a workshop on mathematical modeling of the spread of human immunodeficiency virus and the demographic impact of acquired immune deficiency syndrome, 15–17 October 1987. National Academy Press, Washington, DC.
Institute of Medicine (1992). Emerging infections: microbial threats to health in the United States. National Academy Press, Washington, DC.
International Clearinghouse for Birth Defects Monitoring Systems (1998). Annual report 1998. International Centre for Birth Defects, Rome.
Jaffe, H.W., Choi, K., Thomas, P.A., et al. (1983). National case control study of Kaposi’s sarcoma and Pneumocystis carinii pneumonia in homosexual men: epidemiologic results. Annals of Internal Medicine, 99, 293–8.
John, T.J., Samuel, R., Balraj, V., and John, R. (1998). Disease surveillance at district level: a model for developing countries. Lancet, 352, 58–61.
Jones, T.S., Liang, A.P., Kilbourne, E.M., et al. (1982). Morbidity and mortality associated with the July 1980 heat wave in St Louis and Kansas City, Missouri. Journal of the American Medical Association, 247, 3327–31.
Kallen, B., Hay, S., and Klinberg, M. (1984). Birth defects monitoring systems: accomplishments and goals. In Issues and reviews in teratology (ed. H. Kalter), Volume 2, pp. 1–22. Plenum, New York.
Kaufman, J.S., Asuzu, M.C., Rotimi C.N., Johnson, O.O., Owoaje, E.E., and Cooper, R.S. (1997). The absence of adult mortality data for sub-Saharan Africa: a practical solution. Bulletin of the World Health Organization, 75, 389–95.
Kaunitz, A.M., Rochat, R.W., Hughes, J.M., et al. (1984). Maternal mortality surveillance, 1974–1978. Centers for Disease Control Surveillance Summaries, 33, 5SS–8SS.
Klauber, M..R. (1975). Space–time clustering tests for more than two samples. Biometrics, 31, 719–26.
Klaucke, D.N. (1994). Evaluating public health surveillance. In Principles and practice of public health surveillance (ed. S.M. Teutsch and R.E. Churchill), pp. 158–74. Oxford University Press.
Koo, D. and Birkhead, G.S. (1998). Prospects and challenges in implementing firearm-related injury surveillance in the United States. Not a flash in the pan. American Journal of Preventive Medicine, 15 (Supplement 3), 120–4.
Koo, D. and Wetterhall, S.F. (1996). Historical and current status of the National Notifiable Diseases Surveillance System. Journal of Public Health Managment and Practice,2, 4–10.
Kullback, S. and Cornfield, J. (1976). An information theoretic contingency table analysis of the Dorn study of smoking and mortality. Computers and Biomedical Research, 9, 409–37.
Langmuir, A.D. (1963). The surveillance of communicable diseases of national importance. New England Journal of Medicine, 268, 182–92.
Langmuir, A.D. (1976). William Farr: founder of modern concepts of surveillance. International Journal of Epidemiology, 5, 13–18.
Lechat, M.F. and Dolk, H. (1993). Registries of congenital anomalies: EuroCAT. Environmental Health Perspectives, 101 (Supplement 2), 153–7.
Lobet, M.P., Stroobant, A., Mertens, R., et al. (1987). Tool for validation of the network of sentinel general practitioners in the Belgian health care system. International Journal of Epidemiology, 16, 612–18.
Lui, K.J. and Kendal, A.T. (1987). Impact of influenza epidemics on mortality in the United States from October 1972 to May 1985. American Journal of Public Health, 77, 712–16.
Lynberg, M.C. and Edmonds, L.D. (1992). Surveillance of birth defects. In Public health surveillance (ed. W. Halperin, E.L. Baker, and R.R. Monson), pp. 157–77. Van Nostrand Reinhold, New York.
McBean A.M. and Gornick, M. (1994). Differences by race in the rates of procedures performed in hospitals for Medicare beneficiaries.Health Care Financing Review, 15, 77–90.
McCarthy, B.J., Terry, J., Rochat, R.W., et al. (1980). The under-registration of neonatal deaths: Georgia, 1974–1977. American Journal of Public Health, 70, 977–82.
McQueen, D.V. (1999) A world behaving badly: the global challenge for behavioral surveillance. American Journal of Public Health, 89, 1312–14.
Morris, G., Snider, D., and Katz, M. (1996) Integrating public health informatics and surveillance systems in the United States. American Journal of Epidemiology, 2, 24–7.
Mulinare, J., Cordero, J.F, Erickson, J.D., and Berry, R.J. (1988). Periconceptional use of multivitamins and the occurrence of neural tube defects. Journal of the American Medical Association, 260, 3141–5.
Murray, C.J.L. and Lopez, A.D. (1997). Global burden of disease study.Lancet, 349, 1269–76, 1347–52, 1436–42, 1498–504.
Music, S.I. and Schultz, M.G. (1990). Field epidemiology and training programs: new international health resources. Journal of the American Medical Association,263, 3309–11.
Newcombe, H.B. (1988). Handbook of record linkage. Oxford University Press.
Newhouse, V.F., Choi, K., D’Angelo, L.J., et al. (1986). Analysis of social and environmental factors affecting the occurrence of Rocky Mountain spotted fever in Georgia, 1961–75.Public Health Reports, 101, 419–28.
Nobre, F.F. and Stroup, D.F. (1994). A monitoring system to detect pattern changes in public health surveillance data. International Journal of Epidemiology, 23, 408–18.
Orenstein, W.A., Bart, S.W., Bart, K.J., et al. (1996). Epidemiology of rubella and its complications. In Vaccinating against brain syndromes: the campaign against measles and rubella (ed. E.M. Grunberg, C. Louis, and S.E. Goldson), pp. 49–69. Oxford University Press, New York.
PAHO (Pan American Health Organization) (1994). Dengue fever in Costa Rica and Panama. Epidemiological Bulletin, 15, 9–10.
Parkin, D.M. (ed.) (1986). Cancer occurrence in developing countries. International Agency for Research on Cancer, Lyon.
Parkin, D. (1988). Surveillance of cancer. In Surveillance in health and disease (ed. W.J. Eylenbosch and N.D. Noah), pp. 143–65. Oxford University Press.
Parrish, R.G., Tuckcr, M., Ing, R., and Encarnacion, C. (1987). Sudden unexplained death syndrome in southeast Asian refugees: a review of CDC surveillance. Centers for Disease Control Surveillance Summaries,36, 43–53.
Paterson, J.G. (1988). Surveillance systems from hospital data. In Surveillance in health and disease (ed. W.J. Eylenbosch and N.D. Noah), pp. 49–61. Oxford University Press.
Percy, C. and Dolman, A. (1978). Comparison of the coding of death certificates related to cancer in seven countries. Public Health Reports, 93, 335–50.
Pollack, E.S. and Ringen, K. (1992). Risk of hospitalization for specific nonwork-related conditions among laborers and their families. American Journal of Industrial Medicine, 23, 417–25.
Pommerenke, F.A., Miller, R.W., Srivastava, S., and Ackermann, S.P. (1994). Targeting cancer control: the state cancer control map and data program. American Journal of Public Health, 84, 1479–82.
Quigley, M.A., Armstrong Schellenberg, J.R., and Snow, R.W. 1996. Algorithms for verbal autopsies: a validation study in Kenyan children. Bulletin of the World Health Organization, 74, 147–54.
Raska, K. (1966). National and international surveillance of communicable diseases. World Health Organization Chronicle, 20, 315–21.
Raymond, L. (1997). Techniques of registration. In Cancer incidence in five continents (ed. D.M. Parkin, S.L. Whelan, J. Ferlay, and J. Young), Volume 7. IARC, Lyon.
Remington, P.L.S., Smith, M.Y., Williamson, D.F., et al. (1988). Design, characteristics and usefulness of state-based behavioral risk factor surveillance: 1981–1987.Public Health Reports, 103, 366–75.
Richards, T.B., Henriques, W.D., Croner, C.M., Brown, C.K., Saccenti, J.C., and Berry, P.J. (1999). Toward a GIS sampling frame for surveys of local health departments and local boards of health. Public Health Management and Practice, 5, 65–75.
Rosenblum, L., Buehler, J.W., Morgan, M.W., et al. (1992). Completeness of AIDS case reporting, 1988: a multisite collaborative surveillance project.American Journal of Public Health, 82, 1495–9.
Rousch, S., Birkhead, G., Koo, D., Cobb, A., and Fleming, D. (1999). Mandatory reporting of diseases and conditions by health care professionals and laboratories. Journal of the American Medical Association,282, 164–70.
Sandiford, P., Annett, H., and Cibulskis, R. (1992). What can information systems do for primary health care? An international perspective. Social Science and Medicine, 34, 1077–87.
Sattin, R.W., Rubin, G.L., and Hughes, J.M. (1983). Hysterectomy among women of reproductive age, United States, update for 1979–1980. Centers for Disease Control Surveillance Summaries, 32, 1SS–7SS.
Seligman, P.J., Halperin, W.E., Mullan, R.J., and Frazier, T.M. (1986). Occupational lead poisoning in Ohio: surveillance using worker’s compensation data.American Journal of Public Health, 76, 1299–1302.
Serfling, R.E. (1963). Methods for current statistical analysis of excess pneumonia-influenza deaths. Public Health Reports, 78, 494–506.
Shands, K.N., Schmid, G.P., Dan, B.B., et al. (1980). Toxic-shock syndrome in menstruating women: association with tampon use and Staphylococcus aureus and clinical features in 52 cases. New England Journal of Medicine, 303, 1430–42.
Shen, F.Z., Shou, P.J., and Zhao, G.F. (1984). Influenza surveillance in Shanghai. Chinese Medical Journal, 97, 339–4.
Siegel, P.Z., Brackbill, R.M., Frazier, E.L., et al. (1991). Behavioral risk factor surveillance, 1986–1990. CDC Surveillance Summaries, Morbidity and Mortality Weekly Report, 40 (SS-4), 1–23.
Singh, A. and Roberts, G. (1992). State space modeling of cross-classified time series of counts.International Statistical Review, 60, 321–35.
Skog, O. (1984). The risk function for liver cirrhosis from lifetime alcohol consumption.Journal of Studies on Alcohol,45, 199–208.
Stelling, J.M. and O’Brien, T.F. (1997) Surveillance of antimicrobial resistance: The WHONET program.Clinical Infectious Diseases, 24 (Supplement 1), S157–68.
Stroobant, A.W., Van Casteren, V., and Thiers, G. (1988). Surveillance systems from primary care data: surveillance through a network of sentinel general practitioners. In Surveillance in health and disease (ed. W.J. Eylenbosch and N.D. Noah), pp. 62–74. Oxford University Press.
Stroup, D.F. and Teutsch, S.M. (ed.) (1998). Statistics in public health: quantitative approaches to public health problems. Oxford University Press, New York.
Stroup, D.F. and Thacker, S.B. (1993). A Bayesian approach to the detection of aberrations in public health surveillance data. American Journal of Epidemiology, 4, 435–43.
Stroup, D.F., Thacker, S.B., and Herndon, J.L (1988). Application of multiple time series analysis to the estimation of pneumonia and influenza mortality by age: 1962–1983.Statistics in Medicine, 7, 1045–59.
Stroup, D.F, Wharton, M., Kafadar, K., and Dean, A.G. (1993). An evaluation of a method for detecting aberrations in public health surveillance data. American Journal of Epidemiology, 137, 373–80.
Swan, J., Wingo, P., Clive R., et al. (1998). Cancer surveillance in the U.S.: can we have a national system? Cancer, 83, 9.
Teutsch, S.M. and Churchill, R.E. (ed.) (1994). Principles and practice of public health surveillance. Oxford University Press, New York.
Thacker, S.B. and Berkelman, R.L. (1986). Surveillance of medical technologies. Journal of Public Health Policy,7, 363–77.
Thacker, S.B. and Berkelman, R.L. (1988). Public health surveillance in the United States. Epidemiologic Reviews,10, 164–90.
Thacker, S.B. and Stroup, D.F. (1994). Future directions of comprehensive public health surveillance and health information systems in the United States. American Journal of Epidemiology, 140, 383–97.
Thacker, S.B. and Gregg, M.B. (1996). Implementing the concepts of William Farr: the contributions of Alexander D. Langmuir to public health surveillance and communications.American Journal of Epidemiology, 144, 523–8.
Thacker, S.B., Redmon, S., Rothenberg, R.B., et al. (1986). A controlled trial of disease surveillance strategies. American Journal of Preventive Medicine, 2, 345–50.
Thacker, S.B., Berkelman, R.L., and Stroup, D.F. (1989). The science of public health surveillance. Journal of Public Policy, 10, 187–203.
Thacker, S.B., Stroup, D.F., Rothenberg, R.B., and Brownson, R.C. (1995). Public health surveillance for chronic conditions: a scientific basis for decisions. Statistics in Medicine, 14, 629–41.
Tollman, S.M., Kahn K, Garenne, M., and Gear, J.S. (1999). Reversal in mortality trends: evidence from the Agincourt field site, South Africa, 1992–1995. AIDS 13, 1091–7.
Tufte, E.R. (1983). The visual display of quantitative information. Graphics Press, Cheshire, CT.
Tukey, J.W. (1977). Exploratory data analysis. Addison-Wesley, Reading, MA.
Twigg, L. (1999). Choosing a national survey to investigate smoking behavior: making comparisons between the General Household Survey, the British Household Panel Survey and the Health Survey for England. Journal of Public Health Medicine, 21, 14–21.
Valleron, A.J., Bouvet, E., Garnerin, P., et al. (1986). Computer network for the surveillance of’ communicable diseases: the French experiment. American Journal of Public Health, 76, 1289–92.
Vogt, R.L., LaRue, D., Klaucke, D.N., and Jillson, D.A. (1983). Comparison of active and passive surveillance systems of primary care providers for hepatitis, measles, rubella and salmonellosis in Vermont. American Journal of Public Health, 73, 795–7.
Vogt, R.L., Clark, S.W., and Kappel, S. (1986). Evaluation of the state surveillance system using hospital discharge diagnoses, 1982–1983. American Journal of Epidemiology, 123, 197–8.
Wallenstein, S. (1980). A test for detection of clustering over time.American Journal of Epidemiology, 111, 367–72.
Washino, R.K. and Wood, B.L. (1994). Application of remote sensing to arthropod vector surveillance and control. American Journal of Tropical Health and Hygiene, 50 (Supplement 6), 134–44.
Weddell, J.M. (1973). Registers and registries: a review. International Journal of Epidemiology, 2, 221–8.
Wegman, D.H. (1992). Hazard surveillance. In Public health surveillance (ed. W. Halperin and E.L. Baker Jr), pp. 62–75. Van Nostrand Reinhold, New York.
Weniger, B.G., Limpakarnjanarat, K., Ungchusak, K., et al. (1991). The epidemiology of HIV infections and AIDS in Thailand. AIDS, 5 (Supplement 2), 271–85.
WHO (World Health Organization) (1958). The first ten years of the World Health Organization. WHO, Geneva.
WHO (World Health Organization) (1968). Report of the technical discussions at the 21st World Health assembly on ‘National and global surveillance of communicable diseases’. WHO, Geneva.
Yang, G. (1992). Selection of DSP points in second stage and their representation. Chinese Journal of Epidemiology, 13, 197–201.
Yang, G.H., Stroup, D.F., and Thacker, S.B. (1997). National public health surveillance in China: implications for public health in China and the United States. Biomedical and Environmental Sciences, 10, 1013–19.
Yen, H., Khoury, M.J., Erickson, J.D., et al. (1992). The changing epidemiology of neural tube defects in the United States. American Journal of Diseases in Children, 146, 857–61.
Zeger, S. (1988). A regression model for time series of counts. Biometrika, 75, 621–9.
Zhang, K.X. (1990). Forecast of the incidence of encaphalitis B in Liao Ning province.Chinese Journal of Epidemiology, 1, 67.