5.4 Applications of information systems to public health
Oxford Textbook of Public Health
Applications of information systems to public health
Advances in information technology and information systems
Public health activities supported by information systems
Applying information systems in public health activities
Outline of this chapter
Facility-based data collection
Data collection from payers and providers
Clinical and management information systems
Hospitals and clinics
Public health programmes
Data analysis and policy development
Computerized statistical and epidemiological analyses
Geographical information systems
Linked databases and data warehousing
Data access and dissemination
Public health and medical literature
On-line query systems
Information sharing among public health professionals
Providing health data to community members
Health training and education
Specialized health information providers
Challenges to applying information systems in public health
Information technology and telecommunications infrastructure
Privacy and confidentiality protection
Information technology development challenges
Web page citations
A health information system in the broadest sense is comprised of data as well as procedures to collect, store, analyse, transfer, and retrieve that data. The data may be stored in paper and/or electronic form, and the collection, analysis, transfer, and retrieval may be performed by human beings or electronic information technology, but usually by some combination of both. Information technology consists of computers, the networks and telecommunications systems that connect them, and the software that operates the computers and networks. Dramatic advances in all aspects of information technology are driving profound changes in organizations and societies.
The ultimate purpose of a public health information system is to convert raw data into information that can be used for decision-making. One major challenge in developing information systems for public health is the great diversity of data that must be captured (demographical, clinical, geographical, administrative, financial) and the diversity of sources from which it originates (patients, health providers, laboratories, hospitals, restaurants, and so on). A second challenge is the number and diversity of users of public health information, including policy-makers, public health professionals, managers, community-based organizations, and the population at large. The emerging discipline of public health informatics addresses these challenges by the ‘systematic application of information and computer science and technology to public health practice, research, and learning’ (O’Carroll et al. 2000; Yasnoff et al. 2000) (http://www.nlm.nih.gov/pubs/cbim/phi2001.html).
Information systems are at the core of all public health activities. This chapter reviews how information systems, especially those that employ electronic information technology, are applied in public health, and how those applications are changing as information technology evolves. In addition, it describes some of the important types of information resources, such as population survey data files, that are produced by public health information systems.
Advances in information technology and information systems
Computer hardware is becoming continuously more powerful. Moore’s law is an observation that the number of transistors that can be packed onto a semiconductor chip doubles every 18 months, with corresponding increases in computer power. Since manufacturing costs do not rise nearly as fast, the price per unit of computing power continues to plummet. The capacity of disk storage devices has risen by more than 50 per cent per year for several years, yielding even more dramatic reductions in the price per unit of electronic data storage. These trends have resulted in affordable shared server computers, powerful desktop computers costing less than US$500, portable computers of equivalent power but higher cost, and a growing range of small handheld computers.
Networks and telecommunications
Computers are increasingly being connected to each other via networks: local area networks within buildings, wide-area networks within organizations with multiple locations, and the Internet worldwide. Wide-area networks and the Internet use the publicly available telecommunications infrastructure. The benefits of this increasing interconnection are summarized by Metcalfe’s law, which states that the value of a computer network grows exponentially by the number of computers connected to it, since every new computer can access, and be accessed by, every other computer on the network.
The transmission capacity of a computer network, measured in amount of data transferred per unit of time, is referred to as the network’s bandwidth. Owing to rapid construction and the development of more powerful computerized network switching devices and technologies such as fibre optics, the available bandwidth of networks is growing at a dramatic rate, with concomitant drops in price. The result is that people connected to the robust computer networks found in universities and large corporations take for granted practically instant access to information from computers around the world. Technologies such as cable modems and digital subscriber line telephone service are now making this high bandwidth and low cost available to homes.
Within a decade, most of the population of many industrialized countries will have routine low-cost access to bandwidth sufficient for real-time video and audio communication. Portable and handheld computers will be able to access networks via wireless connections, albeit at lower bandwidth. Bandwidth availability in developing countries will vary widely, both across countries and between urban and rural areas. However, the dropping price of wired, fixed wireless, and satellite telecommunications will provide more and more developing country locations with bandwidth sufficient to support important information systems applications.
Computers require elaborate specialized instructions to store, process, display, and share data. These instructions or programs are known as software. Since software is only a type of information, the marginal cost of duplicating it is nearly zero. However, the writing and testing of software is a very labour-intensive process conducted by skilled programmers. Therefore advances in individual software programs are less dramatic than the increases in hardware and network capabilities, but a new program can be distributed to millions of users in weeks at minimal cost. For example, easy-to-use browser software distributed free enabled the explosion of network and Internet use in the last half of the 1990s.
Software now exists for thousands of applications, including database creation and maintenance, quantitative analyses (e.g. scientific, statistical, epidemiological, financial), communications, management, and entertainment. New software is being written and disseminated continuously worldwide to take advantage of advances in computer hardware and telecommunications.
Public health activities supported by information systems
In order to describe applications of information systems to public health, it is necessary to choose a framework for discussing each of the diverse activities of public health. Such frameworks have been developed from several different perspectives.
The Institute of Medicine (1988) identified three major categories of activities carried out by local health departments: assessment, policy development, and assurance. More specifically, this means that a local health department monitors and assesses health problems and resources of populations within its geographical jurisdiction, promotes healthy behaviours, ensures and promotes a healthy environment, and assures preventive and curative health services. To accomplish these tasks effectively, the health department must be able to manage its own and its contractors’ activities efficiently, evaluate its numerous public health programmes, and involve the community at all stages.
Local health departments accomplish these general goals by carrying out a number of different programmes, each requiring its own organizational unit and specialized expertise. Many of these programmes are mandated by national and/or state law. For example, Title 17 of the California Code of Regulations mandates that county or city health departments offer at least the following services: vital statistics registration, health education, communicable disease control, maternal and child health promotion, environmental health and sanitation, a public health laboratory, nutrition, chronic disease prevention and mitigation, social factors (including community planning), occupational health, family planning, and public health nursing. Health departments often conduct programmes that reduce other risks to the public’s health, including tobacco control, substance abuse prevention and treatment, injury and violence prevention, and oral health promotion. Finally, health departments often also manage facilities that provide personal health services, although these may not fall under the definition of public health activities in the strictest sense.
State and national government health agencies establish broad health policies, set mandates for local health departments, aggregate and disseminate health data collected from local jurisdictions, fund some public health programmes, and carry out national population surveys. In some countries, state and national health departments operate some programmes directly.
Internationally, the World Health Organization (WHO) identifies global health priorities, carries out programmes within countries and across borders (such as malaria prevention or health systems strengthening), and collects and disseminates a wide range of country-specific health information.
Applying information systems in public health activities
Public health is a data-intensive field. Paper-based information systems can be found in public health programmes in any country. In industrialized countries, information technology has been applied in public health programmes for decades. Initially, centralized computers were used to aggregate and analyse data collected in the field. Desktop computers made it possible for increasingly sophisticated analyses to be performed on a distributed basis, and telecommunications links allowed the electronic transmission of data. Management and some clinical functions in laboratories, clinics, and hospitals were also computerized. Nevertheless, the actual collection of data in the field and in most facilities is still accomplished with paper records. Recently, many public health organizations and some health departments have begun using the Internet for information dissemination.
Other industries, including banking, transportation, and health insurers, followed a similar path, beginning with centralized automation and moving incrementally to distributed processing and networks. However, the mutually reinforcing advances in all aspects of information technology are creating huge new opportunities to collect, analyse, and share information more rapidly, cheaply, and effectively. Therefore private-sector firms have now moved on to the phase of completely re-engineering their processes to take advantage of these information technology capabilities (Hammer 1990).
While these same benefits could accrue to public health agencies in industrialized countries, they have not yet taken advantage of information technology to transform fundamentally how they accomplish their missions. The reasons include their organizational inertia as government agencies, the above-mentioned challenges of diverse data and users, and limited funds for investment in information systems upgrading. Building upon successful applications in other industries, public health agencies could use information technology to improve operations in several ways. Firstly, core public health activities, ranging from data collection in the field to transmission, analysis, and dissemination, can be fully computerized. Personal health services providers are moving in this direction, with a fully electronic medical record as the ultimate goal (Dick et al. 1997). Such automation ensures that data are shared more quickly, analysed more easily, and are also available to multiple users simultaneously. Secondly, information systems can be implemented to manage more efficiently administrative activities such as procurement, contracting, human resources, and financial management. These systems can be modelled closely on successful similar systems in other industries. Thirdly, data from information systems in different public health programmes can be aggregated and linked together in centralized databases to facilitate population-level analyses, policy development, and dissemination to a wide range of users.
In other industries, re-engineering processes to take advantage of information technology capabilities has resulted in dramatic efficiency and performance improvements, better customer service, and more effective analyses of company-wide data to formulate management strategy. In public health, the fundamental goals will not be changed by information technology, nor will the basic activities such as disease surveillance, treatment, education, facility inspection, and epidemiological analysis. However, innovative agencies will probably transform how those activities are performed. For example, geographical analyses of disease patterns have been essential to public health since John Snow’s work (http|://www.ph.ucla.edu/epi/snow.html). Geographical information systems (GIS) software that make it possible to combine and display geographically coded data very easily are just beginning to be widely available to public health professionals.
Outline of this chapter
The outline for this chapter’s discussion of information systems applications to public health is a synthesis of the frameworks for public health activities presented above. It describes how information systems support a wide range of public health activities, emphasizing current, emerging, and potential uses of information technology. In some cases, the actual data (such as surveys) are described in some detail. Most examples are from industrialized countries due to the greater penetration of information technology there, but several developing country applications are noted. The topic areas are as follows.
Data collection: vital statistics, population surveys, disease surveillance, facility-based data collection, data from providers and payers.
Clinical and management information systems: hospitals and clinics, public health programmes, telemedicine, quality-of-care measurement.
Data analysis and policy development: computerized statistical and epidemiological analyses, health indicators, GIS, linked databases, data warehousing
Data access and dissemination: public health and medical literature, downloadable datasets, on-line query systems, information sharing among public health professionals, providing health data to community members
Health training and education: distance learning, educational campaigns, curriculum development, specialized health information providers
Challenges to applying information systems in public health: information technology and communications infrastructure, funding availability, privacy and confidentiality protections, information technology development challenges.
All public health activities rely on accurate population health data, including data on events such as mortality, population health status and behaviour, communicable disease case and treatment reports, and utilization of health-care services. Each of these major types of data is usually collected by a specialized public health programme, using its own customized information systems. The types of data, the programmes that collect them, and the information systems used, are described in this section.
Natality and mortality are the main categories of vital statistics information. Each birth and death event is recorded, so that vital statistics registries are an ongoing census of these events in the population. Additional data items about each event, such as race/ethnicity of a newborn’s parents or causes of deaths, are also recorded in a standardized fashion. For example, the California death record has over 100 data items.
The collection of vital statistics data is highly computerized in industrialized countries. For example, the Automated Vital Statistics System (AVSS) is used to input birth and death data in all 58 counties in California. These records are aggregated at the state level. The National Center for Health Statistics (NCHS) compiles national-level natality and mortality data files in its National Vital Statistics System (NVSS). Summary and detail files from NVSS allow analyses of deaths by factors such as location, age group, race, gender, year, and cause of death (http://www.cdc.gov/nchs/about/major/dvs/mortdata.htm).
The American Census Bureau, as well as many state and county governments, use vital statistics data to make intercensal population projections because the Census Bureau only conducts a complete census every 10 years. Vital statistics can also be used for planning public health services. For example, the number, location, and characteristics of births can help project the need for children’s health services. Mortality data are essential for planning disease prevention programmes.
Vital statistics activities lend themselves to computerization. In California, birth records are input to AVSS directly where births occur. In developing countries, accurate and complete recording of vital events, even with paper records, is the highest priority. However, regional registries can be computerized at modest cost, allowing data to be aggregated easily on a national basis; analyses and report generation can then be computerized as well.
Although vital statistics are collected as a census, many other important types of information necessary for public and health services research analyses are collected from surveys of representative samples of the population of a nation or region. This includes information on the determinants of health, the health status and needs of communities, health services access and insurance coverage, and health behaviours.
Given the diverse types of information to be gathered, industrialized countries conduct a broad spectrum of health surveys, each designed to collect specific data from a defined population. Some of the major national health surveys in the United States are briefly described below as examples of the data collection methods employed and the resulting information resources available for public health analyses.
The National Health Interview Survey (NHIS) is conducted by the NCHS. This in-person survey, conducted annually, targets a nationally representative cluster sample of 40 000 households. In addition to underlying demographic data on each household, the NHIS collects data on health status and health services utilization, including acute and chronic conditions, activity limitations due to health problems, insurance coverage, and utilization of outpatient and inpatient health services. Specialized modules, covering topics such as cancer prevention and control, are added periodically (http://www.cdc.gov/nchs/nhis.htm).
The National Health and Nutrition Examination Survey (NHANES), also conducted by the NCHS, is a much more intensive survey of 5000 randomly sampled people annually. Respondents complete interviews and in-depth physical examinations and testing. The data collected include diet, nutrition, health behaviours, and risk factors. This combination of behavioural and biomedical data from a nationally representative population allows prevalence estimates of diseases and risk factors, as well as analyses of the links between health behaviours and health status (http://www.cdc.gov/nchs/nhanes.htm).
The National Health Care Survey (NHCS) is composed of several surveys drawing data on patient characteristics and services utilization at a wide range of health provider organizations: hospitals, ambulatory surgery facilities, ambulatory care clinics, nursing homes, and hospices. These surveys are also conducted by the NCHS and draw their data from medical records at nationally representative samples of facilities; tens of thousands of records are reviewed in each survey. Some of the surveys are annual, others periodic (http://www.cdc.gov/nchs/nhcs.htm).
The Medical Expenditure Panel Survey (MEPS) is conducted by the Agency for Healthcare Research and Quality (AHRQ). This in-person survey follows panels of households over a 2.5-year period; households are selected as a subsample of households from the NHIS. Households provide detailed data on their insurance coverage and utilization of and payment for health services, as well as personal health information, allowing these data to be linked longitudinally. The household survey is supplemented by data collection at health providers and employers to provide a more complete picture of health insurance and health expenditures (http://www.meps.ahrq.gov/).
The Behavioral Risk Factor Surveillance System (BRFSS) is a telephone survey conducted by the Centers for Disease Control (CDC). The BRFSS is a concatenation of surveys conducted in each state, comprising over 150 000 interviews annually. Data are collected on a variety of health risk and behaviours, including tobacco and alcohol use, exercise and diet, chronic disease prevention and screening, and injury prevention. Individual states can add questions tailored to their public health needs (http://www.cdc.gov/nccdphp/brfss/index.htm).
Private foundations often conduct surveys that provide data useful for public health analyses. For example, the National Survey of America’s Families (NSAF). Designed to assess the effect of policy changes such as welfare reform, the NSAF oversampled low-income households and collected a range of data on family demographics and welfare, including health. Primarily a telephone survey, the NSAF collects data from a nationally representative sample of over 44 000 households (http://newfederalism.urban.org/nsaf/).
Although comprehensive health surveys are also very useful in developing countries, resource constraints severely limit their scope and frequency. An ongoing series of Demographic and Health Surveys (DHS) has been funded primarily by the United States Agency for International Development (USAID). Dozens of surveys have been conducted in countries throughout the world. Each survey covers a nationally representative sample of about 5000 households, with data collected in person. Comparable instruments are used to gather data on health and nutrition, family planning, maternal and child health, child survival, HIV/AIDS, and reproductive health (http://www.measuredhs.com/).
For local health assessment and planning, rapid surveys with sample sizes of a few hundred can be conducted rapidly and inexpensively (Husein et al. 1993; Kipp et al. 1994; Satia et al. 1994). These advantages make rapid surveys especially attractive in developing countries, since a small trained staff and a simple microcomputer are essentially the only infrastructure needed. Rapid survey methods are described in more detail in Chapter 6.15.
Information systems are used to support all phases of survey design and execution in industrialized countries. In-person survey interviewers use portable computers, and computer-assisted self-interview software may be used to collect data on particularly sensitive topics. Telephone survey interviewers use automated questionnaires on computer-assisted telephone interview software (Aday 1996). These computerized data collection systems feed data directly into a computerized database of results. Other specialized software is used for other aspects of survey design, analysis, and management, including sample design, random digit dialling, call centre management, data cleaning and imputation, and calculation of weights (e.g. for non-response and adjustment to population proportions). Information systems for disseminating survey results via query systems and downloadable public use datasets are described later in this chapter. Copies of survey instruments and documentation can also be made available on the Internet, enabling users to analyse and interpret survey results correctly.
Telephone surveys, portable computers for field interviewers, and Internet dissemination of data may not be practical in developing countries. However, inexpensive computers can still significantly enhance efficiency when used to design the survey and manage data collection. Data from paper survey instruments can be key-punched at modest cost, allowing computerized data cleaning, analysis, and report preparation.
In future years, some surveys in industrialized countries will become even more computerized, collecting data over the Internet. Already, interactive data collection forms are straightforward to implement. However, Internet data collection will probably not supplant in-person or telephone methods, as the response rate to Internet surveys is expected to mirror the relatively low response rates of mail surveys.
Surveillance of communicable diseases is a fundamental public health activity (Drotman and Strassburg 2001). Surveillance techniques and information systems are discussed only briefly here, since they are the subject of Chapter 6.16.
In industrialized countries, laws require that several dozen communicable diseases be reported to local health departments by doctors, clinics, hospitals, and laboratories. These data are in turn reported to state/provincial health departments, and thence to national government health agencies. In developing countries with less robust health infrastructures, sentinel sites may be the focus of communicable disease reporting. These are facilities that provide services to a large enough population to capture significant numbers of patients with the target diseases and have the resources for ongoing data collection (Woodall 1988). Sampling from a network of carefully chosen locations can provide reporting representative of the population, as is done in China (Chunning 1992).
Even in industrialized countries, much of the communicable disease reporting is still accomplished by paper forms submitted to the local health department. There the data from the form are input into computerized databases, and reporting to state and national levels is often electronic. However, the initial paper form step is unreliable and can be too slow for rapid response to disease reports. For example, slow reporting hinders the identification of outbreaks of emerging infectious diseases, such as West Nile virus, thereby making responses less prompt and effective.
Large facilities with more comprehensive computerized information systems, such as laboratories and hospitals, are beginning to use electronic reporting in some jurisdictions. As more and more doctors’ offices are connected to the Internet, and capabilities for secure Internet transmission of confidential information expand, more communicable disease reporting will move to that medium.
The CDC is developing a comprehensive electronic communicable disease reporting system, the National Electronic Disease Surveillance System (NEDSS), to address the shortcomings of the current communicable disease reporting systems in the United States (http://www.cdc.gov/od/hissb/docs.htmnedss). Its components will include the following:
data standards for uniform generation, transmission, and aggregation of communicable disease reports
record matching software to eliminate duplicate reports of the same case of a communicable disease
a common user interface for all computerized CDC reporting systems
standardized Internet-based secure transmission links from health departments to the CDC
standardized data definitions to facilitate data sharing and analysis.
Facility-based data collection
Large facilities, particularly those operated or funded by government health agencies, can be required to report data on patients seen there. These data can be very useful in describing patterns of disease and treatment, as well as the underlying demographic data on patients. However, a major shortcoming of such data is that patients using the reporting facilities may not be representative of the population overall. This is a particular problem in developing countries, where access to health care is very limited and many people with diseases do not present to facilities for treatment. Nevertheless, in many countries these data are the best available (Cibulskis and Izard 1996).
The most appropriate use of facility-based data collection is the standardized reporting of procedures that are almost always performed at large facilities, or diseases almost always diagnosed or treated there. For example, in the United States, hospitals in 35 states are required to report data on inpatient discharges (Love 2000). Required data on each discharge includes length of stay, diagnosis, mortality, patient demographics, and payer. A state government department aggregates the data and removes personally identifiable data so that complete files can be made public. These data are then used for a range of management, health services research, and policy analyses.
However, inpatient discharge reporting does not capture all data on procedures because ambulatory surgeries are becoming more common, and are performed in free-standing centres as well as in hospitals. Therefore some states have now begun to require that ambulatory surgeries be reported as well, whether performed at hospital or at a centre certified by Medicare (Love 2000).
Cancer registries are another facility-based system that provides data of great value for cancer epidemiology, health services research, and environmental health analyses. For example, California requires that all diagnosed neoplasms be reported, along with treatment data and underlying patient demographics (http://www.ccrcal.org/). All hospital and facilities that treat cancer are required to file reports, as are doctors who treat cancer themselves. Reported data are aggregated at the state level, personally identifiable data are removed, and complete files made available for public use.
Facility-based data collection can be very information-technology intensive. Reported data is often compiled from clinical or management information systems at facilities, data is reported via tapes or electronic transmission of data files, central databases are large and complex, and data files are made publicly available in CD-ROM form or over the Internet.
Data collection from payers and providers
Data on disease patterns and health services utilization can be collected from payers, such as health insurers, or directly from providers when they are reimbursed by government health agencies. Payers who reimburse doctors and hospitals on a fee-for-service basis receive claims data for each outpatient encounter or inpatient stay. These data are limited in clinical and demographic detail because they are collected for billing purposes, but they can cover very large segments of the population. Data collection from providers can also be effective when a government payer requires encounter or discharge data to be submitted as a condition of receiving capitation or global budget allocations. However, since a claim for each encounter need not be submitted under a capitated payment system, ensuring that providers submit complete and timely data is a major challenge, and so the resulting datasets may not be fully complete. No matter how encounter data are collected, privacy concerns must be addressed by removing personally identifiable fields from data files.
Medicare is a near-universal health insurance programme for all American residents who are 65 years of age and older and/or disabled. It reimburses fee-for-service for over 80 per cent of its beneficiaries, and so the resulting claims data are quite comprehensive. The remaining beneficiaries are in enrolled health maintenance organizations (HMOs) which receive capitated payments; encounter data are not available for these people. The Health Care Financing Administration (HCFA) provides a 5 per cent sample of fee-for-service claims to approved users. These data are used for health services research and policy analyses (http://www.hcfa.gov/stats/pufiles.htmpufcat).
Medicaid is the American health insurance programme covering people in poverty. It is operated and financed jointly by state governments and the HCFA. Many states are attempting to reduce the programme’s cost by moving Medicaid beneficiaries to HMOs. Therefore the HCFA is now working with states, HMOs, and providers to obtain encounter data (http://www.hcfa.gov/medicaid/m2082.htm).
In the long term, electronic medical records will facilitate the capture and sharing of data by health providers. Ideally, data captured at the bedside or in a clinic can be electronically transmitted to central payer or government data repositories, and there aggregated across the entire population. However, for reasons discussed below, this vision is many years from practical realization.
Clinical and management information systems
Health providers—clinics, general practitioners, and hospitals—have historically used paper-based medical records to record clinical information. Hospitals in industrialized countries have computerized many administrative functions, as well as some clinical functions. Clinics have also automated some activities, such as appointment scheduling. These information systems are discussed in this chapter because they can contribute to public health in several ways: enhancing the efficiency and effectiveness of health facilities operated by government agencies, serving as rich sources of data on disease patterns and health services utilization, and providing data for health services quality measurement. This section also describes information systems which support the activities of public health programmes, and telemedicine systems to help deliver appropriate health-care services to isolated populations.
Hospitals and clinics
Hospitals have the most comprehensive information systems of any health services provider organizations, because they command resources and can clearly benefit from automation (Austin and Boxerman 1998). Most hospitals in the United States find it cost-effective to purchase complete information systems from commercial vendors, such as Siemens Medical Solutions Inc. or McKesson-HBOC, rather than develop them in house. These comprehensive hospital information systems automate admission, billing, purchasing, and other tasks, as well as some aspects of hospital-based clinical services, such as radiology, laboratory, and pharmacy.
Hospitals in wealthier developing countries are increasingly purchasing information systems as well. Nevertheless, purchasing and installing any such large information system is expensive and risky. The Pan-American Health Organization (PAHO) has recently published a very useful book that lays out in detail how to navigate through the process of defining requirements, selecting vendors, and implementing the information system (PAHO 1999).
Most clinics also purchase the information systems that automate functions like scheduling and claims submission. Medical records systems suitable for outpatient practice are being developed, but are only at the early stages of implementation (http://www.health-infosys-dir.com/yp_hc.htm).
The fully electronic medical record has been an elusive goal for over 20 years (Dick et al. 1997). Despite technological advances, the volume and diversity of health data have frustrated attempts to develop such systems. Incremental progress has been made, such as automating some hospital functions, providing computerized doctor order entry, and digitizing dictated reports such as hospital discharge summaries. However, doctors still resist taking the time to type, and variations in medical vocabulary make automated recognition of clinical concepts the most difficult technical challenge. However, installation of computers and networks throughout hospitals has made it possible for some institutions, such as the United States Veterans Administration, to assemble very comprehensive computerized medical records.
Use of the Internet to streamline transactions between provider organizations is increasing, and could significantly improve the efficiency and timeliness of health provider operations. Electronic data interchange is currently used to transmit routine transaction data such as claims data for payment, eligibility files from plans to providers, and electronic payments (Austin and Boxerman 1998). Broader business-to-business applications are the fastest growing application of the Internet in other industries, and health-care providers will probably follow the trend. The largest entity currently providing electronic data interchange and business-to-business services to health-care firms in the United States is Healtheon-WebMD (http://www.webmd.com/).
Public health programmes
As described above, local health departments in industrialized countries establish programmes to carry out their legally mandated functions, such as communicable disease control, environmental health, substance abuse prevention/treatment, and the public health laboratory. Each of these programmes may have its own information system, built upon a database containing information about people, facilities, and service providers with whom the programme interacts. These information systems are essential to the effective functioning of the programmes, and also provide essential data on the health of the underlying population, as the following examples show.
Information systems to support communicable disease investigation and treatment were discussed above under the topic of surveillance. These information systems are person-oriented, i.e. their major data elements are demographics of infected people and contacts, disease characteristics, provider IDs, and treatment status. Other person-oriented information systems support nutrition and maternal and child health promotion programmes.
Environmental health programmes carry out inspections of facilities that prepare food as well as other facilities that serve the population, such as public swimming pools. These information systems are facility-oriented rather than person-oriented. For example, Los Angeles County, California, has an Environmental Health Management Information System database of records of inspections carried out at facilities, including facility characteristics, violations found, sanctions applied, and dates of correction.
Substance abuse prevention and treatment programmes in the United States often accomplish their mission by funding contractors that provide services to people with substance abuse problems. For such programmes, an information system must be provider-oriented as well as person-oriented. That is, its database must contain contractor characteristics, number of clients served, funding, and contract status, as well as data describing the people under treatment.
The public health laboratory processes a very large number of specimens (hundreds of thousand per year) for a very wide range of diseases, providing results throughout the community as well as to other public health programmes. Independent and hospital laboratories use transaction-oriented information systems to capture data electronically from instruments and store the results in a database. Public health laboratories can purchase and install these commercially developed systems with minimal customization.
These public health programme information systems currently rely on paper-based data collection in the field, although data collection in some public health facilities may be computerized. Each public health programme inputs the paper reports and maintains its own database. If paper reports are received and input by public health programmes in a timely manner, these systems can respond to public health hazards, such as communicable disease outbreaks, with adequate speed.
However, since timely input of paper forms is usually problematic, more data collection is likely to become electronic. For example, reporting of adverse health events by the public, such as foodborne illnesses contracted at restaurants, can easily be accomplished over the Internet (http://www.lapublichealth.org/phcommon/complaints/phcomp.cfm). As mentioned above, other disease surveillance activities can also be transitioned to the Internet. Public health field staff who collect large amounts of data, such as public health nurses administering directly observed therapy to tuberculosis patients, or food preparation facility inspectors, can input data to portable or handheld computers in the field, with wireless or network uploading to the central database. However, these systems are only in the early stages of development. They face many of the challenges that must be overcome by the electronic medical record, including diverse data arising from field encounters and variations in vocabulary, as well as the expense of equipping large field staffs with portable computers.
Health departments also perform many tasks generic to any large organization, including accounting and finance, human resource management, procurement, contracting, and equipment inventories. Information systems to perform these functions can be purchased from commercial vendors to other industries or government agencies, or developed by suppliers to those industries. In small health departments or in developing countries, desktop computer spreadsheets and database programs can be adequate to accomplish these functions.
Information technology allows expert clinicians to treat patients at a distance, via telecommunications links. These telemedicine services range from voice telephone communication to videoconferencing and image transmission requiring very high-bandwidth connections (Field 1996; Reid 1996). Telemedicine may significantly improve public health by allowing facilities in poor or rural areas to access clinical expertise.
Telemedicine can be adopted to the available computing power and telecommunications bandwidth (D. Krasnow, 2000, unpublished work). The simplest use of telemedicine is the telephone service using voice and/or fax. For example, a nurse in a rural clinic can call an urban-based doctor for advice on how to treat a particular type of injury. After one or two consultations about such injuries, she will be able to treat them on her own. Similarly, a doctor in a remote clinic can fax a patient’s history, physical examination, and test results to a medical school specialist, receiving the consultation results back the next day.
More sophisticated uses of telemedicine involve capturing images with a digital camera or scanned photograph, and transmitting them to an expert for consultation. For example, photographs of unusual dermatological lesions can be transmitted, with the dermatologist e-mailing back treatment instructions or further questions. The telephone service is adequate for transmitting still images. Inexpensive video cameras can also transmit images during real-time consultation, allowing the remote doctor to zoom in to examine the patient in greater detail. With a bandwidth of 384 kb/s or higher, real-time discussion between the patient and the remote doctor is possible.
Highly sophisticated applications of telemedicine are being developed in industrialized countries. Teleradiology is the most common, where images are transmitted for interpretation. Another ideal use of telemedicine is the treatment of rare diseases, where a remote expert doctor can prescribe treatment based on images. For example, neuroblastomas are rare ocular tumours in infants and are fatal if untreated. With a special handheld camera, images can be taken and transmitted to academic medical centres worldwide for evaluation and treatment recommendations.
Applications of telemedicine can combine simple and sophisticated technology (D. Krasnow, 2000, unpublished work). For example, retinal scans of diabetics can indicate the need for treatment to prevent blindness. However, existing retinal cameras are very large and expensive, and so the patient must go to the specialist. Affordable ophthalmoscopes that incorporate digital cameras are being developed, so that images can be taken in rural clinics or mobile units and then transmitted via the telephone service to specialists for evaluation. In rural populations with high diabetes prevalence, such as Native Americans, this capability could significantly improve secondary prevention of diabetes-related complications.
Cheaper computers and digital cameras, as well as more widespread high-bandwidth telecommunications, will foster the growth of telemedicine. Even developing countries are installing networks of sufficient bandwidth to support highly effective telemedicine, such as the ISDN links from Lima to regional hospitals in Peru.
In any country, technology is often the easiest part of telemedicine implementation. To achieve success, the people who use the systems must also be adequately trained and provided with appropriate incentives to use it. In addition, processes must be in place to integrate the use of the technology routinely into the provision of care (Krasnow and Rodrigues 1998).
Improved quality in the delivery of health services contributes to public health by conserving resources and by improving the health of patients. Information systems are essential for efficiently measuring the quality of health services. In industrialized countries, quality measurements focus on all potential types of problems: underuse of efficacious procedures, overuse of procedures whose risks exceed their benefits, and misuse that leads to avoidable complications (Chassin 1998). In developing countries, underuse because of poor access to services is the overwhelmingly most severe problem, followed by misuse (Peabody et al. 1999). In addition, information systems will play an increasing role in improving the process of care.
Measurement of the quality of health care can be broken down into the measurement of three different aspects (Donabedian 1980; Blumenthal 1996; Brook et al. 1996).
Structure: institutional capabilities and qualifications.
Process: technical and interpersonal aspects of the care provided to patients.
Outcome: patients’ morbidity, mortality, functional status, and quality of life.
Structural aspects are the easiest to measure, but have only an indirect effect on outcomes. Information systems are used in constructing structural quality measures, for example monitoring doctor credentialling at hospitals.
Data to construct process measures can be derived from hospital or clinic information systems, and specialized analysis software used to analyse the resulting data. When process criteria are clearly linked to outcome improvements, such as prescribing b-blockers to patients following myocardial infarction, process measurements that fall short of goals directly indicate where quality improvement interventions should be targeted. Information systems can also be used for ongoing monitoring after interventions are implemented (Mclaughlin and Kaluzny 1999). Clinical information systems, especially computerized order entry systems or electronic medical records, can incorporate clinical practice guidelines. When properly designed and integrated into the organizational context of a clinic or hospital, such information systems can significantly improve the quality of care, for example by reducing the incidence of medication errors (Evans et al. 1998).
Health outcomes are what patients and providers ultimately want to improve, but measurement of outcomes must adjust for confounding risk factors, such as the severity of patients’ underlying illnesses and the presence of comorbidities. Nevertheless, successful outcome measurements provide important policy and clinical data. For example, small-area variation studies have shown that rates of common surgical procedures vary greatly over small geographical distances, without correlation to the disease patterns of the underlying populations (Wennberg 1999). In New York State, clinical data have been collected and sophisticated statistical risk adjustment models applied to compare mortality after coronary artery bypass graft surgery at all hospitals in the state. Hospitals performing low volumes of the surgery were found to have higher risk-adjusted mortality. This measurement programme has been credited with improving outcomes by reducing the fraction of coronary artery bypass graft surgeries performed at lower-volume hospitals (http://www.health.state.ny.us/nysdoh/research/heart/heart.htm). Compiling data from facilities in sufficient clinical detail to construct risk-adjusted outcome measures requires sophisticated information systems.
Data analysis and policy development
The information systems described so far provide data that can be analysed to provide information for planning and policy development, including epidemiological studies, resource allocation decisions, program design and evaluation, and quality assessment. Specialized software has been developed to perform these analyses and make information available in formats usable by decision-makers and communities.
Computerized statistical and epidemiological analyses
Almost all statistical and epidemiological analyses in industrialized countries are now performed using computerized analysis software. The increase in memory storage capacity and processing power of desktop and portable computers enables even very large datasets to be analysed on inexpensive computers. The use of microcomputers in epidemiology is discussed in Chapter 6.15.
Powerful statistical programs such as SAS (http://www.sas.com/) can now be run on microcomputers. Other programs such as Stata (http://www.stata.com/) or SPSS (http://www.spss.com/spss10/index.htm) also offer very robust statistical analysis capabilities. Analysis of survey data requires special statistical procedures to account properly for the sample design, but survey data analysis programs like SUDAAN also run on microcomputers (http://www.rti.org/patents/sudaan/sudaan.html). Stata also offers some survey data analysis capabilities.
Epidemiology software is also available. The widely used EpiInfo program can be downloaded free from the Internet (http://www.cdc.gov/epiinfo/). This software performs a full range of epidemiological functions, and a basic mapping program (EpiMap) is also available at the same website. In conjunction with inexpensive microcomputers, such free software makes it feasible for analysts in almost any developing country to perform computerized data analyses.
Geographical information systems
A GIS is a database of different types of information, all linked to a common geographical co-ordinate system (http://wwwdb.csu.edu.au/division/dit/span/spatial_technology/gis/what_gis.html). For example, a GIS might include population characteristics and census area boundaries, or health facility locations and street maps. A GIS primarily displays data as maps, a very effective way to display large amounts of information in easily understandable form. Locations of events or facilities can be presented, or areas of the map colour coded, to indicate differing levels of a population characteristic. GISs are applicable to numerous aspects of public health (Yasnoff and Sondik 1999).
Inexpensive computers with the ability to generate complex graphics rapidly have made GIS technology widely available. Specialized software is used to compile the database and generate the maps. The two most common commercial GIS systems are distributed by the Environmental Systems Research Institute (ESRI) and MapInfo. While very powerful, these systems are relatively costly and require substantial user training to be used most effectively. EpiMap GIS software is freely available from the CDC (http://www.cdc.gov/epiinfo/index.htm), and is designed to link with EpiInfo. GIS capabilities are also built into many Internet sites. The American Census Bureau’s Topological Integrated Geographic Encoding and Referencing (TIGER) mapping system allows users to generate maps of population characteristics derived from census data (http://www.census.gov/geo/www/tiger/index.html).
A GIS analysis consists of several steps. The first is to obtain basic geographical data layers, such as street grids, political or census boundaries, or pollution source locations. The next step is to geocode the health events of interest, such as deaths or reported communicable disease cases. In geocoding, each event is assigned a latitude/longitude co-ordinate, by which it will be related to other data layers. Geocoding is not a perfect process, however, since address data collected in surveys may be incorrect or garbled. After geocoding, maps displaying the desired data layers can be produced and refined. Individual events may be displayed as points on the map. Average values for defined areas, such as census tracts within a county or states within a nation, can also be used to colour code map regions. Maps can be modified interactively by the user to address particular health questions, by adding or subtracting layers, zooming in and out, or changing colour-coding schemes. GIS databases can perform other analyses in addition to producing descriptive statistical maps, such as comparing rates over time, calculating optimal locations for new facilities, or correlating disease patterns to pollution sources. Figure 1 is an example of pollution source locations in California’s ‘Silicon Valley’, which could be correlated with geographical distributions of diseases potentially attributable to pollution.
Fig. 1 Groundwater contamination sites in Santa Clara County, California (‘Silicon Valley’).
Statistical and epidemiological methods must be properly incorporated into GIS analyses for public health. For example, disease rates in small geographical areas may be based on small numbers of cases, and therefore may be highly variable from year to year. Spatial statistical techniques have been developed to interpolate rates from observed data to locations between observations.
Indicator sets describe the health of a population in a summary fashion, including features such as the socio-economic environment, disease patterns, access to health services, and mortality. Indicators are useful for comparisons across populations, analyses of health trends over time, and development of policies to address a community’s most important health problems. One example set is the core health indicators developed by PAHO, which it publishes for its member countries. Some of these indicators are listed in Table 1.
Table 1 Selected PAHO Core Health Indicators (http://www.paho.org/English/SHA/bsindcvr.htm)
Developing an indicator report for a community is a multistep process (Durch et al. 1997) (see also Chapter 5.2). The first step is collaboration among stakeholders to choose the indicators to be measured, given the state of development and most pressing problems faced by the community. Secondly, data to calculate the indicators must be compiled from many different sources. For example, each of the data collection information systems described in this chapter might be a data source for one or more indicators. Thirdly, indicators must be calculated and published. Computers and networks are very useful in compiling and analysing the data and formatting it for presentation. Fourthly, policy-makers and community-based organizations can use the results as a basis for discussion, policy decisions, and resource allocation. However, once the process for collecting and analysing data is in place, future cycles are less labour intensive, and the data can be compared with the baseline period to evaluate the policies that are implemented.
Many indicator sets are available, which can be used to choose individual indicators and as templates for a community’s own efforts. Examples include the following.
The PAHO Core Health Data indicators are appropriate for comparisons both in and across developing countries.
The United States government’s Healthy People 2010 objectives establish goals for a broad range of public health objectives (http://www.health.gov/healthypeople/publications/default.htm).
The Institute of Medicine book Improving Health in the Community: A Role for Performance Monitoring (Durch et al. 1997) describes the process by which communities develop indicator sets and contains extensive lists of potential indicators.
The Health Plan Employer Data and Information Set (HEDIS) was developed to assess the quality of HMOs. While it has substantial shortcomings, it is the most widely used indicator set for comparing the quality of these organizations that are responsible for maintaining the health of their covered populations (http://www.ncqa.org/Pages/Programs/HEDIS/index.htm).
Sets of health indicators highlight health disparities, as well as health system strengths and weaknesses, that can be the targets of policy initiatives, but do not provide a single measure of the population’s health status. An indicator that combines measures of both morbidity and mortality is very useful for broad comparisons across communities defined by geography, race/ethnicity, or income. Disability-adjusted life-years (DALYs), a method for doing this based on disability weighting for diseases and health conditions, is described in detail in Chapter 2.9.
Linked databases and data warehousing
In-depth health policy and public health analyses often combine data on many different aspects of a population’s health, such as those summarized by health indicator sets. However, compiling the raw data for these analyses is the most labour-intensive part of the effort, since it resides in information systems developed and maintained by separate public health programmes, other government agencies, hospitals, and clinics. Nevertheless, the benefits of comprehensive analyses can be worth the effort, for example, to investigate patterns of infant mortality by combining vital statistics and hospital discharge data.
Private firms face similar challenges, with information systems in different departments containing different aspects of information about the same customers. As a result, firms have created data warehouses that summarize data from operational information systems in different departments (Marietti 1997). Inexpensive server computers with large amounts of hard disk storage lower the hardware costs of building a data warehouse, and firms can use existing local area networks and wide-area networks to upload data to the data warehouse. Specialized software is necessary to query these very large databases efficiently, and to provide a user interface that supports users ranging from managers to professional analysts.
Public health organizations could use data warehouses to aggregate data from different programmes, as well as from providers and payers, to assemble an overall picture of population health. Such a data warehouse would be of use to researchers, community-based organizations, and providers, in addition to analysts and managers in health departments. For example, data warehouses could provide currently unavailable comprehensive data for public health priority setting and resource allocation (UCLA 1997).
Despite this great promise, several technological and organizational challenges must be overcome before public health data warehouses can become widespread.
Different information systems have different formats for data such as personal identifiers, geographical locations, and medical terminology. (Data can be aggregated more easily in a health-care payer than in public health organizations, since the payer has a unique identifying number for each enrolled member.) Therefore the uploading of data from operational systems to the data warehouse must standardize data element formats, which is a time-consuming process.
Combining data from vital statistics registries, health surveys, and disease surveillance systems poses additional challenges (see Chapter 5.2). Surveys produce sample data, surveillance reports come from providers, and vital statistics are censuses, and so the data warehouse must employ different statistical analysis techniques for data from different sources. Proper epidemiological adjustments must also be automated if the data warehouse is to accept queries from less sophisticated users.
Querying large databases on multiple dimensions (e.g. age, gender, race/ethnicity, and location) simultaneously challenges the capabilities of current software and requires careful database design when the data warehouse is being constructed.
Privacy and confidentiality protection is challenging. On the one hand, detailed identifying data is necessary to link data from different sources. On the other hand, this linkage raises the risks from unauthorized uses of the data. Therefore access to data warehouse datasets that contain personally identifiable information must be carefully controlled, and outputs to users must be screened for confidentiality protection (e.g. by suppressing data for small cell sizes).
Gaining community stakeholder co-operation in sharing data from sources outside the public health organization, such as hospitals or health plans, can be difficult (Multnomah County Health Department 1999).
Some important steps are being taken towards the development of public health data warehouses. The Population Health Information System (POPULIS) in Manitoba aggregates data from many different sources to provide comprehensive population health data (Roos et al. 1996); Canada’s single-payer health insurance system helps nurture the necessary co-operation among stakeholders. In the United States, some states, such as Wisconsin, have developed datasets on many aspects of population health that could be linked together (http://badger.state.wi.us/agencies/oci/ohci/). Finally, the on-line query systems discussed below can be incremental steps to full data warehouses.
Data access and dissemination
The Internet has made a vast array of public health information available worldwide at the click of a mouse. Once information has been collected or created, dissemination via the Internet is much faster and simpler than by print or physical digital media such as CD-ROMs. Types of public health information available on line include documents (e.g. books, papers, reports, brochures), downloadable datasets, and query systems. Each of these types of information is described in more detail below. Internet data access tools vary in sophistication from those intended for analysts with advanced training to ones targeted at the general public.
A special type of information resource, the centralized index or bibliography, has been developed to make all these information sources easily accessible to users. The broadest indexes are the Internet ‘search engines’. These are databases of web page content, compiled by software that automatically accesses and catalogues all types of web pages. Users can perform simple or complex keyword searches of all these web pages simply by accessing the search engine’s main web page. For example, Google (http://www.google.com/) indexes over 1 billion web pages and has a sophisticated algorithm to rank search results by likely relevance to the user’s query. Portals are indexes targeted at users in specific disciplines. MDConsult (http://www.mdconsult.com/) provides a wide range of medical information to doctors, including news summaries and searchable databases of journals and clinical practice guidelines. Some portals (including MDConsult) charge a membership fee for full access to their content. Bibliographies of Internet resources in particular disciplines are often maintained by libraries or non-profit organizations, and can offer users very efficient access to relevant websites. Useful bibliographies of public health Internet resources are provided by the University of California at Berkeley (www.lib.berkeley.edu/PUBL/Internet.html), Johns Hopkins University (http://support.jhsph.edu/sph/intresources/), and the University of Iowa (http://www.lib.uiowa.edu/hardin/md/).
Internet data access requires relatively little information technology infrastructure: a microcomputer, either direct connection to a local area network connected to the Internet or a telephone line and modem to connect to an Internet service provider, and several pieces of inexpensive software, including a browser program (such as Netscape or Microsoft’s Internet Explorer), the free Adobe Acrobat Reader (http://www.adobe.com/products/acrobat/readermain.html) to read documents in .pdf format, e-mail software, word-processing and spreadsheet software to open downloaded files, and virus detection/removal software to protect against harmful computer viruses attached to downloaded files.
Increased availability of these tools and of high-bandwidth telecommunications will make Internet public health resources available to more users, and will allow access to larger and more interactive databases. Conceptually, and soon practically, any product that can be put in digital format can be shared via the Internet.
Public health and medical literature
More and more of the published academic and policy literature relevant to public health is becoming available on-line. The largest resource for accessing the biomedical, public health, and health services research literature on-line is the United States National Library of Medicine (NLM). NLM’s bibliographic database, Medline, is now accessible via the Internet (http://www.ncbi.nlm.nih.gov/entrez/query.fcgi). Full citations and abstracts can be downloaded.
The full text of journals and books is also increasingly available on-line. The Morbidity and Mortality Weekly Report published by the CDC, an essential tool for dissemination of public health data, is now fully available via the Internet (http://www.cdc.gov/mmwr/). Some prominent medical journals, such as the British Medical Journal, are freely accessible in full text. Others, such as the New England Journal of Medicine, make abstracts and some articles available free, but require a subscription fee for complete access. Ovid (http://www.ovid.com/) is a service that provides the full text of a large number of journals in return for a licensing fee. Universities and other organizations also make medical textbooks available on line to staff and students. Electronic versions of these reference materials can be better than print versions, offering more frequent updates and fully searchable text. Finally, government agencies can make their reports, which are in fact public goods, available on line. The American Institute of Medicine (http://www.nap.edu/) has begun making many of its books accessible this way.
Public-use versions of many datasets from vital statistics registries or surveys are being made available for downloading via the Internet. Others are available in magnetic tape or CD-ROM format for a fee and/or under confidentiality restriction agreements. This section will focus on datasets available via the Internet.
Demographic data for many geographical regions can be downloaded, and are fundamental inputs to public health analyses. For example, data on the age and gender of residents within specified geographical boundaries are necessary to calculate mortality or disease prevalence rates. The Data Extraction System at the United States Census Bureau Internet site (http://www.census.gov/) provides access to a public-use versions of data from the decennial census as well as from other ongoing surveys of income and other population characteristics. Users define the parameters of the data that they require, i.e. variables and geographical boundaries, and receive the dataset by e-mail. Other demographic data, such as vital statistics on births and deaths, can also be accessed on line. One example is the United States NCHS National Death Index (part of the NVSS) described above.
Data obtained from many of the health surveys and disease surveillance systems described above are also available via the Internet. For example, public-use datasets from several United States NCHS surveys and other national surveillance systems are available via the dedicated Internet data portal CDC WONDER (http://wonder.cdc.gov/). After choosing a survey or type of surveillance data, users receive a dataset that meets the parameters they define, through a process similar to that described above for the American census data. DHS surveys provide comparable high-quality data for many developing countries, and these datasets are also available for download (http://www.measuredhs.com/). For users wishing to learn about downloadable data sources available on particular topics, the University of Michigan maintains a list of numerous such resources (http://www.lib.umich.edu/libhome/Documents.center/sthealth.html).
On-line query systems
Analysts with the necessary training and software can manipulate downloaded datasets to answer their particular questions. However, users without such training, including managers, policy-makers’ staffs, and other public health practitioners, often need quantitative information about population health as inputs to their decision-making. For example, information about rates of different diseases is essential for the rational allocation of limited resources across programs designed to prevent those diseases. On-line query systems can provide answers directly to users logged on via local area network, wide-area network, or the Internet. These information systems are much more flexible than printed reports, which show only a few of the many potential ways to present large datasets. They accept structured queries, usually via a system of dialogue boxes that prompt users to define the parameters of their query.
A wide range of types of data are being made available in this fashion, including demographics, health indicators, and health services utilization. For example, several American states offer population and demographics query capabilities. Figure 2 is an example of such a dialogue box from a population data query system offered by the Utah Department of Health (http://hlunix.hl.state.ut.us/hda/population/). The PAHO core health indicators described above can be queried via the Internet (http://www.paho.org/English/SHA/ihomeibs.htm). An example output is shown in Fig. 3. The AHRQ offers an Internet query system called HCUPnet (http://www.ahrq.gov/data/hcup/hcupnet.htm) for hospital utilization data collected as part of its Healthcare Quality and Utilization Project (HCUP). This system is an example of a sophisticated but user-friendly query system. It offers outputs in time trends as well as in tabular form. In some cases, databases that can be downloaded can also be queried directly. An example of this is the database of results from DHS surveys from many developing countries (http://www.ahrq.gov/data/hcup/hcupnet.htm).
Fig. 2 Utah population projection. (Source: Utah Department of Health.)
Fig. 3 Core data for the Americas (September 1998).
Underlying any interactive query system is a database of information categorized along the dimensions that can be queried. This may be a dataset composed of individual observations (such as vital statistics registries), or even multiple linked datasets as described above. Statistical analyses are performed by calling preprogrammed routines, which must also contain the necessary confidentiality and privacy protection conditions. Therefore, to provide faster response to users and to ensure confidentiality, many query systems draw their responses from large, preformatted summary tables.
To serve users who are not trained analysts effectively, the user interface of an interactive query systems should be as simple as possible to learn. To be as useful as possible, they should provide not only cross-sectional output tables, but also time trends and bar charts for comparisons over time and across groups. Maps can also be used, both to select geographical regions as part of a query and to display results by colour-coding map regions.
Information sharing among public health professionals
The Internet greatly improves the efficiency with which public health professionals can share information on best practices. Such sharing is especially important in public health, because while development of new procedures and programmes is costly and time-consuming, public health professionals in different cities, states, and countries usually perform very similar functions and can borrow ideas from each other easily.
The simplest way for individual professionals to share information via the Internet is a USENET e-mail newsgroup. In this method, people who share an interest in a specific topic subscribe to an e-mail list. All members of the list receive the questions, answers, and notices posted to the list by its members. For example, the author subscribes to such a list server of people sharing an interest in the design of state health surveys. Many common methodological issues and survey questions are posted and discussed by members. Some lists are moderated by their founders to make the list as useful as possible by weeding out inappropriate postings.
Community-based organizations in the same region or with shared goals can also easily share information and best practice ideas with each other. E-mail contacts are the simplest to set up. With more co-ordination, groups of community-based organizations can create, share, and update electronic inventories of health facilities, programmes, and other resources in their region, to facilitate referrals and collaboration.
More structured information sharing can be sponsored by government agencies or non-profit organizations. For example, several United States government and non-profit organizations have formed the ‘Partners in Information Access for Public Health Professionals’ programme (http://www.nnlm.nlm.nih.gov/partners/) to facilitate access to a variety of public health data sources. Sharing is also possible on an international level. The International Clearinghouse on Health Systems Reform Initiatives (ICHSRI) compiles information on health systems reform initiatives in developing countries (http://www.insp.mx/ichsri/). Users can access the site to learn about reform initiatives, compare them, and discuss related issues and challenges. In the coming years, improvements in automated language translation software may further widen the international sharing of best practices among governmental and non-governmental organizations.
Providing health data to community members
Individual members of the population are increasingly turning to the Internet as a source of health information. Tens of millions of Ameican residents are now doing so, and that number grows as Internet access expands (Wellner 2000). To the extent that consumers are thus better informed about disease prevention, health promotion, and the appropriate utilization of health services, the Internet will have a beneficial impact on public health. However, sites that contain inaccurate information or information biased due to commercial sponsorship pose a risk to the health of on-line consumers.
Government and non-profit organizations offer health information over the Internet as a public service. One of the best examples is the Cancer Information Service at the National Cancer Institute (http://cis.nci.nih.gov/). This site, and its associated toll-free telephone number, are extremely useful resources for consumers faced with the need to learn about cancer diagnosis and treatment. Local public health departments can also post information, such as advisories of health hazards, on their websites for easier access. The Internet is an ideal medium for publishing reference materials, and community resource guides are a reference that can assist those seeking health services. For example, the People’s Guide, a comprehensive reference on how to obtain health and other social services in California, is available on line (http://www.peoplesguide.org/) as well as in print.
Numerous for-profit websites have been founded to provide health information to consumers. The business-to-consumer sites, such as DrKoop.com (http://www.drkoop.com/) and WebMD (http://shn.webmd.com/), offer a wide range of information on health promotion, specific conditions, and treatments. The best of these sites are easy to use, often contain well-researched and well-written information, and draw millions of viewers. However, they have struggled to find commercial sponsors to support their ongoing operations, and the long-term viability of many of these business-to-consumer sites is currently in doubt. Other for-profit health services providers, including health plans, doctor groups, and hospitals, also maintain websites. While these sites may contain useful health information, their primary purpose is marketing to new patients and providing services more efficiently to current patients.
Search engines and well-designed general health websites make it relatively easy for consumers to find information on the Internet about the health topics of interest to them. However, the quality of that information is not guaranteed, and most consumers are not trained to evaluate the validity of the information they access. For example, large numbers of ineffective or dangerous treatments are promoted via the Internet (http://www.familyInternet.com/quackwatch/). No uniform enforceable standards for health information quality have yet been developed, although some outlines for such standards are emerging (Winker et al. 2000). At the present time, consumers must still carefully evaluate the sponsorship and the source of health information they access via the Internet (Cooke 1999).
Health training and education
Information systems can enhance both training for public health professionals and health education programmes targeted at consumers. Information systems have been incorporated into consumer health education for many years, especially since inexpensive microcomputers began to offer powerful graphics capabilities. High-bandwidth network access will allow computerized educational content, including content that supports user interactivity, to be distributed even more widely. Overall, the basic health education and training methods are not likely to be replaced, but some of them may be transformed or expanded by the use of information technology and telecommunications (Chamberlain 1996).
This section will first discuss the application of information systems to distance learning, for training of public health professionals. It will then describe the effects of information technology and telecommunications on several modes of consumer health education, including campaigns, curriculum materials, and the exchange of information among groups with specialized health interests. (Internet dissemination of health information to consumers is discussed at the end of the previous section of this chapter. General principles of health education are presented in Chapter 7.3.
Distance learning is a relatively inexpensive way to teach many students in dispersed locations from a central locus of expertise, such as a school of public health or medicine. Given the shortage of adequately trained public health personnel, especially in rural areas and developing countries, it can be an important way to leverage limited public health resources. For example, the CDC and several partner organizations established the Public Health Training Network (PHTN) in 1993 to provide distance learning services for the state and local public health workforce. Over 1 million people have received training and information via PHTN to date (http://www.cdc.gov/phtn/).
Distance learning can be conducted in several different modes, with increasing degrees of interactivity (and cost): videotape or CD-ROM, Internet or print course materials, audio or video broadcast, and audio or video conferencing. Audio or video can be delivered via telephone lines, radio and television, satellite, local area networks or wide-area networks, or the Internet. Students can interact with the instructor using e-mail or on-line chat sessions. Several of these delivery modes and technologies can be combined in one program, such as Internet distribution of course materials, video delivery of lectures, and e-mail feedback to instructors.
Availability of cheaper and higher-bandwidth telecommunications should accelerate the application of the more interactive modes of distance learning in public health. For example, acceptable quality interactive video conferencing is possible using inexpensive computer-mounted video cameras and a transmission bandwidth of 384 kb/s, achievable using ISDN or digital subscriber line telephone lines (V.G. Winting, personal communication, 2000).
In industrialized countries, distance learning is increasingly focusing on video delivery or interactive video conferencing. For example, the four schools of public health in California installed video-conferencing equipment and dedicated ISDN lines to provide training to rural health departments in the state; webcasting capability is currently being added (V.G. Winting, unpublished data, 1999; J.M. Nunez, personal communication, 2000).
Less expensive delivery modes can still be effective, and are much more practical in developing countries. For example, downloadable curriculum materials for self study or for local trainers/teachers can be easily disseminated via the Internet. The Global Health Network Supercourse (Anonymous 1999) is a series of lecture materials on topics such as epidemiology, developed by participating public health faculty in many countries, that are made available worldwide via the Internet.
As mentioned above with regard to telemedicine, the technology component of distance learning is often not the most challenging. A useful framework for evaluating the practicality of a distance learning application is that the application must be simultaneously:
available (i.e. technologically feasible in the context of a particular country)
accessible to the target users (e.g. convenient to their workplaces)
If any one of these criteria is absent, the application is unlikely to succeed as planned (V.G. Winting, personal communication, 2000).
Health education campaigns are designed to convey a specific message, such as safe sex practices or smoking cessation, to a specific audience. These campaigns, one of the techniques broadly known as social marketing, are widespread in both industrialized and developing countries (Glanz et al. 1997). Their messages have traditionally been delivered using broadcast and print media, as well as community worker outreach. However, as Internet use expands, it makes possible the targeting of messages to narrower and narrower niche audiences. Commercial advertisers are increasingly using Internet advertising for this purpose, and health education campaigns can be expected to adopt this practice as well (D. Glik, personal communication, 2000).
Health journalism is another means of educating consumers. It is growing in prominence, with coverage of biomedical discoveries and public health themes. As more journalistic content is distributed via the Internet, health journalism is also likely to migrate there (D. Glik, personal communication, 2000). Similarly, entertainment media can be used to communicate health education messages, and expanded use of computerized entertainment should offer opportunities to deliver health education (D. Glik, personal communication, 2000).
Computerized multimedia presentations, incorporating graphics and audio as well as text, are effective vehicles to present large amounts of information to target audiences. The viewer can proceed through the material at his or her own pace and replay sections as necessary. Interactive multimedia allow the viewer to make queries or answer questions posed by the software. Multimedia tools have been developed for several public health education topics, such as maternal and child health promotion (D. Glik, personal communication, 2000). Such tools can also be particularly helpful for patients with serious chronic health conditions. For example, they can teach preventive health behaviours to children with diabetes or asthma (Robitaille 2000).
Multimedia tools have mostly been distributed on CD-ROMs. These are cheap to manufacture and distribute, and can be played back on inexpensive microcomputers. They are usually used in institutional settings, such as clinics or hospitals, where the viewer can have access to a computer (D. Glik, personal communication, 2000). However, high-bandwidth networks and telecommunications make it possible to deliver large files easily, offering opportunities to distribute multimedia educational materials to much wider audiences via the Internet.
Nevertheless, creating the content itself will remain the bottleneck in developing effective multimedia health education materials (Chamberlain 1996). Crafting and testing messages and tailoring the interactive presentation requires experts in health education, graphics design, and video production. These skills are costly, and the development process is time-consuming. Although Internet dissemination makes the marginal cost of distribution negligible, substantial funding will still be required to design, produce, and test effective content.
Specialized health information providers
While the Internet can provide health information to the general population cheaply, it can be even more useful to people suffering from particular health conditions. These people, especially if they have physical disabilities or live in rural areas or small cities, previously had very limited opportunities to share knowledge and experiences with other people suffering from the condition. However, the Internet makes it possible to aggregate these small populations on a national or international basis. These on-line grassroots groups can provide education and mutual support, as well as be vehicles for political activism. Non-profit health organizations can also use their websites to disseminate educational information about the conditions or health issues they address.
Both websites and e-mail groups can be used for communication among these condition-specific on-line communities. Their numbers and membership can be expected to grow as Internet access expands throughout the world. At the time of writing, the Yahoo Internet directory lists over 8450 websites devoted to specific diseases and health conditions (http://dir.yahoo.com/Health/Diseases_and_Conditions/). Commercial enterprises can also arise to serve the needs of these communities, such as eBiocare.com, a pharmacy specializing in genetically engineered medications for people with six different health conditions (http://www.ebiocare.com/index.asp).
Challenges to applying information systems in public health
The previous sections have outlined many ways in which information systems can contribute to improving public health. Continued technological advancement and innovative applications of those technologies in organizations will certainly foster even more ideas in the years to come. However, several challenges must be overcome before this full potential can be realized. This final section summarizes some of those obstacles to more widespread application of information systems to public health.
Information technology and telecommunications infrastructure
Many of the potential advances described above will require improved and expanded information technology and telecommunications infrastructure, in both industrialized and developing countries. Applications in industrialized countries, such as multimedia health education or fully electronic disease surveillance systems, depend on the widespread availability of inexpensive high-bandwidth telecommunications, such as digital subscriber line telephone lines or cable television modems. In the United States, businesses and government agencies are installing these capabilities rapidly, but only a very small minority of households (less than 5 per cent) have them at present. In other industrialized countries, deployment of these telecommunications systems is much less advanced. Even within the United States, telecommunications infrastructure in rural areas and poor urban areas lags far behind that in wealthy urban areas.
In developing countries, even the availability of computers and reliable telephone service cannot be taken for granted in public health agencies or health-care provider organizations, let alone among the general population. However, even inexpensive computers and voice telephone connections can support appropriately tailored information systems applications with large benefits, such as Internet delivery of educational materials or voice-based telemedicine. Investments in computer hardware can be focused on leveraging the skills of trained analysts at regional and national levels. Information can be transferred among regions or to the national level using physical storage media. As described in Chapter 5.3, advances in wireless communications will allow important public health applications of information systems to proceed long before hardwired telecommunications services are universal.
In most industrialized countries, funding for the core public health activities of disease prevention and health promotion, as well as primary health care, have historically been underfunded in comparison to the provision of curative health services. In developing countries, the poverty of the population and limited government budgets for health make this funding misallocation even more severe.
Unfortunately, information systems development and implementation require significant initial investments in computer hardware, software development, network infrastructure, and training before benefits can begin to accrue. This will remain a major hurdle to the broader application of information systems in public health. Therefore public health leaders must become more knowledgeable about information systems (Yasnoff et al. 2000) and argue vigorously to policy-makers when the benefits of such investments clearly exceed the costs.
Privacy and confidentiality protection
Health data are among the most sensitive of personal information, and organizations that collect such data from individuals are obligated to protect it from unauthorized use or disclosure. However, wider applications of information systems in health care mean that more and more personal health data are available in electronic form, and are therefore subject to new and worrying disclosure risks. Public concerns over protecting the privacy of personal health data are especially strong in countries like the United States, where the majority of the population is not covered by universal health insurance.
Organizations that collect personal health data, including public health agencies, health services providers, and health financing organizations, must protect against three different types of confidentiality breaches.
Access to stored health information by unauthorized people. This is accomplished by controlling access to information systems with tools like passwords and restricted access to the most sensitive types of data.
Interception of information while it is being transmitted from the source to the collecting organization, or between organizations. This is accomplished using encryption technology.
Sharing of information among organizations without the originating person’s full consent. The solution to this problem is not a technological one, but rather requires health organizations to adopt and enforce strict procedures with regard to sharing the information they have collected.
In the United States, federal legislation carrying significant penalties will soon establish the framework for the protection of consumer health information. The ‘administrative simplification’ provisions of the Health Insurance Portability and Accountability Act (HIPAA) of 1996 require that health organizations comply with national standards for the protection of health data and electronic sharing (‘transactions’) of that data. Regulations implementing this requirement were published in October 2000, and organizations have 2 years to comply. Almost any information system containing personal health data may be affected by HIPAA requirements (http://www.jhita.org/).
Data collected through vital statistics systems, disease surveillance, and health surveys is often highly sensitive and personally identifiable. However, there are important public health reasons for making these data as widely available as possible. Agencies that collect such data must therefore strike a balance between easy access and necessary privacy protection. Several techniques are employed to accomplish this. Access to complete data files should be restricted to authorized users working under clear confidentiality procedures. Public-use data files are stripped of all personal identifying information, such as name or address. Because combinations of individual characteristics (e.g. age, race, income, and education) can allow salient individuals to be identified in small population divisions, geographical identifiers in public-use datasets may be very general (e.g. state only). Users who wish to see lower-level detail, or to link different datasets, may be required to do so at special data centres, whose staff supervise analyses to prevent confidentiality breaches. In the future, it is possible that public access to some datasets will be allowed only via query systems, with even professional analysts being required to work through data centres to access complete datasets.
Information technology development challenges
Development and implementation of large information systems are risky in any industry. The technical sophistication, limited funding, and confidentiality requirements of public health information systems all increase the risk of failure if the development process is not well managed. Some risks can be mitigated by borrowing technologies, such as GIS, that have been refined in other industries. In addition, proven information systems development methods can be applied as successfully in public health as in any other industry. On the technical side, effective health services software development methodologies are well documented (Austin and Boxerman 1998; PAHO 1999). Organizational leadership skills are also necessary to prevent development problems in large information systems (Lorenzi and Riley 1995). While these methods are documented and proven, applying them correctly is a key responsibility of public health managers.
Despite the formidable challenges to overcome, broader application of information systems can improve our ability to achieve the goals of public health. Rapid technological evolution makes it impossible to predict future trends precisely, but what we can already realistically envision is remarkably exciting.
Aday, L.A. (1996). Designing and conducting health surveys: a comprehensive guide. Jossey-Bass, San Francisco, CA.
Anonymous (1999). The Global Health Network Supercourse: epidemiology, the Internet, and global health. Telemedicine Journal, 5, 303–7.
Austin, C.J. and S.B. Boxerman (1998). Information systems for health services administration. Health Administration Press, Chicago, IL.
Blumenthal, D. (1996). Quality of health care. Part 1: What is it? New England Journal of Medicine, 335, 891–4.
Brook, R.H., McGlynn, E.A., et al. (1996). Quality of health care. Part 2: Measuring quality of care. New England Journal of Medicine, 335, 966–70.
Chamberlain, M.A. (1996). Health communication: making the most of new media technologies—an international overview. Journal of Health Community, 1, 43–50.
Chassin, M.R. (1998). Is health care ready for Six Sigma quality? Milbank Quarterly, 76, 565–91, 510.
Chunning, C. (1992). Disease surveillance in China. Morbidity and Mortality Weekly Report, 41(Supplement), 111–22.
Cibulskis, R. and Izard, J. (1996). Monitoring systems. In Health policy and systems deveopment: an agenda for research (ed. J. Kanovsky). WHO, Geneva.
Cooke, A. (1999). Quality of health and medical information on the Internet. Clinical Performance and Quality Health Care, 7, 178–85.
Dick, R.S., Steen, E.B., et al. (ed.) (1997). The computer-based patient record: an essential technology for health care. National Academy Press, Washington, DC.
Donabedian, A. (1980). The definition of quality and approaches to its assessment. Health Administration Press, Ann Arbor, MI.
Drotman, D.P. and Strassburg, M.A. (2001). Sources of data. In Epidemiologic methods for the study of infectious diseases (ed. J.C. Thomas and D.J. Weber). Oxford University Press.
Durch, J.S., Bailey, L.A., et al. (ed.) (1997). Improving health in the community: a role for performance monitoring. National Academy Press, Washington, DC.
Evans, R.S., Pestotnik, S.L., et al. (1998). A computer-assisted management program for antibiotics and other antiinfective agents. New England Journal of Medicine, 338, 232–8.
Field, M.J. (ed.) (1996). Telemedicine: a guide to assessing telecommunications in health care. National Academy Press, Washington, DC.
Glanz, K., Lewis, F.M., et al. (ed.) (1997). Health behavior and health education: theory, research, and practice. Jossey-Bass, San Francisco, CA.
Hammer, M. (1990). Reengineering work: don’t automate, obliterate. Harvard Business Review, 68, 104–12.
Husein, K., Adeyi, O., et al. (1993). Developing a primary care information system that supports the pursuit of equity. Social Science and Medicine, 36, 585–96.
Institute of Medicine (1988). The future of public health. National Academy Press, Washington, DC.
Kipp, W., Kielman, A., et al. (1994). Monitoring of primary health care services: an example from Western Uganda. Health Policy and Planning, 9, 155–60.
Krasnow, D. and Rodrigues, R.J. (1998). International perspectives. Telemedicine: practicing in the information age (ed. S.F. Viegas and K. Dunn). Lippincott–Raven, Philadelphia, PA.
Lorenzi, N.M. and Riley, R.T. (1995). Organizational aspects of health informatics: managing technological change. Springer-Verlag, New York,
Love, D. (2000). State data sources. Presented at Association for Health Services Research Annual Meeting, Los Angeles, CA.
Mclaughlin, C.P. and Kaluzny, A.D. (1999). Continuous quality improvement in health care: theory, implementatiion, and applications. Aspen Publishers, Baltimore, MD.
Marietti, C. (1997). The data warehouse. New uses for old data. Healthcare Informatics, 14, 93–6, 98, 100.
Multnomah County Health Department (1999). Designing a public-private integrated health information system for use in local public health planning and policy development. Multnomah County Health Department, Portland, Oregon.
O’Carroll, P., Yasnoff, W., et al. (ed.) (2000). Public health informatics and information systems. Aspen Publishers, Baltimore, MD.
PAHO (Pan American Health Organization) (1999). Setting up healthcare services information systems: a guide for requirements, analysis, application specification, and procurement. PAHO, Washington, DC.
Peabody, J.W., Rahman, M.O., et al. (1999). Policy and health: implications for development in Asia. Cambridge University Press.
Reid, J. (1996). A telemedicine primer: understanding the issues. Innovative Medical Communications, Billings, Montana.
Robitaille, S. (2000). Orphan technologies look for a home. California Healthline. http://www.chcf.org/features/index.cfm?itemID=1326
Roos, N.P., Black, C., et al. (1996). Population health and health care use: an information system for policy makers. Milbank Quarterly, 74, 3–31.
Satia, J.K., Mavalankar, D.V., et al. (1994). Micro-level planning using rapid assessment for primary care health services. Health Policy and Planning, 9, 318–30.
UCLA (1997). Report of review of public health programs and services, Los Angeles County, Department of Health Services. UCLA School of Public Health Technical Assistance Group, Los Angeles, CA.
Wellner, A.S. (2000). Casting the health.net. American Demographics, 22, 46–9.
Wennberg, J.E. (1999). Understanding geographic variations in health care delivery. New England Journal of Medicine, 340, 52–3.
Winker, M.A., Flanagin, A., et al. (2000). Guidelines for medical and health information sites on the Internet: principles governing AMA websites. Journal of the American Medical Association, 283, 1600–6.
Woodall, J.P. (1988). Epidemiological approaches to health planning, management, and evaluation. World Health Statistics Quarterly, 41, 2–10.
Yasnoff, W.A. and Sondik, E.J. (1999). Geographic information systems (GIS) in public health practice in the new millennium. Journal of Public Health Management and Practice, 5, ix–xii.
Yasnoff, W.A., O’Carroll, P.W., et al. (2000). Public health informatics: improving and transformng public health in the information age. Journal of Public Health Management and Practice, 6, 65–75.
Web page citations
http://badger.state.wi.us/agencies/oci/ohci/. ‘Wisconsin Office of Health Care Information.’ Wisconsin State Government. Accessed 6 October 2000. Updated 31 August 2000.
http://cis.nci.nih.gov/. ‘Cancer Information Service.’ National Cancer Institute. Accessed 7 October 2000. Updated 5 October 2000.
http://dir.yahoo.com/Health/Diseases-and-Conditions/. ‘Diseases and Conditions.’ Yahoo. Accessed 7 October 2000.
http://familyInternet.com/quackwatch/. ‘Quackwatch.’ Quackwatch. Accessed 7 October 2000. Updated 29 January 2000.
http://hlunix.hl.state.ut.us/hda/population/. ‘Utah Population Projection.’ Utah Department of Health. Accessed 7 October 2000.
http://newfederalism.urban.org/nsaf/. ‘National Survey of America’s Families.’ Urban Institute. Accessed 6 October 2000.
http://shn.webmd.com/. ‘WebMD.’ WebMD. Accessed 7 October 2000.
http://support.jhsph.edu/sph/intresources/. ‘Internet Public Health Resources.’ Johns Hopkins University. Accessed 7 October 2000.
http://wonder.cdc.gov/. ‘CDC WONDER.’ CDC WONDER. Accessed 7 October 2000.
http://www.ahrq.gov/data/hcup/hcupnet.htm. ‘Healthcare Cost and Utilization Project.’ Agency for Healthcare Research and Quality. Accessed 7 October 2000. Updated August 2000.
http://www.ccrcal.org/abouttheccr.html. ‘California Cancer Registry.’ California Cancer Registry. Accessed 6 October 2000.
http://www.cdc.gov/. ‘Public Health Training Network.’ Centers for Disease Control and Prevention. Accessed 7 October 2000.
http://www.cdc.gov/epiinfo/. ‘Epi Info.Epi Map.’ Centers for Disease Control and Prevention. Accessed 6 October 2000. Updated 5 June 2000.
http://www.cdc.gov/epiinfo/index.htm. ‘Epi Info and Epi Map.’ Centers for Disease Control and Prevention. Accessed 6 October 2000. Updated 5 June 2000.
http://www.cdc.gov/mmwr/. ‘Morbidity and Mortality Weekly Report.’ Centers for Disease Control and Prevention. Accessed 7 October 2000. Updated 6 October 2000.
http://www.cdc.gov/nccdphp/brfss/index.htm. ‘Behavioral Risk Factor Surveillance System.’ Centers for Disease Control and Prevention. Accessed 6 October 2000. Updated 28 September 2000.
http://www.cdc.gov/nchs/about/major/dvs/mortdata.htm. ‘Mortality Data from the National Vital Statistics System.’ Centers for Disease Control and Prevention. Accessed 6 October 2000. Updated 12 September 2000.
http://www.cdc.gov/nchs/nhanes.htm. ‘National Health and Nutrition Examination Survey.’ Centers for Disease Control and Prevention. Accessed 6 October 2000. Updated 25 July 2000.
http://www.cdc.gov/nchs/nhcs.htm. ‘National Health Care Survey.’ Centers for Disease Control and Prevention. Accessed and updated 24 July 2000.
http://www.cdc.gov/nchs/nhis.htm. ‘National Health Interview Survey (NHIS).’ Centers for Disease Control and Prevention. Accessed 6 October 2000. Updated 12 September 2000.
http://www.cdc.gov/od/hissb/docs.htmnedss. ‘National Electronic Disease Surveillance System.’ Centers for Disease Control and Prevention. Accessed 6 October 2000. Updated 9 August 2000.
http://www.census.gov/. ‘U.S. Census Bureau.’ US Census Bureau. Accessed 7 October 2000.
http://www.census.gov/geo/www/tiger/index.html. ‘Topologically Integrated Geographic Encoding and Referencing system.’ US Census Bureau. Accessed 6 October 2000. Updated 3 October 2000.
http://www.db.csu.edu.au/division/dit/span/spatialtechnology/gis/whatgis.html. ‘Geographic Information System.’ Chicago State University. Accessed 6 October 2000.
http://www.drkoop.com/. ‘Drkoop.’ Accessed 7 October 2000.
http://www.ebiocare.com/index.asp. ‘eBioCare.com.’ eBioCare.com. Accessed 7 October 2000. Updated 5 October 2000.
http://www.google.com/. ‘Google.’ Google. Accessed 6 October 2000.
http://www.hcfa.gov/medicaid/m2082.htm. ‘Medicaid Statistical Information System.’ Health Care Financing Administration. Accessed 6 October 2000. Updated 17 May 2000.
http://www.hcfa.gov/stats/pufiles.htmpufcat. ‘Public Use Files.’ Health Care Financing Administration. Accessed 6 October 2000. Updated 29 September 2000.
http://www.health.gov/healthypeople/publications/default.htm. ‘Healthy People 2010.’ US Department of Health and Human Services. Accessed 6 October 2000.
http://www.health.state.ny.us/nysdoh/research/heart/heart.htm. ‘Information for Researchers: Heart Disease.’ New York State Department of Health. Accessed 6 October 2000. Updated October 2000.
http://www.insp.mx/ichsri/. ‘International Clearinghouse of Health System Reform Initiatives.’ Accessed 7 October 2000.
http://www.jhita.org/. ‘Health Insurance Portability and Accountability (HIPAA).’ Joint Healthcare Information Technology Alliance. Accessed 7 October 2000. Updated 20 July 2000.
http://www.lapublichealth.org/phcommon/complaints/phcomp.cfm. ‘REPORT-A-PROBLEM PAGE.’ LA PUBLICHEALTH.ORG Public Health Programs. Accessed 6 October 2000.
http://www.lib.berkeley.edu/PUBL/Internet.html. ‘Public Health Resources the Internet.’ UC Berkeley. Accessed 7 October 2000. Updated 25 September 2000.
http://www.lib.uiowa.edu/hardin/md/. ‘Hardin Meta Directory of Internet Health Sources.’ University of Iowa. Accessed 7 October 2000. Updated 5 October 2000.
http://www.lib.umich.edu/libhome/Documents.center/sthealth.html. ‘Statistical Resources on the Web.’ University of Michigan. Accessed 7 October 2000.
http://www.mdconsult.com/. ‘MD Consult.’ MD Consult. Accessed 6 October 2000.
http://www.measuredhs.com/. ‘Demographic and Health Survey.’ Accessed 6 October 2000.
http://www.meps.ahrq.gov/. ‘Medical Expenditure Panel Survey.’ Agency for Healthcare Research and Quality. Accessed 6 October 2000. Updated 24 February 2000.
http://www.nap.edu/. ‘National Academy Press.’ National Academy Press. Accessed 7 October 2000.
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi. ‘National Library of Medicine: PubMed.’ National Center for Biotechnology Information. Accessed 7 October 2000.
http://www.ncqa.org/Pages/Programs/HEDIS/index.htm. ‘Health Plan Employer Data and Information Set (HEDIS).’ NCQA. Accessed 6 October 2000.
http://www.nlm.nih.gov/pubs/cbm/phi2001.html. ‘Current Bibliographies in Medicine: Public Health Informatics.’ National Library of Medicine. Accessed 23 July 2001. Updated April 2001.
http://www.nnlm.nlm.nih.gov/partners/. ‘Partners in Information Access for Public Health Professionals.’ Accessed 7 October 2000. Updated 23 March 2000.
http://www.ovid.com/. ‘Ovid Technologies.’ Ovid Technologies. Accessed 7 October 2000.
http://www.paho.org/English/SHA/bsindcvr.htm. ‘HEALTH SITUATION IN THE AMERICAS: BASIC INDICATORS 1995–1998.’ Pan American Health Organization. Accessed 6 October 2000.
http://www.paho.org/English/SHA/ihomeibs.htm. ‘Pan American Health Organization.’ Accessed 7 October 2000.
http://www.peoplesguide.org/. ‘L.A. Coalition to End Hunger and Homelessness.’ LA Coalition to End Hunger and Homelessness. Accessed 7 October 2000.
http://www.ph.ucla.edu/epi/snow.html. ‘John Snow.’ Department of Epidemiology, UCLA School of Public Health. Accessed 23 July 2001.
http://www.rti.org/patents/sudaan/sudaan.html. ‘SUDAAN.’ SUDAAN. Accessed 6 October 2000. Updated 31 March 2000.
http://www.sas.com/. ‘SAS.’ SAS Institute. Accessed 6 October 2000.
http://www.spss.com/spss10/index.htm. ‘SPSS.’ SPSS. Accessed 6 October 2000.
http://www.stata.com/. ‘STATA.’ STATA. Accessed 6 October 2000.
http://www.uiowa.edu/~geog/health/. ‘Geographical Information Systems and Public Health.’ The University of Iowa. Accessed 23 July 2001.
http://www.webmd.com/. ‘WebMD.’ WebMD. Accessed 6 October 2000.