Question answering
for general practitioners
An information presentation module for the IMIX demonstrator
Question answering
for general practitioners
An information presentation module for the IMIX demonstrator
M.C.G. van Langen
Universiteit Twente, October 2005
Business Information Technology
Electrical Engineering, Mathematics and Computer Science (EEMCS)
Language, Knowledge and Interaction (TKI)
Information Systems and Change Management (IS&CM)
dr. M. Theune (TKI)
dr.ir. H.J.A. op den Akker (TKI)
dr.ir. A.A.M. Spil (IS&CM)
Preface
This paper is my master's thesis for the study Business Information Technology at the
University of Twente. My assignment was part of the IMIX project. This is an NWO research
program aiming at the development of knowledge and technology needed to find specific
answers to specific questions in Dutch documents. I was allowed to choose a research subject
relevant for the IMIX project myself.
During my study I have focused on both language technology and healthcare. I thought it would be nice to integrate these subjects in my master's thesis. The IMIX project provided a very good context for this combination, because the results of this project are integrated in a question answering system (which incorporates a lot of language technology) for the medical domain.
In concert with my supervisors I decided to investigate the suitability of such a question answering system for professional use by general practitioners, and to develop part of the language technology that would be needed to accommodate the requirements of this special group. This assignment turned out to be a great combination of all subjects I was confronted with during my study. I hope you enjoy reading it!
Ede, October 2005
Mieke van Langen
Dankwoord (in Dutch)
Mijn afstudeeropdracht is redelijk soepel verlopen. Hoewel het een groot project was en ik veel
alleen werkte omdat ik intern afstudeerde, heb ik mij aardig aan de planning kunnen houden en
ligt hier een verslag waar ik erg tevreden mee ben. Dit was echter niet mogelijk geweest zonder
hulp van anderen. Ik ben dan ook heel blij met de steun en medewerking die zoveel mensen mij
bij deze opdracht hebben verleend. Ten eerste zijn dat natuurlijk mijn afstudeerbegeleiders die
uitgebreid de tijd genomen hebben om mij te helpen een goed onderwerp te kiezen, en ook
daarna veel tijd en moeite besteed hebben aan het kritisch lezen en becommentariëren van
mijn verslagen.
Ten tweede ben ik veel dank verschuldigd aan Anita Verhoeven en de huisartsen die ik heb mogen interviewen. Anita Verhoeven heeft mij gastvrij ontvangen in Groningen, en mij veel duidelijk gemaakt over de mogelijkheden voor en de praktijk van het informatie zoeken van huisartsen. De vijf huisartsen die ik (sommige zelfs twee keer) geïnterviewd heb voor mijn onderzoek, ben ik erg dankbaar voor de tijd die ze voor mij hebben kunnen vrijmaken, zeker gezien de tijdsdruk waaronder zij momenteel moeten werken.
Ten derde heb ik bij het evalueren van mijn ontwerp veel hulp gehad van Wauter Bosma en alle proefpersonen die mijn vragenlijst ingevuld hebben. Wauter Bosma had toegang tot een werkende versie van de IMIX demonstrator en was steeds bereid antwoorden op mijn voorbeeldvragen te genereren.
En "last", maar zeker niet "least" zijn daar mijn vriend en mijn ouders. Zij hebben mij niet alleen tijdens mijn afstudeeropdracht, maar ook gedurende de rest van mijn studie steeds in alle opzichten gesteund. Mijn studietijd was door mijn gezondheidsproblemen geen gemakkelijke periode, maar dankzij hun steun heb ik mijn studie toch kunnen afronden en kan ik nu, gezond en wel, als ingenieur een nieuwe start maken.
Allemaal heel erg bedankt!
Executive summary
This research was part of the IMIX (Interactive Multimodal Information eXtraction) project. This
project is an NWO (Netherlands Organization for Scientific Research) research program aiming
at the development of knowledge and technology needed to find specific answers to specific
questions in Dutch documents. The results of the IMIX project are integrated in an interactive
multimodal question answering system for the medical domain. In their work, general
practitioners are confronted with large quantities of and needs for information. Therefore, it was
investigated whether such a medical question answering system would be suitable for
professional use by general practitioners, and how answers should be presented for this user
group.
Based on literature research and interviews with general practitioners, it was concluded that a medical question answering system could primarily be used by general practitioners for answering questions for patient education. Such a question answering system should be accessible via the Internet, it should search for answers only in information sources that were marked as reliable by medical professionals, its response time should be short enough to enable use during medical consultations, and it should recognize ICPC coding and other medical slang in the question. Further, to be able to use this system, general practitioners must have access to a computer with Internet connection and a printer in their consulting rooms. In this way, they can search for answers on their questions during the medical consultation and possibly print the answer and give it to the patient. Besides, a web portal for general practitioners has been designed that could help general practitioners in keeping an overview of all different information types they can find on the Internet.
Answers retrieved by a question answering system for general practitioners should be presented together with a link to their sources, and a checkbox to enable the general practitioner to indicate whether he wants it to be printed or not. In this way different answers on the same question retrieved from different sources can be integrated into one consistent view. Besides, algorithms have been developed that integrate different answers retrieved from the same source into one concise answer, possibly extended with sentences from their context.
Finally, it was found that general practitioners would also like to have information technology to search for dermatological images, and for contact information of regional health professionals and medical organizations. It is therefore recommended that the applicability of image retrieval and information extraction technology for general practitioners is also investigated.
Managementsamenvatting (in Dutch)
Dit onderzoek maakt onderdeel uit van het IMIX (Interactive Multimodal Information eXtraction)
project. Dit is een onderzoeksprogramma van het NWO (Nederlandse Organisatie voor
Wetenschappelijk Onderzoek) dat zich ten doel stelt om kennis en technologie te ontwikkelen
die nodig zijn om specifieke antwoorden op specifieke vragen in Nederlandstalige documenten
te vinden. The resultaten van het IMIX project worden geïntegreerd in een interactief
multimodaal question answering systeem voor het medische domein. Omdat huisartsen
behoefte hebben aan en gebruik kunnen maken van een grote hoeveelheid en verscheidenheid
aan informatie, is onderzocht of zo'n question answering systeem geschikt zou zijn voor
professioneel gebruik door huisartsen, en hoe antwoorden door dit systeem gepresenteerd
zouden moeten worden voor deze gebruikersgroep.
Op basis van literatuuronderzoek en interviews met huisartsen, wordt geconcludeerd dat een medisch question answering systeem voor huisartsen vooral geschikt zou zijn om vragen voor patiëntenvoorlichting te beantwoorden. Zo'n question answering systeem zou beschikbaar moeten zijn via het Internet, het zou alleen naar antwoorden moeten zoeken in informatiebronnen waarvan medici hebben aangegeven dat ze betrouwbaar zijn, de reactietijd van het systeem zou dusdanig kort moeten zijn dat vragen beantwoord kunnen worden tijdens het consult, en het systeem moet medische terminologie en ICPC codes in de vraag kunnen begrijpen. Daarnaast moeten huisartsen, om zo'n systeem te kunnen gebruiken, een computer met Internetverbinding en een printer in hun spreekkamer hebben. Op die manier kunnen ze hun vragen tijdens het consult beantwoorden en eventueel de antwoorden ook uitprinten om ze mee te geven aan de patiënt. Verder is er een web portal gemaakt om huisartsen een overzicht te bieden van alle verschillende soorten informatie die ze op het Internet kunnen vinden.
Antwoorden die door een question answering systeem voor huisartsen gevonden worden, moeten gepresenteerd worden met links naar de bronnen waarin ze gevonden zijn en aankruisvakjes zodat de huisarts per antwoord kan aangeven of hij het wil uitprinten. Op die manier worden antwoorden op dezelfde vraag die in verschillende bronnen gevonden zijn, geïntegreerd in één overzicht. Daarnaast zijn er algoritmes ontwikkeld waarmee verschillende antwoorden die uit dezelfde bron komen geïntegreerd kunnen worden in één beknopt antwoord en antwoorden eventueel uitgebreid kunnen worden met extra zinnen uit de omgeving van het antwoord.
Tenslotte blijken huisartsen ook behoefte te hebben aan informatietechnologie waarmee ze naar dermatologische plaatjes en naar adresgegevens van plaatselijke gezondheidszorgorganisaties kunnen zoeken. Daarom wordt aanbevolen om ook de geschiktheid van image retrieval en information extraction technologie voor gebruik door huisartsen te onderzoeken en systemen te ontwikkelen waarmee deze informatie kan worden gevonden.
Contents
Question answering for general practitioners
1 Introduction
During and after consultations, general practitioners use a lot of information. Next to the
information they receive from the patient, they also look up information on controversial or rare
topics, diagnosis, treatment and investigations, and information for patient education [MCW05].
This information is needed not only in consulting rooms, but also during patient visits.
Due to the rise of evidence based medicine the use of medical knowledge by general practitioners has become even more important, but the amount of medical information also increases rapidly [VER99]. To address the problem of quickly finding the relevant information among these large document collections, intelligent information technology is needed. In this master's thesis it is investigated how question answering technology can help general practitioners meeting their information needs.
Context of the research
This research is part of the IMIX project. This project concerns the development of question and related work on question answering relevant for description of the IMIX project.
1.1.1 Question
answering
A question answering (QA) system "accepts questions in natural language form, searches for answers over a collection of documents and extracts and formulates concise answers" [MS03]. A general QA system architecture consists
• question analysis;
• document retrieval;
• answer extraction;
• answer selection [JMR03]; and
• response formulation [MS03].
(Graphical) user interface
Figure 1 General question answering system architecture
The question analysis component transforms a natural language question into a retrieval query and classifies the question with respect to its expected answer type (e.g. the name of a person,
Mieke van Langen
a date, a location, etc.). The document retrieval component takes the retrieval query as input and returns a set of documents relevant for the query. These documents are retrieved from a document collection with the aid of an indexer. The answer extraction component extracts possible answers from the retrieved documents. The answer selection component returns a ranked list of the extracted answers. Finally, the response formulation component formulates a natural language response to the natural language question. All five components possibly make use of (natural language processing) resources. A (graphical) user interface may be used to facilitate the interaction between the user and the QA system.
Compared to traditional information retrieval (IR) technology, QA systems only return concise answers to the user, instead of entire documents in which the user has to find the answer himself. Actually, IR systems are used as document retrieval component in QA systems. QA thus extends IR.
Two attempts are being made at developing open domain QA systems for Dutch. ‘Open domain' refers to the document collection from which the answers are extracted. An open domain QA system could search in any type of document collection for answers on all possible domains. A ‘closed domain' QA system is targeted at documents on a specific domain and only answers questions on this domain, like for example medical questions. One of the open domain Dutch QA projects, named "Question answering for Dutch using dependency relations", is executed at the Rijksuniversiteit Groningen [BOU03]. This project makes use of existing QA technology combined with dependency analysis based on full syntactic parsing of both the question and the potential answer fragments. For this purpose the Alpino Dependency Parser for Dutch [BNM01] is used. The other Dutch project, executed at the University of Amsterdam, has resulted in a multi-stream architecture for question answering [JMR03]. In this architecture each stream represents a different approach for QA, such as table lookup, pattern matching, an existing QA system for English combined with automatic translation, and web answering. Each stream has its own strengths and thus suits some question types more than others. The system's final answer is taken from the combined pool of answers generated by the suitable QA streams. Both projects don't concentrate on response formulation. The result of these QA systems is thus a ranked list of answers returned by the answer selection component.
1.1.2 The IMIX project The IMIX (Interactive Multimodal Information eXtraction) project is an NWO (Netherlands Organization for Scientific Research) research program aiming at the development of knowledge and technology needed to find specific answers to specific questions in Dutch documents [NWOa]. The results of this research program are integrated in the IMIX demonstrator. This demonstrator is an interactive multimodal QA system for the medical domain. It consists of two parts: a text based part covering the complete medical domain and a multimodal part focusing on the RSI domain only [OS04]. The demonstrator is targeted towards naïve users who have no knowledge of the domain and little technical knowledge [VP05]. Compared to the other Dutch QA systems described in the previous section, the IMIX demonstrator has a restricted domain. However, in contrast with those systems, it incorporates modules for response formulation and speech input and output.
IMOGEN (Interactive Multimodal Output Generation) is a part of the IMIX project [NWOb]. It aims at the development of multimodal information presentation modules for the output of QA systems. Next to response formulation, these modules include speech generation and the use of graphics. The information presentation modules developed for IMOGEN constitute the IMOGEN demonstrator, which is part of the IMIX demonstrator. The IMOGEN sub modules will firstly be developed for the RSI domain only.
To investigate whether a QA system like the IMIX demonstrator is also suitable for professional use, in this master's thesis an IMOGEN sub module is developed that is targeted towards general practitioners. The input for this module consists of ranked lists of answers (produced by
Question answering for general practitioners
the IMIX demonstrator's question answering modules). The output is a presentation of the answers that would be most convenient for Dutch general practitioners.
1.2 Research
question
The research question to be answered in this master's thesis is:
Which information needs of Dutch general practitioners can be satisfied by a question
answering system and how should the answers be presented?
To answer this research question, the following sub questions have to be answered:
1. What are the information needs of Dutch general practitioners? 2. How are these information needs satisfied now? 3. How do Dutch general practitioners use and appreciate computers in their work?
The integration of the IMOGEN sub module built for this research and currently used general practitioner information systems falls outside the scope of this research. The sub module will not be implemented in general practices, because the IMIX demonstrator only intends to show the status, progress, and results of the research carried out in the IMIX program [NWOa]. The sub module is thus only developed to show whether and under which conditions a system like the IMIX demonstrator suits professional use.
1.3 Research
To answer the research questions described above, information is needed on the information needs and use of Dutch general practitioners and on the way they use and appreciate computers in their work. A lot of research has been executed on the information seeking behavior of professionals from different disciplines. Leckie et al. [LPS96] developed a model of the information seeking of professionals derived from research on engineers, health care professionals, and lawyers. This model was found to be appropriate to explain the information seeking behavior of Dutch general practitioners [
Characteristics of information needs
Figure 2 A model of the information seeking of professionals
Mieke van Langen
According to this model, work roles and the related tasks undertaken by professionals prompt particular information needs, which in turn give rise to the information seeking process. The information needs arising form a specific task are influenced by a number of variables, including factors relating to the individual (age, profession, specialization, career stage, geographic location, etc.), and general characteristics of the information needs. Examples of these characteristics are: context (internally or externally prompted), frequency (recurring or new), predictability (anticipated or unexpected), importance (urgency), and complexity.
The information seeking process elicited by an information need is influenced (as depicted in the model) by the sources of information, the awareness of information, and the outcomes. Besides, the outcomes may also influence the sources of information and the awareness of information. Sources of information can be formal or informal, internal or external, oral or written, and personal (own knowledge and experience). Awareness of information refers to direct or indirect knowledge of the various information sources and the perceptions about the information seeking process or about the retrieved information. Variables of this factor include familiarity and prior success with a certain search strategy or information source, trustworthiness, packaging (medium or format), timeliness, cost (financial, psychological, physical), quality, and accessibility (physical proximity, language). Outcomes are the results of the information seeking process. The optimal outcome is that the information need is met and the professional accomplishes his task. However, the outcome may also be that the information need is not satisfied and further information seeking is required. In this case feedback is provided (possibly altering the factors influencing the information seeking process), and a second round of information seeking is undertaken. It is also possible that an outcome from one task associated with a specific role unexpectedly benefits the professional in another role.
A QA system would be one of the possible sources of information, thus influencing the information seeking process. Its goal would be to make the information seeking process more efficient and to improve the outcomes. Therefore, it should improve some of the variables related to the awareness of information. The QA system is also an information seeker itself, however. Its performance is thus in turn influenced by the information sources it uses itself. The relation of the QA system to the information seekthe QA system is positioned inside the sources of information ellipse, to emphasize that it would be only one of the information sources a general practitioner could use. This source can be accessed via a user interface, which is depicted separately. The information sources used by the QA system are also positioned within the larger sources of information ellipse, because these are sources that are probably already available for general practitioners. It is expected that only a subset of the general practitioners' information needs can be met by a QA system. This is depicted by a box questions within the information needs box. The outcomes a QA system returns are also a subset of all search results, depicted by the answers box within the outcomes box.
To answer the research question, it must firstly be investigated for which work roles, tasks, and information needs of Dutch general practitioners a QA system could support the information seeking process, and which variables related to the awareness of information could be improved by a QA system as opposed to other information sources. Besides, because a QA system is an information seeker itself, it must be investigated which information sources it could consult, and how it should search these sources for relevant information. Finally, because a QA system runs on a computer, it must be investigated how Dutch general practitioners use and appreciate computers in their work. To answer these questions, literature on general practitioners and on medical informatics has been reviewed. Besides, interviews have been conducted with a few Dutch general practitioners and an expert on information use by Dutch general practitioners.
Secondly, based on the findings from literature and the interviews, requirements have been specified for the IMIX demonstrator, and a design has been made for an IMOGEN sub module for general practitioners consisting of a response formulation component and a graphical user
Question answering for general practitioners
interface. Prototypes have been constructed for both components. These prototypes have been evaluated by naïve users and a subset of the general practitioners who participated in the previous interviews.
Characteristics of information needs
Sources of information
Figure 3 A QA system integrated with the model of information seeking
Finally, based on the results of the evaluation of the information presentation module, conclusions were drawn and recommendations were made concerning the conditions under which a system like the IMIX demonstrator would suit professional use.
Structure of the paper
This existing literature on general practitioners' work roles and tasks, information needs, information sources, and computer use is described. The interviews conducted with Dutch general practitithe results of the literature research and the interviews are related to the design of the IMIX demonstrator, and the design for the information presentation module for general practitioners is e formulation technology developed for the information he results of this research are evaluated. And this research are presented and discussed.
Mieke van Langen
Question answering for general practitioners
2 Literature on information use by general practitioners
Like in all areas, in medical sciences the volume of information grows exponentially [VER99].
Detmer and Shortliffe [DS97] stated already in 1997 that every year more than 360,000 articles
are published in medical journals, making knowledge diffusion to physicians rather slow. They
refer to a study, which found that two years after wide publication, only 50% of the general
practitioners knew that laser surgery could save the sight of some of their diabetic patients.
Westberg and Miller [WM99] state that "because of the ever-increasing size of biomedical
literature and the complexity of modern health care practices, physicians could spend ours to
weeks reading texts and seeking expert opinions for each patient they encounter." It is thus
increasingly difficult, but also increasingly important for physicians to find the information they
need.
In this chapter the information use by general prwork roles and tasks of Dutch general practitioners are described. The information needs of general practitioners are described in the next section. the role and use of computers in the general practice are described. Finally, conclusions are drawn with respect to the possibility of using a question answering (QA) system for information seeking by general practitioners.
The general practice
This research especially deals with Dutch general practitioners. Dutch general practitioners work in solo practices, in duo or group practices, or in primary health care centers. In contrast to their American colleagues, they never work in hospitals [VER99]. The Dutch general practitioner acts as a gatekeeper to secondary care. He is therefore expected to manage a wide range of medical problems, giving rise to high information needs.
General practitioners not only see patients, but also have to learn, perform research, educate and manage. Verhoeven [VER99] discerns five different roles of general practitioners, see ent tasks and thus with different information needs.
Work role
Service provider
Professional reading, attending conferences and meetings
Writing publications, speaking at conferences
Planning, curriculum development
Administrator/manager Managing own practice
Table 1 Work roles and tasks of general practitioners
The role of service provider is common to all professionals [LPS96]. Physicians spend most of their time in this role and the tasks associated with patient care create their greatest need for information. Professionals also have a role of learner. They have to keep up with the advancements in their field, and upgrade their education and skills by taking courses [LPS96]. Tasks associated with this role include professional reading, and attending conferences and meetings. The third role, researcher, is not performed by all general practitioners. Most Dutch general practitioners primarily provide patient care. Only some of them combine this with research. Tasks associated with the role of researcher are writing publications and speaking at conferences. As an educator, general practitioners teach medical students and general practice trainees. Tasks associated with this role include planning and curriculum development. Finally, as an administrator and manager, general practitioners have to manage their own practice.
Mieke van Langen
In this research only the role of service provider is considered, because this is the rolein which medical questions may arise that should be answered quickly, thus making aquestion answering system potentially useful. In the roles of learner and researcherthe general practitioner also wants medical questions to be answered, but in this casegenerally complete articles are needed to (scientifically) answer these questions, notjust concise answers.
The work as service provider consists of medical consultations both in the general practitioner's consulting room and at patients' homes. Medical consultations generally consist of the following phases [DN96]:
data gathering and recording; searching databases (medical records, suitable drugs, etc.); choosing a course of action; documentation; providing explanations; and arranging any future consultations.
The phase of data gathering and recording provides the input for the phase of searching databases. The latter phase especially concerns searching information and might thus be supported by a QA system. The outcome of this phase is used to support the phases of choosing a course of action and providing explanations to the patient.
In principle, patient visits consist of the same phases as consultations in the consulting room. However, during patient visits the general practitioner cannot make use of the same resources he uses in his consulting room. Therefore, the phases of searching databases and documentation might be a little harder, also complicating the phases of choosing a course of action and providing explanations.
A mobile device containing a question answering system could really improve thegeneral practitioner's possibilities of searching databases during patient visits and therefore potentially also improve patient care.
2.2 Information
A lot of research has been done on the needs and use of medical knowledge by general practitioners. Quantitative estimates of the information needs of physicians in their role of service provider vary greatly, however. Different studies result in different estimates because they differ on definition of terms, subjects, setting, and method of data collection. Gorman [GOR95] tries to structure these studies by defining different types of information and different types of information needs. The information titioners encounter a lot of obstacles when they try to address their information needs, however. The factors determining whether an information need is pursued and satisfied or not
The information needs described in the following sections are only those of the general practitioner in his role of service provider. Therefore, only questions arising during medical consultations are taken into account. Information needs that are met by regularly reading medical journals or randomly "browsing" for information without a real question in mind are considered to be the information needs of the general practitioner as a learner.
2.2.1 Types of information Gorman [GOR95] identifies five types of information used type, patient data, refers to information about a specific person. It includes the patient's medical
Question answering for general practitioners
history, observations from physical examination, and results of diagnostic testing. This information is usually obtained from the patient himself, his family and friends, and the medical record. These data fall outside the scope of a potential QA system, for it is not convenient to consult the patient by a QA system and it is assumed that electronic medical records are well enough organized to make a QA system superfluous. Patient data might be included, however, in the questions a general practitioner would submit to a QA system.
Type of information Description
Refer to a single person
Medical knowledge
Generalizable to many persons
Population statistics
Aggregate patient data
Logistic information
How to get the job done
Social influences
How others get the job done
Table 2 Information types
Medical knowledge is general information that is applicable to the care of all patients. It includes scientific medical knowledge, but also the accumulated informal experience of the general practitioner. Medical knowledge could be sub classified according to classic textbook categories (etiology, pathophysiology, clinical manifestations, diagnosis and differential diagnosis, treatment, and prevention) or according to organ system domain categories (dermatology, rheumatology, neurology, etc.). Next to the classic textbook categories Magrabi et al. [MCW05] discern a separate category, namely patient education. Questions about patient education deal with the need for information to better inform patients about their conditions or to increase their compliance with the treatment. Whereas information on for example etiology, diagnosis, or treatment is used to support the medical consultation phase of choosing a course of action, information for patient education is used for the phase of providing explanations.
Population statistics refer to aggregated data about groups or populations of patients. This includes formal population statistics, but physicians also use their personal knowledge of recent illness patterns in the community as a form of informal epidemiological information.
Logistic information refers to local knowledge about how to get the job done, often specific to a practice setting or payment mechanism. As examples Gorman [GOR95] mentions information about required forms, coverance by insurers, and referral lists of medical care organizations (which is typical for the American situation). He doesn't mention, however, information about which physician a patient should be referred to from a medical instead of an economical point of view. I assume general practitioners sometimes need to find out which physician performs a particular treatment he thinks is convenient for a particular patient. This information seems to be on the boundary of medical knowledge and logistic information. Logistic information is usually local and can best be obtained from human sources such as office and hospital staff or colleagues. This type of information is therefore not suitable for a QA system.
Social influences refer to knowledge about the expectations and beliefs of others, especially colleagues, but also patients, families, and others in the community. This type of information can evidently not be provided by a QA system. It may be of influence, however, on the general practitioner's behavior concerning a QA system, but this falls outside the scope of this research.
Mieke van Langen
Medical knowledge and population statistics can surely be dealt with by a questionanswering system. Actually, the IMIX demonstrator used for this research [NWOa] is aquestion answering system dealing with questions about medical knowledge andpopulation statistics. Its document collection includes the Spectrum MedicalEncyclopedia (aiming at the general public) and the Merck medical data (aiming atmedical professionals as well as the general public). Besides, for the RSI domain,additional data are obtained from (among others) the RSI patient association, TNO Arbeid, Arbobondgenoten, Ergo-Direct, and Stichting RSI Nederland.
2.2.2 Types of information needs Next to types of information, Gorman [GOR95] also identifies different types of information unrecognized needs. These can be inferred from measurement of physician knowledge or observation of clinical practices. Information systems that depend on the physician to seek information can't succeed until the physician recognizes that a need exists. To address unrecognized needs, information systems should be designed to do so, for example by issuing automatic reminders or by automatically informing physicians of additional diagnostic possibilities not initially considered by the physician. When a QA system would be used for this purpose, it should thus extract implicit questions from the data the general practitioner enters into the general practitioner information system during the phase of data gathering and recording. The extraction of implicit questions falls outside the scope of this research however.
Type of information need
Description
Unrecognized need
The physician is not aware of the information need or knowledge deficit
The physician is aware that information is needed
Information seeking occurs
Information seeking succeeds
Table 3 Types of information needs
Secondly, Gorman identifies recognized needs. These are needs articulated by the physician. A question being articulated by a physician doesn't guarantee, however, that the answer is actually necessary to benefit the patient or the practitioner, in other words, that it is really a ‘need'. Recognized needs for which some information seeking behavior is executed are called pursued needs. If the pursuit of a particular need is successful, this need is also called a satisfied need.
A question answering system can only answer pursued needs, because it cannot read the physician's mind for recognized needs. The general practitioner should enter thequestion into the system himself, making it a pursued need. The aim of the questionanswering system would be to answer the entered question, in other words to turn the pursued need into a satisfied need. Besides, it should be designed to tempt thegeneral practitioner to enter all his medical questions, turning as much as possiblerecognized needs into pursued needs.
Apart from the definitions for information and information needs, the definition of a question is also tricky, because medical questions tend to be multi-factorial. They can contain questions
Question answering for general practitioners
within a question [GOR95]. Most of them are complex, and patient-, problem-, and practitioner-specific. Physicians therefore usually first need to tell the patient's story, to explain the context of the question. These stories often contain information from several of the five different information types. See Appendix A for a sample of typical questions asked by general practitioners.
Results from different studies on the information needs of physicians can apparently differ greatly when different definitions for information and information needs are used. Estimates for general practitioners range from 0.07 to 1.8 questions per patient encounter [GOR95]. In a questionnaire executed among 226 Dutch general practitioners in 1996 [VNB99] general practitioners indicated that questions for which they needed answers arose 6.9 times a week (the amounts ranged from 0.04 to 50). These general practitioners were also asked to record their most recent question, which is thus a recognized information need. Of these questions 50.5% dealt with therapeutic problems, and 24.8% with diagnostic problems. Circulatory, musculoskeletal, and digestive were the top three system domain categories the questions dealt with.
Most other studies were executed in English speaking countries. Following, a few of their findings relevant for a Dutch QA system are described. Magrabi et al. [MCW05] analyzed the queries Australian general practitioners submitted to an experimental online evidence system, which were thus pursued information needs. This online evidence system was an information retrieval system in which users not only had to enter keywords, but could also select a search filter concerning the type of question (disease etiology, diagnosis, treatment, prescribing, or patient education). In this study 43% of the questions dealt with therapeutic problems (35% with treatment and 8% with prescribing), 40% with diagnostic problems, 10% with patient education, and 7% with disease etiology. Gastrointestinal, dermatology, and musculoskeletal were the top three system domain categories for which information was searched. A drawback of this study is that it only deals with the questions pursued with this online evidence system. Questions that were not pursued or that were answered with other means were not considered. However, the questions pursued with an online evidence system probably resemble those that could be answered by a QA system, because both systems can only make use of electronic resources.
Ely et al. [EOE99] collected 1101 questions (recognized information needs) from 103 American general practitioners. They searched these data for generic questions. The most frequently used question structures were "What is the cause of symptom X?", "What is the dose of drug X?", "How should I manage disease or finding X?", "How should I treat finding or disease X?", and "What is the cause of physical finding X?". Besides, Ely et al. found that older patients and female patients elicited more questions than younger and male patients respectively, and that younger physicians asked more questions than their older colleagues.
Barrie and Ward [BW97] collected 85 medical questions (recognized information needs) from 27 Australian general practitioners. They found that physicians in solo or duo practices asked significantly fewer questions per consultation than those in larger practices.
For a Dutch question answering system this means questions can be expected (indescending order of frequency) on diagnostic problems, treatment, patient education,prescribing, and disease etiology. Gastrointestinal, dermatology, musculoskeletal, andcirculatory will likely be the most frequent system domain categories. Knowledge offrequently used generic questions can be used for the design of the question analysiscomponent of the system. Finally, it is expected that younger physicians andphysicians working in larger practices will be more likely to use a question answeringsystem, because they generally have more questions than older general practitionersand general practitioners working in solo or duo practices.
Mieke van Langen
2.2.3 Pursuing information needs In a questionnaire executed among 226 Dutch general practitioners in 1996 [VNB99] general practitioners were found to immediately pursue an answer to their questions in 76% of the cases and in 85% of these cases a (partial) answer was found. This means that 65% of all recognized information needs were turned into satisfied needs, and 24% of the recognized information needs were not even pursued. Gorman and Helfand [GH95] even found that 70% of the questions arising in general practice are never pursued. This is quite a large difference. According to Gorman [GOR95], this might be due to differences in the definition of terms, subjects, setting, and method of data collection. Both studies examined general practitioners not working in hospitals, and both studies concerned recognized information needs about medical knowledge. However, Gorman and Helfand [GH95] observed American general practitioners and recorded their questions during patient care, while Verhoeven et al. [VNB99] collected Dutch general practitioners' questions by sending them a questionnaire in which they were asked to record their most recent question and whether or not they pursued this question. This is a great difference in method of data collection. In Verhoeven's research general practitioners might have been tempted to record an information need that they pursued because that's what they remember or because of social influences, resulting in a much higher rate of pursued questions.
Gorman and Helfand [GH95] found two factors that predicted the pursuit of information needs: the physician's belief that a definitive answer existed, and the urgency of the patient's problem. Ely et al. [EOE99] collected 1101 questions from 103 American general practitioners. They found that only questions about drug dose were routinely pursued and that an answer was found to 80% of the pursued questions. Both findings are consistent with those of Gorman and Helfand.
In the literature lots of different barriers are identified that complicate the search for information by general practitioners. Apart from lack of time and information overload, Ely et al. [EOE02] have identified different obstacles for each of the following steps in asking and answering questions:
recognizing an information gap; question formulation; searching for relevant information; answer formulation; and using the answer to direct patient care.
In the research of Ely et al. the general practitioners only performed the first two steps and the last step themselves. Information searching, and answer formulation were done by experts who tried to answer questions generated by general practitioners. These are exactly the steps that would be executed by a QA system. The obstacles found for these steps are therefore highly important for this research.
The obstacles identified by Ely et al. [EOE02] arrelated to recognizing an information gap deal with the transformation of an unrecognized need into a recognized need. Sometimes physicians are unaware of a gap in knowledge when they make a decision. In this case they have an information need, but they don't recognize it. They might also suppress a recognized information need because of time pressure, embarrassment, personal characteristics, or characteristics of the clinical setting.
Question formulation refers to modifying the question in order to be able to find relevant literature. For example, patient specific questions should be generalized, patient data could be added to focus the search, potential supplementary questions could be anticipated for, specific words might be changed, etc. When a QA system is used for answering medical questions, a dialogue might be needed to overcome the obstacles related to this step.
Question answering for general practitioners
Six different sorts of obstacles related to the searching for relevant information step are identified. The first is failure to initiate the search. Reasons for not pursuing information needs include doubt about the existence of relevant information, insufficient justification (when the question is not important enough to justify a search), lack of time, and the availability of consultation (sometimes general practitioners just refer patients to specialists rather than learn enough about the problem to manage it themselves). Detmer and Shortliffe [DS97] also mention the ignorance of the availability of relevant information as a reason for not pursuing information needs.
Steps in asking and answering
Obstacles
questions
Recognizing an information gap
Lack of awareness of an information need Suppression of a recognized information need
Question formulation
Inability to answer patient specific questions with
general resources
Missing patient data Uncertainty about the scope Difficulties modifying the question
Searching for relevant information
Failure to initiate the search Uncertainty about the searching strategy Inadequate (availability of) resources Inadequate information Inadequate evidence Inadequate use of evidence
Answer formulation
Failure to directly or completely answer the
Too long or too short answer Answer directed at the wrong audience Difficulty addressing unrecognized information
Discomfort with formulating an answer to be used
Using the answer to direct patient care
Answer not trusted Answer no longer needed Answer inadequate
Table 4 Obstacles to answering medical questions
Secondly, general practitioners may be uncertain about the right searching strategy. They may have difficulties with selecting the appropriate resources, be uncertain about how to know when all the relevant evidence has been found so that the search can stop, don't know the meaning of null search results, etc. The meaning of null search results is also important when developing a QA system, because when no articles are found on a certain treatment or when a relevant article doesn't mention the treatment, this doesn't necessarily mean that there is no treatment, but sometimes a null search result might be a clear answer.
Thirdly, resources might be inadequate. They might be badly accessible, badly indexed, poorly organized, not clinically oriented, not trusted, not current, not allowing real time interaction with
Mieke van Langen
the searcher, or a certain topic might not be included in a resource that should logically include it. Other obstacles are inappropriate descriptors of resources [VBM95], the cost of resources, difficulties learning or using many resources, and variable quality of the information [WM99].
The fourth obstacle, inadequate information, deals with information that is incorrect, not current, vague, unnecessarily cautious, biased, or fails to anticipate supplementary information needs, differentiate between different diagnoses, define terms, or adequately describe clinical procedures. Verhoeven et al. [VBM95] also mention the overload of irrelevant information as an obstacle to finding the right answer.
The fifth and sixth group of obstacles related to the information searching concern the evidence. When studies don't address the medical question, don't compare the relevant treatments, or don't study the outcome or population of interest, they may deal with the right subject, but still not be relevant for the question. Besides, relevant evidence might be badly synthesized or hardly applicable.
Obstacles related to answer formulation include failure to directly or completely answer the question, too long or too short answers, answers directed at the wrong audience, and difficulty addressing unrecognized information needs that are evident in the question. Besides, non-physician searchers indicated they were not comfortable formulating an answer that would direct patient care.
Finally, the step using the answer to direct patient care was sometimes not executed, because the answers were not trusted, too late or inadequate.
Different suggestions to overcome all these obstacles are summarized in literature. Verhoeven et al. [VBM95] suggest improved accessibility of information resources by computerization, education in the use of information sources, and improved accessibility to library facilities. Besides, she argues journal articles should be tailored more to the general practitioner's daily work. Magrabi et al. [MCW05] suggest that search systems for electronic resources should be preprogrammed with specialist bibliographic knowledge to save the physician's time. For example, the online evidence system investigated by Magrabi et al. used search filters (such as ‘diagnosis' or ‘treatment') that added specialist keywords to the query entered by a general practitioner that have been shown to significantly enhance the quality of search results, but are unlikely to be known by the general practitioner.
Some of these solutions can't go without each other. For example, computerization has the potential to offer general practitioners access to loads of information, but studies indicate that general practitioners have difficulty finding the most relevant resources and selecting the appropriate search terms [WM99]. Therefore, also education in the use of resources and/or search programs with specialist bibliographic knowledge are needed to make computerization a real solution. QA could also make computerization a solution, because it eliminates the need for education in the use of resources (for the only thing the physician has to do is entering a question in natural language). Besides, a QA system could also incorporate bibliographic knowledge for searching the right information.
Ely et al. [EOE02] think authors should anticipate the needs of busy physicians. For example, when authors name a certain drug, they could include essential prescribing information, because this may be an unrecognized or supplementary information need when a physician has a question about this drug; resources could be written in a question and answer style; resources should be kept current by the ongoing surveillance of physicians' changing questions; and research should be initiated and funded based on questions without adequate answers. These are all issues that could not be solved by a QA system. Ely et al. also indicate however, that the modification of questions from the way they were originally stated by the general practitioners often proved very helpful for searching the right information. This is an issue that might be addressed by a QA system.
Question answering for general practitioners
A question answering system could deal with the following obstacles:
obstacles related to question formulation could be overcome by modifying the
query, possibly using a dialogue;
because a question answering system is an information seeker itself, it could take
away the uncertainty about the search strategy to be followed by the general practitioner;
the system could improve the accessibility of other electronic resources, because
it eliminates the need for general practitioners to directly interact with thoseresources;
problems with poorly organized information could be accounted for, when the
system adequately synthesizes information from different sources.
However, most of the obstacles regarding information searching remain when aquestion answering system is used, because they are inherent to the resources used. The obstacles related to answer formulation could only be overcome by a questionanswering system when questions are correctly interpreted and the right resourcesare used. Besides, the system should correctly interpret null search results.
2.3 Information
When general practitioners pursue their information needs, they can use a lot of different information sources. In this section only the sources of medical knowledge and population statistics are considered, because these are the information types that are relevant for a QA system. In the first subsection different types of information sources are discussed. In section e based medicine used by general practitioners are described. Finally, the general practitioner's information seeking behavior with respect to the different information resources (influenced by the awareness of information) is discussed in section
2.3.1 Types of information sources For medical knowledge and population statistics three different types of information sources can Printed sources include general practitioners' own books and journals (their private medical libraries), but Dutch general practitioners may also address the libraries of the local hospitals they refer their patients to. Besides, the Dutch Institute for Research of Health Care (NIVEL) in Utrecht provides medical information on demand for general practitioners from their own library, and the Royal Dutch Academy of Arts and Sciences (KNAW) in Amsterdam (owning the largest medical journal library in the Netherlands) provides journal articles to general practitioners [VER99].
Printed sources for patient education are provided by the Dutch College of General Practitioners (NHG), the Scientific Institute of Dutch Pharmacists (WINAp), patient organizations, associations of specialists, hospitals, drug manufacturers, etc. The Dutch College of General Practitioners publishes patient brochures and patient letters [NHGp]. Patient brochures provide general information on frequently occurring disorders and the measures the patient can take to prevent or cure them. Patient letters have been written especially for patients suffering a specific disease. These letters provide detailed information on the disease and its treatments.
Printed sources of formal population statistics include published descriptions of disease prevalence in the medical journal literature [GOR95].
Human resources include colleagues, specialists, and office staff. Specialists contain a lot of domain-specific knowledge, but their time is limited. Westberg and Miller [WM99] state that modern academic health care centers may be able to satisfy many of the general practitioners' information needs by providing Internet-mediated access to their electronic and human
Mieke van Langen
information sources. They propose a triage model for doing so. With this model information requests are firstly mapped to electronic resources. Only when a request doesn't seem to map well, access is provided to human resources. Human information on population statistics can be provided by public health departments [GOR95].
Information sources
Examples
Drug reference books, private books and journals, library books and journals, journal articles received from others
Other general practitioners, specialists, office staff
Electronic sources
Cd-rom, online databases, Internet
Table 5 Information sources for medical knowledge
Electronic sources include information on cd-rom or information that is accessible via a modem or Internet connection. The biggest electronic source is of course the World Wide Web. The World Wide Web potentially provides the general practitioner with all the latest information on medical issues. Physicians are often frustrated, however, by the difficulty in finding reliable and relevant information on the Web quickly [DS97]. They want resources that are practitioner oriented, produced by reputable sources, and cover a specific topic in medicine. Because of the Web's rapid growth and lack of controls, its organization is poor, and validity and reliability of sources found on the Web are questionable [WM99]. These shortcomings render a substantial amount of Web information unsuitable for direct clinical application. A strategy for use of the Web to support clinical practice could be locating and using anchors of known high quality [WM99]. The major Internet search systems are not discriminatory in what they index and their index methods are word based. Detmer and Shortliffe [DS97] argue that a medical retrieval system should make use of content based instead of word based index methods. Whereas word based index methods index documents by the words occurring in the documents, with content based index methods documents are indexed by a mostly fixed set of general terms (not necessarily occurring in the documents) describing what the documents are about. Documents could for example be indexed by the controlled-vocabulary terms from the Medical Subject Headings (MeSH). Besides, representation methods that add contextual information to portions of documents may help improve retrieval relevance by focusing retrieval in only relevant semantic regions.
One of the most mentioned sources of electronic medical information is Medline. This is a bibliographic database provided by the National Library of Medicine, containing citations to the last 40 years of medical literature [DS97]. One of the programs for searching Medline is PubMed [EPM], which is publicly available through the Internet. Next to citations PubMed provides a summary for most articles. Sometimes also free full-text articles are available in PubMed Central or elsewhere on the Internet (in which case a link is provided). Otherwise, one needs to go to a library or subscribe to the publisher to get the full-text article. Medline can be helpful for answering medical questions. However, finding specific answers to questions can be time-consuming and expensive because of the effort required to search through a sometimes large set of relevant publications [WM99].
Most of the printed sources for patient education are also provided electronically. For example, patient brochures and patient letters are published on the website of the Dutch College of General Practitioners [NHGp]. Besides, the Scientific Institute of Dutch Pharmacists electronically publishes information on the most frequently used medicines [APO].
Electronic sources of population statistics include electronically available published descriptions of disease prevalence in the medical journal literature, but also aggregate patient data from electronic medical records [GOR95]. Published statistics are not always applicable to a given
Question answering for general practitioners
local population because of differences in ethnic composition, local vectors of disease, or lifestyle differences, but the increasing use of electronic medical records improves capabilities for analyzing data of the local population. For example, the Dutch National Information Network for Primary Care (LINH) is a network of 93 automated general practices with over 360,000 patients [LINH]. These general practitioners continually collect data on diseases, consultations, drug prescriptions, and referrals. These data are used to generate representative, continuous, quantitative, and qualitative information on the care provided by Dutch general practitioners, but could also be used to generate population statistics.
Gorman et al. [GAW94] investigated to what extent the questions arising in general practice could be answered using only online medical journal literature. They randomly collected a set of 60 questions from American general practitioners not working in hospitals. Medical librarians tried to find answers to these questions in online resources. The general practitioners themselves evaluated the information found. In 56% of the cases physicians judged the information to be relevant for their question. In 46% of the cases the information provided a clear answer to their question. These 46% might include cases where no relevant information was found, because sometimes this is a clear answer to questions such as "Is there any information on new therapies for disease X?". In 40% of the cases physicians expected the information would have had an impact on their patient, and in 51% of the cases they expected the information would have had an impact on themselves or on their practice. These percentages highly exceed the current use of electronic resources by general practitioners [VNB99], suggesting that a QA system might be able to answer a lot of questions currently being answered with human or printed resources.
A question answering system could only make use of electronic sources. Luckily,printed sources and sources for patient education are increasingly available inelectronic form, making them also accessible by a question answering system. Human resources can not be consulted by a question answering system, but the informationthey present might also be available via printed and electronic resources.
2.3.2 Sources of evidence based medicine Evidence based medicine is an approach to clinical practice in which physicians base their decisions and actions on appropriate evidence from the patient's history, examination, laboratory data, and scientific medical knowledge [VER99]. Practicing evidence based medicine is guided by the following principles:
formulating the question; searching the literature for relevant information; selecting the articles; appraising the evidence for validity and usefulness; and applying the evidence in everyday practice.
Literature research must be guided by scientific strategies and should satisfy the same criteria as research in general. It should thus be valid, reproducible, and verifiable. Bias should be limited. For example, studies yielding statistically significant differences between groups are far more likely to be reported than those in which no differences were found. Therefore, to minimize publication bias, both published and unpublished studies need to be included and criteria for inclusion and exclusion must be accounted for. It is recognized however that critically evaluating all literature is unrealistic for busy physicians. Instead, general practitioners rely on evidence based resources, like guidelines, critically appraised articles, and systematic reviews.
Dutch guidelines for general practitioners are issued by the Dutch College of General Practitioners (NHG). These guidelines include the NHG-Standaarden [NHGs], which prescribe the actions a physician should undertake concerning diagnosis and treatment of certain
Mieke van Langen
diseases, and the NHG-formularium [NHGf], which gives pharmacotherapeutic advise. Both are evidence based, but they only provide guidelines. They don't give any information about the effectiveness of different therapeutical options. English guidelines are provided for example by the National Guideline Clearinghouse [NGC].
Critically appraised articles are published by several journals [VER99]. For example Evidence-based Medicine and ACP Journal Club together publish the cd-rom Best Evidence [BE96], and the Journal of Family Practice also publishes critically appraised articles.
Systematic reviews can be subdivided into qualitative systematic reviews and quantitative systematic reviews. For a qualitative systematic review the medical literature is searched for all relevant information on a specific disease, in order to formulate the best approach to diagnosis or treatment. For example, Clinical Evidence (published by the British Medical Journal) [BMJ] is a printed and electronic source providing information on the evidence of the effectiveness of different therapies. Each subject is started with relevant medical questions, and then the best available evidence is summarized to answer these questions. Besides, a list is provided of the interventions covered, categorized according to whether they have been found to be effective or not. Clinical Evidence doesn't make any recommendations. It is updated every six months in print and monthly online.
Quantitative systematic reviews (or meta-analyses) try to answer medical questions, using rigorous statistical analysis of pooled research studies. For example, the Cochrane Library (published by the Cochrane Collaboration) [CC] is an electronic journal that provides quantitative systematic reviews. Whereas qualitative reviews consider all reported treatments for a specific disease, quantitative reviews concentrate on the evidence for only one treatment and summarize the statistical data of all relevant studies to get more significant information about the effectiveness of this treatment.
Next to guidelines, critically appraised articles, and systematic reviews, there are also question answering services that provide physicians with evidence based answers (generated by for example a clinical librarian) on their medical questions. Usually these services require that general practitioners submit their questions in PICO (Patient or Problem, Intervention, Comparison intervention, Outcomes) format in order to direct the search to relevant and precise answers [CEBM]. In this format firstly the patient's problem and characteristics are described, then the intervention the physician is considering, then (if relevant) the alternative intervention to which the physician wants the intervention to be compared, and finally the outcomes the physician wants to reach with this intervention. For example, the NLH Question-Answering Service [NHS] is a pilot project that tries to answer medical questions. The answers provided by this service consist of the original question, the interpretation of this question, the text of the answer, and the references used. Verhoeven and Schuling [VS03] developed a question answering service for Dutch general practitioners to investigate whether general practitioners use this service and what the costs are. General practitioners used this service minimally, but it was found that they used the service more often when they personally knew the person who answered their question. It turned out to be possible to answer general practitioner's questions within the required timeframe, and the costs were on average 200 Euro per question. Coumou and Meijman [CM03] suggest that these costs should be covered by the patient's health insurer, because literature research is quicker, cheaper, and sometimes more useful than for example blood tests or MRI scans.
Because general practitioners seem very reluctant to use question answering services, and a single search executed by a medical librarian costs on average 200Euro, a question answering system is probably not suitable to answer the type ofquestions submitted to a question answering service. Besides, it is not realistic to expect a question answering system to critically appraise all information sources. Aquestion answering system should therefore only consult evidence based resources
Question answering for general practitioners
for answering medical questions. An electronic source like the World Wide Web or even Medline is thus not suitable as document collection for a medical questionanswering system when a physician wants to practice evidence based medicine(which he should). However, most evidence based information sources are in English and not freely available, except for the guidelines issued by the Dutch College ofGeneral Practitioners (NHG). When other resources are used, the information foundby the question answering system should be presented in such a way that the general practitioner is able to evaluate the answers himself.
2.3.3 Using information sources The information sources most frequently used by general practitioners are human based (colleagues), followed by private books (tertiary literature), journals (primary literature) [VBM95], and continuing medical education (such as classes and conferences) [WM99]. Libraries and printed or online bibliographies are much less used. In 1999 medical computer applications, telemedicine and the World Wide Web were the least used information sources [WM99]. In a survey among Dutch general practitioners conducted in 1996 [VNB99] only 3% of the general practitioners indicated they sometimes used the Internet for answering patient-specific questions.
Variances in the use of information sources exist among general practitioners. Factors influencing this behavior are presented in physical, functional and intellectual accessibility of the resource. Physical accessibility concerns how close the resource is to the general practitioner. Distances are diminishing however as information increasingly becomes available via general practitioners' desktop computers. Functional accessibility concerns the time and energy needed to search the information. As for electronic sources, a QA system could seriously decrease the time and energy needed to find relevant information. Intellectual accessibility concerns the understandability of the information. This depends on the intelligence of the practitioner, but also on the organization of the information.
Description
Physical, functional, and intellectual
Availability, searchability, and understandability
accessibility of the resource
Participation in research or teaching
Rural vs. urban physicians
Practice characteristics
Solo practice vs. health center
Stage of information gathering
Analysis, decision, etc.
Table 6 Factors influencing information seeking behavior
The second factor is the age of the general practitioner. Younger physicians tend to use libraries and printed sources more frequently than older physicians. Thirdly, physicians who engage in research or teaching use journals, conferences, libraries and online databases more often than others. The fourth factor is social context. Rural physicians tend to perform less online searches than urban physicians do. The fifth factor is practice characteristics. Physicians in solo practices use journals most, whereas physicians in health centers usually consult colleagues. Finally, the sixth factor is the stage of information gathering. In calling attention to
Mieke van Langen
new information printed material is mostly used, during analysis personal contact is most important, and in the decision stage refresher courses are the most important information source.
Human based resources are preferred for several reasons [GOR95]. Firstly, because many medical questions have a narrative character, they can be easier asked to a colleague than to printed or electronic sources. Secondly, information seeking behavior by general practitioners is not only determined by the need for medical knowledge, but also by the need for commiseration, affirmation of professional relationships, feedback about their own knowledge and practices relative to those of others, etc. These needs can't be easily fulfilled by printed or electronic sources. Thirdly, there may be a need for higher-order information than descriptive medical knowledge, such as confirmation, explanation, analysis, synthesis, and judgment that takes into account the complexity of the patient's case and combines it with an expert understanding of the issues involved. Finally, general practitioners need an answer to a patient care problem, not just information relevant to a query. With this respect, human sources understand best what the general practitioner needs.
Electronic information sources are not frequently used, because the variables of awareness of electronic sources are generally negative. In the survey conducted in 1996 [VNB99] Dutch general practitioners indicated that they wanted improvements in the World Wide Web on ease of searching, financial costs, time to search, chance of success, and usefulness. No relation was found between the use of electronic sources and the age or type of practice of the general practitioners in this survey. Ideally, electronic information retrieval systems should automatically display relevant summary information and provide links to supporting evidence and analysis. To realize this vision a combination of content, information science methods and technology is required [DS97]. Content already becomes increasingly available electronically. Information science methods are needed to structure this content to achieve optimal retrieval, select resources to best answer particular questions, integrate information from several sources into one consistent view, and provide search interfaces that help users select the appropriate search terms. Technology concerns high-speed data networks, standard protocols, open-systems architectures, and cross-platform applications. In this area the Internet already provides most functionality. Moreover, electronic content increasingly becomes available via the Internet and integration of lots of electronic sources is thus possible. It is therefore most important to concentrate on the information science methods.
A question answering system could implement most of the information sciencemethods mentioned above:
content can be structured by indexing the document collection, preferably content
based (as opposed to word based);
information from several sources is integrated into one consistent view when the
system composes a single answer of all retrieved information;
a question answering system naturally provides a search interface that helps
users select the appropriate search terms, because the user only has to enter aquestion in natural language, which the system will then transform into theappropriate query.
A challenging issue, however, is selecting the right resources. Physicians needassistance with this, because they are not aware of all the resources available toanswer a particular question, nor do they have time to assess which resource is best[DS97]. To automate the resource-selection process, the system must have knowledge of what questions each resource can answer. It is therefore needed toknow the scope, depth, intended audience, currency of information, and reputability ofeach source.
Question answering for general practitioners
As for sources of evidence based medicine, Barrie and Ward [BW97] state that general practitioners may be reluctant to change towards these sources, because they highly value the human judgment and accessibility of their current sources. If general practitioners would experience an excess of unanswered questions or if they were dissatisfied with their current sources, they might be motivated to change resources. However, they seem to find answers to most of their medical questions and are satisfied with the information sources they currently use.
Apart from the established preference of physicians for human information sources, the most important determinants of whether a knowledge resource will be used are its availability and clinical applicability [GAW94]. In a survey among Dutch general practitioners the most important characteristics for choosing an information source were found to be its reliability, usefulness, and ease of searching [VNB99]. Financial costs were considered the least important. The general practitioners indicated they think the following improvements in obtaining information are desirable: better-organized and more practical journal articles; more high quality, evidence based reviews; computer-accessible abstracts of most important articles; computer-accessible medical questions and answers; and more medical guidelines.
A medical question answering system could help in providing computer-accessible abstracts of relevant articles, medical questions and answers and medical guidelines.
However, in order to be used by general practitioners, the information presented by a question answering system should be reliable and easy to apply in clinical practice.
2.4 Computer
Computer applications developed to support clinical decision making for general practitioners can be divided in three categories [WM99], see consists of clinical information systems. These systems manage electronic medical records. They support general practitioners by reliably and efficiently storing and retrieving patient data. Therefore, structured ways of data registration have been developed [DHP98]. An approach for the registration of medical data is SOAP (Subjective findings, Objective findings, Assessment, and Plan). This is a problem-oriented way of recording. Firstly the symptoms stated by the patient (subjective findings) are recorded, then the signs found in medical examination (objective findings), thirdly the diagnosis concluded by the general practitioner (assessment), and finally the prescribed therapy (plan). Further, symptoms, diagnoses and therapies can be specified by specific codes. A coding instrument that is accepted by the Dutch College of General Practitioners (NHG) and the World Health Organization is the ICPC (International Classification of Primary Care). Criteria for the ICPC codes are defined in the ICHPPC-II-Defined (International Classification of Health Problems in Primary Care-II-Defined). There are also ICPC-like classifications for physical examination results and test outcomes (OCO), and for drugs.
Type of general practitioner
Description
application
Clinical information systems
Electronic medical record systems
Clinical decision support systems
Provide information about diagnosis, therapy, and prognosis, based on patient data
Bibliographic and full text information Search for information relevant for answering retrieval systems
medical questions
Table 7 Categories of applications developed for general practitioners
Mieke van Langen
Secondly, clinical decision support systems provide general practitioners with information regarding diagnosis, therapy, and prognosis, based on patient data. A Dutch clinical decision support system that has been introduced countrywide is the Electronic Prescription System (EPS) [LSS01]. This system suggests drug prescriptions and other therapies for a patient, based on patient data, the (ICPC coded) diagnosis, and prescription guidelines issued by the Dutch College of General Practitioners [NHGf, NHGs]. The system can also print a drug prescription and send it directly to the pharmacist. Besides, patient letters [NHGp] can be retrieved with the system. To be able to use this system, the general practitioner should also use a clinical information system, the SOAP approach, and ICPC coding.
Finally, bibliographic and full text information retrieval systems support general practitioners in finding information to answer their questions. An example of a bibliographic information retrieval system is PubMed [EPM] (see the part on electrfor more information on this system).
A question answering system would belong to the category of bibliographic and full text information retrieval systems. Theoretically, a question answering system couldalso be used as a decision support system, but a lot of reasoning is needed for that.
Therefore, the IMIX demonstrator used for this research excludes advisory questions. The question types dealt with by the IMIX demonstrator include:
questions about facts (e.g. What is RSI?); verification questions (e.g. Is RSI chronical?); multiple choice questions (e.g. Is exercise good or bad for RSI?); and quantity questions (e.g. How many people are suffering RSI?).
During the evaluation of the implementation of the Electronic Prescription System (EPS) different surveys were executed on the computer use by Dutch general practitioners. In 1999 95% of the general practitioners reported they owned a computer, and in 2001 this percentage was already 97% [WHB02]. In 2001 100% of the computer owning general practitioners also owned a clinical information system, 94% of them actually used the electronic medical record, 71% also owned the EPS, and 87% of them actually used it. The EPS was even found to be used more often than any printed resource. The electronic medical record was used during the consultation in 86% of the cases, and mostly also the SOAP approach was used. ICPC coding is less commonly used. Only 25% of the electronic medical record using general practitioners indicated they nearly always used ICPC coding, but this amount had already significantly increased since 1999.
The Internet use by Dutch general practitioners was investigated by order of the Dutch College of General Practitioners in 2000 [COX00]. In this research 76% of the general practitioners said they had access to the Internet, but only 37% could access the Internet from their general practice. Almost half the general practitioners without an Internet connection indicated they were considering implementing one before the end of the year. The general practitioners primarily used their Internet connection for e-mail, mostly with colleagues. However, 43% of the general practitioners also indicated they wanted to use websites with medical knowledge (those days the guidelines issued by the Dutch College of General Practitioners were not yet available via the Internet). When medical knowledge is available electronically, the general practitioners would rather access it through Internet than on cd-rom. 34% of the general practitioners indicated they would like to receive some education on searching medical knowledge on the World Wide Web.
Several general practitioner information systems and their implementations and evaluations have been described in literature. They range from simple administrative systems, to advanced decision support systems. None of them really uses QA technology, though some information
Question answering for general practitioners
retrieval systems get close. In the remainder of this section findings from literature are described that could be useful for the design of the information presentation module for a QA system.
During the evaluation of several versions of a Dutch general practitioner information system by Dupuits et al. [DHP98], general practitioners indicated they didn't appreciate the use of mouse input. They found the act of moving a mouse for data entry during a consultation too disturbing. Besides, the mouse with the underlay took too much space on their desks. Interfaces based on the use of windows and function keys were well appreciated. Speech input was also not feasible for practical reasons. Due to the diversity of voices, only the voices of a limited number of persons were recognized. Besides, disturbing sounds from the immediate surroundings occasionally caused the voice controller to randomly generate words.
Findings from a user centered development process of a general practice medical workstation [RHF92] included that there is no best way to perform the work process, because medical practice is extremely complicated and users are highly variable. General practitioner information systems must therefore be equally flexible. Further, it was found that simple clear presentations are often more effective than sophisticated attempts to provide intelligent summaries, because physicians are good at recognizing patterns if the information is clearly presented. This research also indicated that some physicians find the use of a mouse difficult. It is to be questioned, however, whether this still is the case, because over ten years have proceeded from then. Further, several overall constraints were abstracted from the results of this research. Firstly, a general practitioner information system must always be interruptible. If the general practitioner turns away from the system, it must be in precisely the same state when he returns to it. Secondly, all functional options must be immediately visible at the user interface. The use of multi-state buttons should thus be minimized. Thirdly, there should be no modes. All functions should be possible at every phase of the consultation, since the course of the consultation is highly variable and unpredictable.
Different evaluations of the EPS have been executed. The evaluation of the implementation of the system by the Dutch Institute for Research of Health Care (NIVEL) [WHB02] concentrated on the frequency of use and practical barriers for using the system (such as problems concerning the ICPC coding system, or lack of training). Lagendijk et al. [LSS01] primarily suggest different implementation methods to increase the general practitioners' motivation to work with the system. They state that the most important issues for general practitioners are better communication (with hospitals, pharmacies and colleagues) and reduction of time pressure. An application for general practitioners should therefore contribute to one or both of these issues to increase general practitioners' motivation. Boonstra [BOO03] investigated the subjective reasons for the limited acceptance of the EPS. He concluded that to improve general practitioners' acceptance of the system it should, among others, be designed to fit the consultation process (it should not disturb, but enhance the communication with the patient); it should suggest alternative therapies rather than only one therapy (this would recognize and strengthen the self esteem of general practitioners as medical professionals); it should be designed so that users could add new therapies or local agreements on therapies; and patients should be informed about the features and advantages of the system (to avoid patients having a lower esteem of physicians using the system).
For a question answering presentation module these findings implicate that themodule should be very flexible. It should preferably accommodate multiple inputmodes, but at least input should not be restricted to mouse or speech input only.
Further, the module should provide simple clear presentations as much as possible,and all functional options must be immediately visible at the user interface. Becausethe module must always be interruptible, it should not incorporate any timeouts.
Finally, to achieve motivation to work with the system it should at least save thegeneral practitioner time (for it cannot improve communication), and the systemshould not give the impression that the presented answer is the only best way,
Mieke van Langen
because general practitioners want to have the feeling that as medical professionalsthey are in control of the decision process, not the system.
2.5 Conclusions
Based on the literature reviewed in this chapter, the following conclusions can be drawn with
respect to the work roles, tasks, and information needs of Dutch general practitioners suitable to
be supported by a QA system, the variables related to the awareness of information that could
be improved by a QA system, the information sources that could be consulted by a QA system,
the way the system should search these sources for relevant information, and the user interface
of the system.
Work roles and tasks A QA system could support the general practitioner primarily in his work role of service provider, during the phase of searching databases, in order to help the general practitioner choosing a course of action and providing explanations to the patient. Especially when the system is available on a mobile device, it could really improve the physician's tasks associated with the role of service provider, namely patient care.
Information needs The types of information that can be provided by a QA system are medical knowledge and population statistics. General practitioners' questions regarding these information types generally deal with diagnosis, treatment, patient education, prescribing, and disease etiology. The most frequently needed medical knowledge categories are gastrointestinal, dermatological, musculoskeletal, and circulatory knowledge. The document collection of the system could therefore consist of scientific medical literature on these subjects, sources for patient education, and formal population statistics. Besides, a Dutch QA system might provide some information specific for the Dutch situation that is on the boundary of medical knowledge and logistic information, like which hospital performs which treatments.
It is expected that younger physicians and physicians working in larger practices are more likely to use a QA system, because they generally have more questions than their older colleagues and general practitioners working in solo or duo practices.
Awareness of information The aim of the system should be to transform most of the recognized information needs concerning the information types provided by the system into satisfied needs. A lot of recognized needs are never pursued, however. The system should be designed to encourage general practitioners to pursue their information needs. Therefore, physicians should be convinced that a definitive answer exists and that this can be found with the QA system. Besides, the system should save the general practitioner time.
Further, there are some variables related to the awareness of information that could be improved by a QA system as opposed to other information sources. A QA system could take away the uncertainty about the search strategy to be followed by the general practitioner, it could improve the functional accessibility of other electronic resources, and it could improve the intellectual accessibility of poorly organized electronic resources, when it adequately synthesizes information, possibly from different sources.
Sources of information General practitioners seem to use very little electronic information sources, while a QA system can only access electronic resources. This shouldn't be a problem, however, because research indicated that about half of the recognized information needs could be satisfactorily answered with online resources (also meaning that the other half of the information needs might not be met by a QA system). Only resources that are current, reliable, and suitable for direct clinical application should be used. Further, to allow for evidence based medicine, the system should
Question answering for general practitioners
primarily use evidence based sources such as those provided by the Dutch College of General Practitioners (NHG)
Information seeking The QA system should of course satisfy the pursued needs. Therefore, it is important that the system correctly interprets and modifies the question in order to find the right information and formulate a good answer, because general practitioners' questions generally are very complex. A dialogue might be needed to accomplish this.
To achieve optimal retrieval and select the right information sources to best answer particular questions, the system should index the available information (preferably using a content based indexing method). Further, the system needs to know the scope and depth of each source, it must incorporate bibliographic knowledge, and it should correctly interpret null search results.
Concerning the answer formulation, the system should not give the impression that the presented information is the single best way for clinical practice. To allow for evidence based medicine, the system should enable critical evaluation of the answers by providing links to supporting evidence and analysis.
User interface The information presentation module of a QA system should be very flexible. Input should not be restricted to mouse or speech input only, information should be presented as simply and clearly as possible, and all functional options must be immediately visible at the user interface. The module must not incorporate any timeouts.
Mieke van Langen
Question answering for general practitioners
3 Interviews with general practitioners
Most literature cited in the previous chapter is a few years old and based on non-Dutch
research. Coumou and Meijman [CM03] summarized the available literature published between
1992 and 2002 on the information seeking behavior of general practitioners. The only Dutch
research on the information needs and use of general practitioners was executed by Verhoeven
et al. [VBM95, VER99, VNB99]. Besides, the only articles dealing with answering general
practitioners' questions concerned bibliographic information or document retrieval systems, or
human question answering services. Computerized question answering (QA) systems have not
yet been used in the medical field. To get more in-depth information about Dutch general
practitioners and their expectations of a QA system, interviews were conducted with a few
Dutch general practitioners and Anita Verhoeven. he results of the interviews, and with respect to QA for general practitioners are discussed. The results of the interviews are not
always consistent with the findings from literature described in the previous chapter. In section
literature and the interviews are analyzed.
3.1 Method
Because great variations exist among general practitioners and their use of information
systems, this research does not aim to develop a system that would be equally appreciated by
all Dutch general practitioners. Requirements for the system will therefore not be based on
quantitative research but on qualitative interviews with Anita Verhoeven and only a few Dutch
general practitioners. Verhoeven also was a general practitioner, but because of a shortage of
available general practices, she turned to another job. Nowadays, she is medical librarian and
information specialist at the University of Groningen. In the 1990s she executed a PhD research
on the information needs of general practitioners [VER99] (which is frequently cited in the
previous chapter). The information she gave in the interview is partly used to complete the
literature survey in the previous chapter, and partly to comment on the results of the interviews
with general practitioners described in the next subsection.
For the interviews with general practitioners thirteen general practitioners, working in ten different general practices (mostly in Ede, the Netherlands), were sent an introductory letter. The next week they were asked by telephone if they were willing to contribute to this research. Five of them, each working in another general practice, agreed to participate, four men and one woman. Two of them worked in a primary health care center; the others worked in duo practices. The interviews were semi-structured and covered the topics of information needs, information sources and computer use. A general outline of the interview is shown in Appendix B. First of all the general practitioners were explained that this research is about the information needs of general practitioners and concentrates on questions about medical knowledge. Then they were asked how often they are confronted with such questions and whether and when they search for an answer to these questions. Consequently they were asked which resources they generally use to pursue their information needs and whether they think their possibilities for finding answers on their questions are sufficient. The third part of the interview dealt with computer use. Firstly the general practitioners were asked whether they owned a computer at work and which applications they use. Then they were shown a few paper examples of user interfaces for information retrieval systems and a QA system (see Appendix B) to give them an idea what these systems are about. They were asked whether they used any information retrieval systems for medical purposes and whether they think a QA system might be useful for their work. Finally, a few questions were asked about their preferences concerning user interfaces.
3.2 Results
In the following subsections the results of the interviews are discussed with respect to the
general practitioners' information needs, information sources, and computer use.
Mieke van Langen
3.2.1 Information
Most general practitioners indicated they rarely meet any medical questions for which they have to look up an answer. Especially the three older physicians said they have a good memory and large experience and only sometimes have to look up information on uncommon cases. Besides, the general practitioners are all supported by a general practitioner information system (Medicom) that incorporates an electronic prescription system, which frequently eliminates the need to look up any information on medication (though one of the general practitioners indicated he rarely agreed with the information provided by the electronic prescription system).
All general practitioners said they normally always pursue their information needs. If possible, they immediately search for an answer during the consultation. Otherwise, they do it afterwards or in the evening hours. During patient visits they rarely search for answers. Some general practitioners even avoid visiting patients as much as possible, because they lack access to most of their information sources when they're not in their consulting room. Besides, their computers are connected to those of the local pharmacy (which is a feature specific for the general practitioner information systems provided by Medicom). Therefore, prescriptions are only executed in the consulting room from where they are directly sent to the pharmacy. When general practitioners are confronted with a medical question during a patient visit that needs an immediate answer, they call a specialist. Less urgent questions are answered when the general practitioner is back in his consulting room. Most general practitioners indicated that the questions they are confronted with during patient visits don't differ from those met in the consulting room. Only one of them thought they did differ, because the patients she visits differ from those who come to her consulting room. The visited patients are generally elderly (having less questions than other patients) or terminal patients (eliciting questions about palliative care). She therefore always carries a manual on palliative care with her when she is visiting patients.
3.2.2 Information
Printed information sources are used by all general practitioners. The most frequently mentioned resources are the Pharmacotherapeutic Directory (Farmacotherapeutisch Kompas), the Diagnostic Directory (Diagnostisch Kompas), the guidelines issued by the Dutch College of General Practitioners (NHG-Standaarden), and books on internal medicine (Interne geneeskunde), dermatology (especially for the images), and microbiology. These books are all in Dutch and organized specifically for direct clinical application by general practitioners. The general practitioners indicated they rarely use books about the basics of medicine, like physiology, because they are not suited for direct clinical application. Dutch journals, like Nederlands Tijdschrift voor Geneeskunde (Dutch Medical Journal) and Huisarts en Wetenschap (General Practitioner and Science), were primarily used by the general practitioners in their roles of learner (for ongoing medical education), not for searching answers on medical questions in their roles of service provider. One of the general practitioners even indicated he thinks journal articles merely present opinions, rather than objective knowledge. He thinks even systematic reviews are essentially subjective, because scientific research is always executed with a specific goal in mind.
Human resources are also used by all general practitioners. They consult both their colleagues in the general practice and specialists. One of the general practitioners indicated he usually sends an e-mail to a specialist when he has any questions. The others rather consult specialists by phone. However, one of them told that when he has diagnostic problems concerning a dermatological disorder, he takes a picture of it with his digital camera and sends the image to a dermatologist by e-mail to ask for advise, which he called ‘tele-dermatology'.
Most general practitioners don't use much electronic resources. Verhoeven indicated in the interview that the primary reason for this is that general practitioners lack time and capabilities to search for information in electronic resources, and also lack time to improve their searching capabilities. The electronic prescription system is the only electronic resource used by all general practitioners. The system was used for retrieving prescription information and patient
Question answering for general practitioners
letters. Some general practitioners also electronically accessed the guidelines issued by the Dutch College of General Practitioners (NHG). The Internet was used to search for answers on medical questions by only two of the general practitioners. One of them primarily used PubMed, the other Artsennet. PubMed provides access to a bibliographic database (Medline). This general practitioner thus frequently ends up with only a summary of a relevant article. When he wants to have the full text, he needs to go to the library, but he hasn't got time for that. Artsennet is a website published by the Royal Dutch Medical Association (KNMG) presenting Dutch medical information for and written by Dutch physicians [KNMG]. Another general practitioner indicated he did sometimes search the Internet for medical information, but primarily in his role of learner, not as service provider, because he thought the information found on the Internet is very general and not suitable for direct clinical application. When answers to medical questions are searched on the Internet, this is usually done after the consultation, because it takes too much time to do this during the consultation.
Verhoeven indicated in the interview that Dutch general practitioners rarely use English information sources. Their primary evidence based resource is the guidelines issued by the Dutch College of General Practitioners (NHG-Standaarden). These are publicly available to all Dutch general practitioners, whereas English evidence based resources generally are not. When general practitioners are searching for medical knowledge that is not available in the guidelines, Verhoeven would recommend firstly consulting Clinical Evidence [BMJ], then the Cochrane Library [CC], and finally scientific literature via PubMed [EPM]. However, none of the interviewed general practitioners used Clinical Evidence or the Cochrane Library. The only English resources used are journal articles.
Most of the general practitioners indicated they think their possibilities of finding answers on their medical questions are sufficient. Especially the older general practitioners generally have few questions. However, one of the younger physicians indicated she sometimes finds it difficult to decide where to search, but when she doesn't know where to find the information, she asks one of her colleagues.
3.2.3 Computer
All general practitioners participating in this research had a computer with cd-rom player in their consulting room. They used a general practitioner information system that incorporated an electronic prescription system. Most of the general practitioners owned some disk with medical knowledge, but they haven't got time to use it or to learn working with it. Especially the older physicians indicated they are used to their working procedures without electronic resources. They see that their younger colleagues or the general practice trainees they teach in their roles of educator more often consult electronic resources, and they think it could be useful, but they don't think they really miss anything in their possibilities of finding answers to their medical questions. Other applications used by the general practitioners are Microsoft Word (mostly for writing letters) and an e-mail program.
The general practitioners all had access to the Internet, four of them also from their consulting rooms, and the other one was planning it, but they vary greatly in the ways they use it. One of the general practitioners indicated that "whenever he needs it, it is down", another primarily used it as a communication channel (for e-mailing medical specialists) instead of an information resource, and the others really used it to look up medical knowledge. Apart from the guidelines issued by the Dutch College of General Practitioners, they used Google, PubMed and Artsennet. Google was thought to be easy for searching, because with all possible keywords lots of results are returned, but difficulties were experienced judging the reliability and applicability of these results. With PubMed and Artsennet it is harder to select the right keywords, but the results are always scientific and aimed at medical professionals. However, most articles cited by PubMed are not freely available.
Most general practitioners found it hard to imagine themselves using a QA system. They think the printed sources they use now are already easily accessible and suitable for direct clinical
Mieke van Langen
application. Besides, they prefer to look up information in these books, because of the context they provide. The guidelines issued by the Dutch College of General Practitioners can already be easily found electronically, eliminating the need for a QA system. Two of the general practitioners indicated they would rather use a QA system for patient education, because they have their own ways for searching the information they need for themselves, while they think the patient letters provided by the electronic prescription system are not always sufficient. One of them suggested he also might want to use a QA system for retrieving dermatological images. He had a book for this, but he thought a QA system might provide access to more images. Another general practitioner would like to use a QA system to retrieve summaries of Dutch recent scientific publications from for example Huisarts en Wetenschap (General Practitioner & Science). And yet another general practitioner would like to use it for retrieving ‘data' like e-mail addresses, telephone numbers, patient organizations, waiting lists, and logistic information. Verhoeven thought a QA system could use the database of an evidence based question answering service as a resource. However, this would restrict the system to answering questions that already have been answered by humans. All general practitioners would prefer concise answers or relevant paragraphs to complete documents. However, they all want some link to the original document, to enable them to read more about the subject if they want to. Besides, they all emphasized the information sources used by a QA system should be up-to-date and reliable. They could thus best be retrieved from the Internet, but only from some selected sites. Finally, one of the general practitioners indicated he would like to use medical language, or even ICPC-codes, in the questions he would submit to a QA system.
The general practitioners participating in the interviews tend to prefer mouse input above using function keys, because function keys are harder to learn, and mouse input is quicker than switching between different menus with function keys. However, the general practitioner information system they use now is still not working with mouse input, but the next version will. Only one of the general practitioners indicated she would rather use keyboard input to avoid RSI problems. The general practitioners reacted rather differing on the suggestion of speech input. Some thought it would be excellent, because it would be much quicker than typing with two fingers, others thought it would only be useful for dictating letters, and one of them indicated using speech input during consultations would be rather disturbing in the contact with the patient. In any case they only want to use speech input when it is working perfectly.
3.3 Conclusions
Based on the interviews, the following conclusions can be drawn with respect to the work roles,
tasks, information needs, awareness of information, and sources of information of Dutch
general practitioners relevant for QA, and the user interface of a QA system.
Work roles and tasks The general practitioners participating in the interviews indicated they lacked their information sources during patient visits. A QA system that is accessible during patient visits could thus be very useful. However, the general practitioners haven't got a personal digital assistant (PDA) or laptop computer yet.
Information needs Because general practitioners vary greatly in their information needs and use, and in their use of computers they can't be expected to use a QA system equally often. Younger physicians seem to have more questions and more often use Internet to search for an answer. They are therefore expected to appreciate a QA system more than their older colleagues.
Awareness of information Most general practitioners preferably search for an answer on their medical questions during the consultation. However, they indicated that searching for an answer on the Internet takes too much time to do this. Therefore, a QA system should reduce the time needed to search an answer on the Internet sufficiently to make possible searching for answers during the consultation.
Question answering for general practitioners
Sources of information The sources general practitioners currently use in printed form (textbooks) are already well organized and easily accessible. There is thus no need to make them accessible via a QA system. Only one of the general practitioners indicated he might want to search for dermatological images (also found in his manual) with a QA system, but other techniques like image retrieval would be more suitable for this. Human sources can't be made accessible via a QA system, but their telephone numbers and e-mail addresses can. One of the general practitioners indicated he would very much like a system that supplied him with this type of information. However, information extraction techniques might be more suitable for retrieving contact information. Finally, electronic resources are especially suitable for use by a QA system, because the system can improve their functional accessibility. Information retrieval systems currently used by general practitioners either return too much irrelevant information or require very specific keywords. A QA system should act as an intermediary selecting the right sources and keywords for the general practitioner.
Not all electronic sources are suitable for a QA system however. The guidelines issued by the Dutch College of General Practitioners (NHG) are already easily accessible via an index. Electronically available Dutch journal articles could be used, but only if they are suitable for direct clinical application, otherwise they could better be accessed via an information retrieval system to allow the general practitioner to read the context of the information. The general practitioners participating in the interviews disagreed on the clinical applicability of journal articles. It is therefore not clear whether they should be included or not. Patient education is freely available on the Internet and widely dispersed. Electronic sources for patient education are therefore especially suitable for a QA system. Moreover, some of the general practitioners indicated they would primarily use a QA system for patient education. Thus, for the present, only information for patient education should be included in a QA system for general practitioners.
Most general practitioners participating in the interviews have access to the Internet from their consulting rooms. Besides, they all indicated they would like a QA system to supply them with the most recent information. Information sources used by a QA system can therefore best be retrieved from the Internet. However, to avoid irrelevant or unreliable information, the QA system should only consult some selected sites.
Information seeking A QA system for general practitioners should be able to handle medical language and preferably also ICPC-coding in the input question. The answer should be presented as a concise answer, with direct links to the sites the information was retrieved from, so the general practitioners can easily retrieve the complete documents if they want to.
User interface Most general practitioners preferably use mouse input, but keyboard input should be accommodated as well, because of RSI prevention. Speech input should only be used when it is working perfectly.
3.4 Discussion
Some of the conclusions drawn from the literature (described in results of the interviews described in this chapter. However, there also are some differences
between the conclusions of both chapters. In this section the results of the literature study are
related to those of the interviews.
Work roles and tasks The literature study was restricted to the information needs of the general practitioner in his role of service provider, because this is the role in which there is a need for concise answers. Some of the information sources mentioned in literature seem to serve the general practitioner primarily in his role of learner however. For example, journal articles, whether in printed or electronic form, are consulted to answer medical questions by only some of the interviewed
Mieke van Langen
general practitioners. The others primarily use them for ongoing medical education, because they think the information presented in journal articles is not suitable for direct clinical application. It is therefore not clear whether journal articles should be consulted by a QA system or not.
hat a QA system could support the general practitioner primarily during the medical consultation phase of searching databases. It is therefore most convenient to use the system during this phase of the consultation, instead of after the consultation. This conclusion is supported by the interviews, in which the general practitioners indicated they preferably search for an answer on their medical questions during the consultation. Therefore, a QA system should reduce the time needed to search for an answer on the Internet sufficiently to make possible searching for answers during the consultation.
The assumption, based on literature, that the medical consultation phase of searching databases might be harder during patient visits than in the consulting room is also supported by the results of the interviews. There are even general practitioners that avoid visiting patients as much as possible for this reason. It is therefore expected that in the end general practitioners will turn to mobile computers, because they surely need mobile access to their information sources. With a laptop, the general practitioner generally has the same possibilities as he has with his desktop computer, thus a QA system wouldn't need to be adjusted for use on a laptop. However, when a general practitioner would like to address the QA system via a PDA, a different user interface would be needed. Perhaps the presentation of the information would also have to be different, because less space is available for information presentation on a PDA.
Information needs Both in literature and in the interviews general practitioners varied greatly in the number of questions they are confronted with during patient care. In a questionnaire executed in 1996 this number ranged from 0.04 to 50 times a week [VNB99]. In this research no relations were investigated between the number of questions and the age of the general practitioner. However, Ely et al. {EOE99] found that younger physicians generally have more questions than their older colleagues. This finding seems consistent with the results of the interviews.
Based on the literature it was expected that general practitioners' questions generally deal with diagnosis, treatment, patient education, prescribing, and disease etiology, and that the most frequently needed medical knowledge categories are gastrointestinal, dermatological, musculoskeletal, and circulatory knowledge [VNB99, MCW05]. These findings seem consistent with the information sources consulted by the general practitioners participating in the interviews.
In the literature it was stated that a lot of recognized information needs are never pursued [GH95]. However, the general practitioners participating in the interviews said they normally pursue all their information needs. This great difference might be due to a difference in research method. In the interviews the general practitioners were asked directly how often they pursued their information needs, whereas in other studies general practitioners were observed [GH95] or they were sent a questionnaire [VNB99]. Questions may be forgotten shortly after they arose. In the case of an observation, such questions are recorded as recognized but not pursued. However, when physicians are directly asked for the percentage of recognized questions they pursue, they don't remember their forgotten questions (which are then taken for unrecognized questions), making the percentage of pursued questions much higher. Besides, the general practitioners participating in the interviews all used an electronic prescription system, making it much easier to answer questions concerning prescription. On the other hand, these questions were already routinely pursued without such a system [EOE99]. It is therefore expected that the actual fraction of pursued questions is somewhat lower than the interviewed general practitioners suggest.
Question answering for general practitioners
The assumption that questions may be easily forgotten is another reason why a QA system should reduce the time needed to search for an answer on the Internet sufficiently to make possible searching for answers during the consultation. To prevent general practitioners from not pursuing their information needs, they must be able to search for an answer to their questions immediately.
Awareness of information Concerning electronic information sources, the literature finding that general practitioners have difficulties in selecting the right sources and keywords [WM99] is confirmed by the interviews. The general practitioners indicated that the information retrieval systems they used either return too much irrelevant information or require very specific keywords.
Sources of information Whereas in literature no relation was found between the use of electronic sources and the age of the general practitioners [VBM95], the interviews suggest that younger physicians tend to use the Internet for searching information more often than their older colleagues. This might be due to the fact that they generally have more questions. Besides, younger physicians probably have learned using the Internet during their medical education, whereas older physicians mostly have had to learn using the Internet themselves (for which they generally lack time), because it is only for the last ten years that the Internet has come into use among the general public. Moreover, older physicians already have their ways of finding answers to their medical questions and they are generally satisfied with these ways. They are thus not really motivated to learn working with new information sources. It is expected that the younger general practitioners who have more questions and are already accustomed to using the Internet, will be most motivated to use a QA system, because for them such a system could really save time when it makes finding the relevant information on the Internet faster and easier. Besides, the time needed to learn working with the system will probably be the lowest for this group of general practitioners.
A QA system is not suitable for finding all relevant information on the Internet, however. Some types of information could better be found with other searching techniques, such as indexes, information retrieval, image retrieval, or information extraction. To help general practitioners choosing the application most suitable for retrieving each information type, some sort of web portal could be constructed that provides access to all different applications and thus to all different information types.
In the previous chapter it was stated that to allow for evidence based medicine, a QA system should primarily use evidence based resources such as those provided by the Dutch College of General Practitioners (NHG) and enable critical evaluation of the answers by providing links to supporting evidence and analysis. However, the general practitioners participating in the interviews (especially the older three) seemed to rely more on ‘experience based medicine' than on evidence based medicine. Besides, the electronic sources provided by the Dutch College of General Practitioners are thought to be already easily searchable and general practitioners indicated they appreciate the context provided by these sources. Only the suggestion that links should be provided to supporting evidence is confirmed by the interviews.
User interface While in previous research general practitioners seemed to prefer keyboard input to mouse input [DHP98, RHF92], the general practitioners participating in the interviews don't mind using a mouse. They even tend to prefer mouse input above using function keys. Probably this is because nowadays general practitioners have become accustomed to using a mouse for their non-medical applications like Microsoft Word and Internet Explorer. They are all looking forward to the next version of their current general practitioner information system that will be accommodated for mouse input. Still, because of concerns of RSI prevention, applications for general practitioners should not be restricted to a single input mode.
Mieke van Langen
Summarizing from both chapters, the following conclusions can be drawn with respect to the requirements and scope of a question answering system for general practitioners:
Information needs
for the present, only questions for patient education will be answered by the
question answering system;
the system should be aimed at physicians who are already accustomed to using
Awareness of information
physicians should be convinced that answers exist on the questions they have
concerning patient education, and that they can be found with the question answering system;
the system should reduce the time needed to search for an answer on the Internet
sufficiently to make possible searching for answers during the consultation;
Sources of information
only information sources that are current, reliable, and suitable for patient
education should be used;
information sources can best be retrieved from the Internet, but only from some
Information seeking
the system should be able to handle medical language and preferably also ICPC-
coding in the input question;
a dialogue might be needed to enable the system to correctly interpret and modify
the system needs to know the scope and depth of each information source; bibliographic knowledge should be incorporated in the system; null search results should be correctly interpreted; a different information presentation module must be designed for use via a PDA; the retrieved information should be presented as a concise answer, with direct
links to the sites the information was retrieved from;
information from several sources should be integrated into one consistent view; the system should not give the impression that the presented information is the
single best way for clinical practice;
User interface
a web portal could be constructed that provides access to all different types of
information for general practitioners (including the question answering system);
the user interface of the question answering system should be very flexible; input should not be restricted to a single input mode; speech input should only be used when it is working perfectly; a different user interface must be designed for use via a PDA; information should be presented as simply and clearly as possible; all functional options must be immediately visible at the user interface; there shouldn't be any timeouts.
Question answering for general practitioners
4 The information presentation module (GIPS)
Based on literature research and interviews with Dutch general practitioners (described in
drawn with respect to the requirements and
scope of a question answering (QA) system for general practitioners. Based on these
conclusions an information presentation module (GIPS: General practitioner Information
Presentation Submodule) has been designed for the IMIX demonstrator. Besides, requirements
have been specified for a web portal that provides access to all different information retrieval
systems relevant for general practitioners (including GIPS).
demonstrator and its components are described. The conclusions presented in the previous chapter are related to the components of the IMIX is prototype of an information portal for general practitioners is described.
The IMIX demonstrator
The IMIX demonstrator is an interactive multimodal QA system. The architecture of the first version of the demonstrator is presented in It comprises six components (shown as rectangles in the figure): a speech recognizer (norisc.asr), two question answering modules (rolaquad.qa and qadr.qa), an output generator (imogen.gen), a text to speech module (imogen.tts), and a graphical user interface (imix.gui).
The components interact with each other by reading and writing structured (XML-based) data on global data stores (pools, shown as ellipses component subscribes to a data pool, it automatically receives messages published to that pool. An advantage of using pools is that the framework is very flexible [HER03]. Because the data producer and data consumer do not have to know each other, components can be added or replaced without causing changes to other components.
The first component, the speech recognizer, is responsible for audio recording. It receives a signal from the pool "control.asr.start" when it can start speech recording, The recognizer may take some time to start up, and as soon as it is ready to receive speech input, it sends out a signal to the pool "control.asr.ready". The recognizer decides by itself when the speech recording ends. When recording is finished and the analysis is complete, a message containing the word graph is sent to the pool "questions".
The two question answering modules are triggered by a message arriving in "questions". This message originates either from the speech recognizer (in the case of speech input) or from the graphical user interface (text input). Each of the question answering modules processes the question and sends a message containing its answer(s) to the pool "answers".
When two answer messages have arrived in "answers", the output generation module transforms them into visual output (containing text and/or pictures), and speech output. Both outputs are simultaneously sent to the "presentation" pool.
The text to speech module starts speaking the text when it arrives in the "presentation" pool. It stops speaking as soon as a signal is received from the pool "control.tts.stop".
The graphical user interface controls the dialogue with the user. After a start screen, the user chooses between text and speech using a button. In case of text, the user types text in a text field, which is sent to the pool "questions". In case of speech, the speech recognizer is started by sending a signal to "control.asr.start", while a "please wait" prompt is presented to the user. When a signal is received in "control.asr.ready", a microphone symbol is displayed. Then, the graphical user interface waits for information to arrive in the "presentation" pool, which it displays. A "continue" button then takes the system back to the beginning, at which a signal is sent to "control.tts.stop" to stop any speech output still going on.
Mieke van Langen
Thus, the information presentation (collected in the pool "presentation") is generated by the output generation module and displayed by the graphical user interface. These are the components that will constitute GIPS. The speech recognizer and text to speech module will be ignored in this study, because they are not yet working perfectly, which is required for use by general practitioners. The two question answering modules will be taken for granted, because they are not part of the IMOGEN project (of which this study is a part). Thus, requirements will be drafted for these modules, but they will not be implemented.
Figure 4 Architecture for the first IMIX demonstrator
In the next subsections the current specifications of the output generation module and the graphical user interface are presented.
4.1.1 Output generation module The output generator subscribes to the pool ". The messages in this pool are XML-based QA documents [THE05]. An example of
The root element of a QA document is "qa". The attribute "engine" of this element specifies which of the two question answering modules produced the message ("rolaquad" or "qadr"). A QA document further contains a question element and a list of answers.
The question element contains the question. Its attribute "mode" specifies the input mode. Its value could be "typed" or "spoken". In GIPS, only typed questions will occur. The question element in turn contains two elements: the question string and the question analysis. The
Question answering for general practitioners
question string is the original string of the question as typed in by the user. The question analysis contains the annotation of the question. The tags used for the annotation differ per question answering module. The attribute "type" of the question analysis element indicates the type of the question. The classifications of question types are also different for each of the two question answering modules.
<?xml version="1.0" encoding="iso-8859-1"?>
<qa engine="qadr">
<question mode=typed>
<string> Wat is RSI ? </string>
<question-analysis type="definition(rsi)">
<node rel="top" cat="whq" begin="0" end="3">
</question-analysis>
</question>
<answerlist nr_answers="3">
<answer rank="1" conf="6.58">
<id source=www.rsi-vereniging.nl#overrsi#misverstanden
<context>
RSI is hetzelfde als een muisarm
</context>
</answer>
<answer rank="2" conf="6.57">
<id source=www.rsi-vereniging.nl#overrsi#misverstanden
<context>
Tegenwoordig is RSI een verzamelnaam ( ' paraplubegrip ' )
voor alle klachten aan armen , nek en schouders .
</context>
</answer>
<answer rank="3" conf="6.57">
<id source="www.arbeid.tno.nl#kennisgebieden#rsi#index"
<context>
Ook RSI is eigenlijk een verkeerde term .
</context>
</answer>
</answerlist>
Figure 5 Example QA document
The list of answers contains the answers produced by the question answering module. The number of answers is specified by the attribute "nr_answers". Each answer element contained
Mieke van Langen
in the list has two attributes: "rank", and "conf". The "rank" indicates the answer's rank within the answer list. Higher ranked answers are more likely to be a good answer to the user's question than lower ranked ones. The "conf" attribute indicates how confident the question answering module is that this answer is a good answer to the user's question. Confidence scores issued by "rolaquad.qa" range between 0.00 and 1.00, and those issued by "qadr.qa" range between 1.00 and 8.00. Next to these attributes, each answer in the list contains two elements: an id and the annotated text of the answer.
The id indicates the location of the answer within the document collection. For web documents "qadr.qa" uses the form www.arbeid.tno.nl#kennisgebieden#rsi#index [BOU04]. Compared to the original URL, slashes are replaced by hashes to be able to treat the ids as filenames. The other question answering module, "rolaquad.qa", indicates the answer document with an integer index key assigned to the document in the private Rolaquad document database [CBD05].
The text of the answer is contained in a core element, which is in turn contained by a context element that possibly gives the surrounding context of the answer. In principal, the core comprises one sentence.
<?xml version="1.0" encoding="iso-8859-15"?>
<!-- the default namespace is defined in pml.xsd -->
<presentation xmlns="pml.xsd">
<!-- presentation content is contained in content elements -->
<!-- this element is supposed to be realized as "speech" -->
<content realization="speech">
This is a P-ml document with a picture.
</content>
<!-- this is a named content element, which can be referred to -->
<!-- from inside other content elements -->
<content name="picture.png" encoding="base64"
content-type="image/png">
<!-- the actual picture is a base64 encoded PNG file -->
</content>
<!-- the visual part of the presentation is actually defined -->
<!-- in HTML -->
<content realization="visual">
<html xmlns="http://www.w3.org/1999/xhtml">
<h1>P-ml document<h1>
This is a P-ml document with a
<!-- "picture.png" refers to the named content element -->
<img src="picture.png" alt="picture" />
</content>
</presentation>
Figure 6 Example of a P-ml document
Question answering for general practitioners
The output generation module transforms the information received in the "answers" pool into an appropriate presentation of the information, which is then published to the "presentation" pool. Presentations are exchanged using P-ml (presentation mark-up language) [IMIXp]. This is a mark-up language that was designed specifically for the IMIX project. An example of a P-ml
The root element of P-ml is "presentation". Within the presentation element several content elements are specified that contain a part of the presentation. A content element has four optional attributes: "realization", "name", "encoding", and "content-type".
The value of the realization attribute specifies the method in which the content element is to be realized. For instance, "speech" means that the content is to be realized by a speech synthesizer, and "visual" means that the content is to be visualized. For GIPS only "visual" content elements will be used.
A content element that has a name attribute can be referred to from elsewhere in the P-ml document. Content elements without a "name" attribute can only affect the presentation if a realization attribute is specified.
The encoding attribute specifies the encoding of non-XML compatible file formats. For instance, PNG images can be base64 encoded.
Finally, the content-type attribute can be used as a hint to interpret the contents of a content element. This is especially useful in combination with the "encoding" attribute.
P-ml does not support content representation, but a content element may contain any other XML element designed for content representation, like an XHTML, SVG or SSML document. However, a content element may also contain an encoded binary format or just plain text.
4.1.2 Graphical user interface The graphical user interface receives input from the user, which it publishes to the pool "questions", and it displays the presentations generated by the output generation module in return.
The graphical user interface has eight different the state transition diagram for the user interface is presented. In the first state, "welcome", a welcome screen is displayed with information about the IMIX demonstrator and a start button. Then the system goes to the second state, "modality", in which the user is asked to choose between text input and speech input. The third to fifth states ("wake_up", "speech_input", and "text_input") deal with the different input modes. After a question has been entered, the system goes to the state "waiting_for_answer". When an answer arrives, the system goes to the state "answer" and the answer screen is displayed. This screen has two buttons: one to start a new question and one to close the graphical user interface. In case a user chooses to start a new question, the system returns to the "modality" state. When the graphical user interface is closed, which can be done in any of the states, the system gets to the final state "end".
4.2 Requirements
In the previous chapter a list of conclusions with respect to the requirements and scope of a QA
system for general practitioners was presented. This list was organized according to a subset of
the elements of the model of information seeking by professionals [LPS96] (see also section
information needs; awareness of information; sources of information; and information seeking.
Mieke van Langen
Besides, a separate category was added for requirements concerning the user interface. Based on all these conclusions, some user requirements can be specified for the IMIX demonstrator.
enter [empty_textbox]
Figure 7 State transition diagram for the first IMIX demonstrator
The conclusions concerning the information needs specify the type of questions that will be dealt with and the user group that will be targeted. These conclusions don't concern a single component of the system, but the problem and scope of the entire system. The requirements related to the awareness of information concern user education, and quality attributes like response time, reliability, and availability. These conclusions thus also concern the entire system. The conclusions with respect to the sources of information implicate requirements on the document collection consulted by the question answering modules.
Nine conclusions were drawn with respect to the information seeking of the system:
1. the system should be able to handle medical language and preferably also ICPC-coding in
the input question;
2. a dialogue might be needed to enable the system to correctly interpret and modify the
3. the system needs to know the scope and depth of each information source;
Question answering for general practitioners
4. bibliographic knowledge should be incorporated in the system; 5. null search results should be correctly interpreted; 6. a different information presentation module must be designed for use via a PDA; 7. the retrieved information should be presented as a concise answer, with direct links to the
sites the information was retrieved from;
8. information from several sources should be integrated into one consistent view; and 9. the system should not give the impression that the presented information is the single best
way for clinical practice.
These conclusions could be categorized according to the steps of asking and answering questions identified by Ely et al. [E
recognizing an information gap; question formulation; searching for relevant information; answer formulation; and using the answer to direct patient care.
The first two conclusions deal with the step of question formulation. This step is now dealt with by the two question answering modules of the IMIX demonstrator, but when a dialogue would be used to correctly interpret and modify the question (as is suggested by the second conclusion), it would be better to include a separate component to manage the dialogue. Actually, at this moment a second version of the IMIX demonstrator is developed that supports dialogue [VP05]. For this purpose a dialogue action manager is included in the architecture. This component continually decides which action to take next, depending on the status and the state of the dialogue with the user [OS04]. In most states, the next action is computed on the basis of the history of the dialogue, the semantic or pragmatic interpretation of the most recent user input, and the information obtained from the databases the dialogue action manager can access.
The third to fifth conclusions related to information seeking concern the step of searching for relevant information. This step is dealt with by the two question answering modules of the IMIX demonstrator. These conclusions therefore implicate requirements for the question answering modules.
Finally, the sixth to ninth conclusions related to information seeking concern the step of answer formulation. This step is dealt with by the output generation module. Requirements for the output generation module could therefore be derived from these conclusions.
The conclusions concerning the user interface implicate requirements for the graphical user interface. Besides, based on these conclusions the decision was taken to leave the speech recognizer and the text to speech module out of the design for the QA system for Dutch general practitioners. Further, one of the conclusions recommended developing a web portal that provides access to all different types of information for general practitioners, including GIPS.
Summarizing, these are the problem and scope of the entire system, and thefunctional requirements for each of the different components:
Problem and scope General practitioners are confronted with questions about patient care during medical consultations. Answers on these questions are needed during the medicalconsultation, but searching for answers on the Internet takes too much time.
Therefore, a question answering system should reduce the time needed to search foran answer on the Internet sufficiently to make possible searching for answers duringthe consultation. For the present, only questions concerning patient education will be
Mieke van Langen
answered by the question answering system, and the system will be aimed atphysicians who are already accustomed to using the Internet. For the system to besuccessful, general practitioners will have to be convinced that answers exist on thequestions they have concerning patient education, and that they can be found with thequestion answering system.
Functional requirements Dialogue action manager
medical language and ICPC-coding in the question must be understood; questions must be correctly interpreted and modified;
Question answering modules
the scope and depth of each information source must be known; bibliographic knowledge should be incorporated in the system; null search results should be correctly interpreted;
Document collection
only information sources that are current, reliable, and suitable for patient
education should be used;
information sources can best be retrieved from the Internet, but only from some
Output generation module
a different output generation module must be designed for information
presentation on a PDA;
the output should be presented as a concise answer, with direct links to the
documents the information was retrieved from;
information from several sources should be integrated into one consistent view; it must be made clear that the output is not the single best way for clinical
Graphical user interface
the user interface should be very flexible; both keyboard and mouse input must be accommodated; a different graphical user interface must be designed for use via a PDA;
4.3 Design
Based on the requirements discussed above, the architecture of the IMIX demonstrator has
been adjusted for use by Dutch general practitioners. The revised architecture is presented in
modules, the output generation module, and the
graphical user interface, an extra module is incorporated in the design: the dialogue action
manager (dam.control). This component is integrated into the architecture according to the
specification of the second version of the IMIX demonstrator [VP05]. However, because the
second version of the demonstrator is not yet implemented, the components that will be
developed for this research probably cannot be integrated with the other components during this
research.
module and the graphical user interface now communicate with the question answering modules through the dialogue action manager. The user input to the dialogue action manager consists of two pools: "user.gui" (pressed buttons), and "user.language.raw" (text). As soon as the definitive question is known, the dialogue action manager may write it to the pool "user.language.analysed". The dialogue action manager communicates with the question answering modules through the pools "questions" and "answers". The output from the dialogue manager is sent to the pool "dam.out". This may be an answer from the question answering modules, or another type of dialogue act, like a question or
Question answering for general practitioners
an informing message. The output generator transforms the signal received in "dam.out" into visual output and sends it to the "presentation" pool. Then the output is displayed by the graphical user interface.
user.language.raw
Figure 8 Architecture for the IMIX demonstrator targeted at general practitioners
The dialogue action manager, like the two question answering modules, will be taken for granted, because it is not part of the IMOGEN project (of which this study is a part). In the previous section two functional requirements were specified for this component: medical language and ICPC-coding in the question must be understood; and questions must be correctly interpreted and modified. The second requirement actually is the goal of the dialogue action manager. However, the dialogue action manager will not deal with the first requirement, because the IMIX demonstrator is aimed at naïve users [VP05]. Therefore, to implement this requirement, without changing the design of the dialogue action manager, the input to this component should not contain any medical language or ICPC-coding. The graphical user interface should therefore ‘translate' these terms before sending them to the "user.language.raw" pool.
In the next subsections the designs for the two components of GIPS (the output generation module and the graphical user interface) are specified.
4.3.1 Output generation module In the first version of the IMIX demonstrator, the output generator subscribes to the pool "answers" [IMIXa]. However, because the dialogue action manager is added to the design of the QA system for general practitioners, the output generator now subscribes to the pool "dam.out" [VP05]. In this pool two types of information occur: answers and other types of dialogue acts. The messages in this pool contain three different elements: dialogue act elements that give dialogue act information; an output element that contains the data to be used by the output generation module; and a context element that gives dialogue context information. The output element contains a text element (consisting of a text string) and/or qa elements that are exact
Mieke van Langen
copies of the messages in the pool "answers" that already exists in the first version of the IMIX demonstrator. Text elements represent dialogue acts other than answers.
When a signal is received in the "dam.out" pool, the output generation module transforms the information into an appropriate presentation, which is then published to the "presentation" pool. The qa elements generated by the question answering modules have to be modified to provide the general practitioner with a comprehensible presentation of the answer to his question. In the requirements it was stated that the answer should be presented as a concise response, with direct links to the documents the information was retrieved from (which should be very reliable), and that information from several sources should be integrated into one consistent view. Besides, it must be made clear that the response is not the single best way for clinical practice.
The two question answering modules each generate a number of answers, which may originate from different sources, but multiple answers may also have been retrieved from the same source. To be able to provide a direct link to the source the answer was retrieved from, and to make clear that the presented response is just a view provided by a particular source (with which a general practitioner could disagree), answers from different sources should not be integrated into one sentence. Instead, an overview should be provided of all different answers In this figure the three answers presented in the are grouped according to the two sources the answers were retrieved from. The first and second answers
RSI is hetzelfde als een muisarm. (RSI is the same as a mouse arm.)
Tegenwoordig is RSI een verzamelnaam ('paraplubegrip') voor alle klachten aan armen, nek en schouders. (Nowadays, RSI is a collective term for all complaints of arms, neck, and shoulders.)
were retrieved from the website of the RSI Association. These two answers have been
Tegenwoordig is RSI, ook wel een muisarm genoemd, een verzamelnaam ('paraplubegrip') voor alle klachten aan armen, nek en schouders. (Nowadays, RSI, also called a mouse arm) is a collective term for all complaints of arms, neck, and shoulders.)
Actually, this integrated sentence is quite advanced. The first answer is transformed into an appositive "ook wel een muisarm genoemd" ("also called a mouse arm") to the second answer. It is unlikely that a natural language processing system will produce such a sentence in the near future, but it would be very nice. The third answer
Ook RSI is eigenlijk een verkeerde term. (Actually, RSI is also a wrong term.)
was retrieved from TNO Arbeid. This answer is presented separately. Together with each group of answers the name of the source is provided as a link to that source. This kind of presentation can be specified in HTML and has to be contained by a visual content element in a P-ml document.
The decision that answers from different sources should not be integrated into one sentence is not universal to the IMOGEN project. Actually efforts are spent on the development of techniques to integrate similar answers retrieved from different documents into one sentence that is more specific or more general than the original answers [MK05].
Question answering for general practitioners
Tegenwoordig is RSI, ook wel een muisarm genoemd,
een verzamelnaam ('paraplubegrip') voor alle
klachten aan armen, nek en schouders.
Ook RSI is eigenlijk een verkeerde term.
Figure 9 Example presentation
A separate requirement for the output generation module specified that a different output generation module should be designed for information presentation on a PDA. In the case of a PDA, there is not enough space to show all different answers on the screen. Therefore, only one source should be selected for which the answer is presented. The answer should be accompanied by the name of the source, but it is useless to provide a link to the complete document the answer was retrieved from, because it would take too much effort to read it on PDA. of presentation could be represented simply as plain text in a content element of a P-ml document.
Tegenwoordig is RSI, ook
wel een muisarm genoemd,
een verzamelnaam
('paraplubegrip') voor
alle klachten aan armen,
nek en schouders.
Figure 10 Example presentation for PDA
To integrate several answers retrieved from the same source into one concise response, possibly aided by the surrounding (annotated) context, some natural language processing is needed. This language technology is described in cnot be modified by the output generation module. The string contained by a text element in the "dam.out" pool will thus be directly copied (as plain text) into a content element of a P-ml document.
4.3.2 Graphical user interface The graphical user interface controls the dialogue with the user. It accepts typed input and pressed buttons from the user and it displays the presentations generated by the output generation module in return.
In the first version of the IMIX demonstrator the graphical user interface has eight different states [IMIXg]. However, for GIPS, the states dealing with the choice of input mode ("modality") and speech input ("wake_up" and "speech_input") are not needed, because only text input will be implemented. The "welcome" state will also be omitted, because one of the requirements for the graphical user interface stated that all functional options should be immediately visible at the user interface. The system will therefore start in the "text_input" state.
When a question or other dialogue act is entered by pressing the ok button or typing [enter] (satisfying the requirement that both keyboard and mouse input must be accommodated), the user turn ends and the system turns to the "waiting_for_answer" state. When a dialogue act arrives that is not an answer, the system returns to the "text_input" state to allow the user to
Mieke van Langen
type a reply. When an answer arrives, the system goes to the "answer" state. From this state the user can either start a new dialogue or close the graphical user interface. Like in the first version of the IMIX demonstrator, the system can go to the "end" state from any of the other states by closing the graphical user interface. Besides, the user should also be able to start a new dialogue from any of the states to prevent the user from getting stuck in a fruitless dialogue. However, the dialogue will never be reset after a timeout, because if a physician turns away from the system, it must always be in precisely the same state when he returns to it. The new state transition diagram fo
presentation [no_answer]
presentation [answer]
Figure 11 State transition diagram for GIPS
An information portal for general practitioners
The general practitioners participating in the inteunanimous in their information needs and their expectations of QA systems. QA systems turned out to be suitable primarily for answering questions for patient education based on information retrieved from the Internet. Other types of questions might also be answered with information retrieved from the Internet, but question answering is not a suitable technique for finding an answer to these questions. To provide an overview of the different techniques available to general practitioners for answering their questions, a web portal could be developed that provides access to all types of applications useful for general practitioners.
Next to GIPS, the portal should provide access to an image retrieval system to retrieve dermatological images, information extraction technology to retrieve data like telephone numbers and e-mail addresses (the social map), the index of the guidelines issued by the Dutch College of General Practitioners [NHGs], and medical information retrieval systems like Artsennet [KNMG] and PubMed [EPM] to retrieve journal articles. Some of these applications, like GIPS itself, image retrieval, and information extraction are not yet available for use by general practitioners. Therefore, when a web portal would be implemented, it should be made very clear which types of information can and which cannot be found with the available applications to avoid general practitioners being frustrated because the application does not return the information they need.
Question answering for general practitioners
A prototype of such a web portal has been implemented in HTML and JavaScript. The lay-out of the portal resembles that of "startpagina.nl" which is a frequently used portal for Internet users. Each application is shown as a textbox headed with a colored title. The color of the title indicates the availability of the application. A blue title indicates that the application is available, a purple title means there is only a prototype of the application, and applications with a red title are not available at all, but might be available in the future, in which case they would be useful for general practitioners. For each application text fields are included in the portal in which the user can directly enter his question, diminishing the number of steps a user would need when he was to enter his question in the application itself after opening it.
GIPS is the only application with a purple title. A prototype of GIPS has also been implemented in HTML and JavaScript. This prototype consists of a simple graphical user interface in which the user can select a question from a list of questions. Then the answer to this question (generated previously with the prototype described in the next chapter) is shown in an answer field. One of the answers also includes a picture. In this prototype dialogue functionality is not implemented, and keyboard input has not been accommodated for.
Screenshots of the web portal and GIPS are shown in Appendix C. The prototypes themselves can be viewed at my website:
Mieke van Langen
Question answering for general practitioners
5 Response
formulation
In the previous chapter the design for GIPS (General practitioner Information Presentation Submodule) was presented. There, the need was expressed for natural language processing (NLP) techniques to integrate several answers into one concise and coherent response. NLP is a subfield of artificial intelligence and linguistics. It concerns the processing and understanding of natural language by computers. NLP is a broad field comprising, among others, speech recognition, natural language generation, question answering, information retrieval, information extraction, and automatic summarization.
In rmulation is presented. which response formulation methods are suitable for GIPS. These methods are elaborated in the implementation of the response formulation task is described.
To avoid confusion about the terminology, in this chapter the term ‘answer' will be used for the answer sentences returned by the question answering modules, while the term ‘response' will be used for the final (concise and coherent) answer generated by the output generation module.
5.1 Related
In previous QA research, little effort has been dedicated to response formulation. Most QA research has focused on improving system performance against a standard set of questions, like the QA track at the TREC conferences [TREC]. These questions require short factoid answers and system performance on these questions is measured by the rank at which the correct answer is returned by the system. No natural language responses need to be formulated for this task.
For the sake of better human computer interaction, response formulation is receiving more and more attention, though. Generally, two response formulation approaches can be discerned that are related to existing approaches for the QA stage of answer extraction: formulation templates sides, research is executed on the use of
5.1.1 Formulation
templates
Kosseim [KPG03] investigated the use of formulation templates. For example, the question
Who is the prime minister of Canada?
could be transformed into the templates:
The prime minister of Canada is <person-name>
<person-name>, prime minister of Canada,
These templates can be used for answer extraction as well as response formulation. In the case of answer extraction, a lot of possible formulations are produced. Then the QA system searches for these formulations in the document collection and instantiates <person-name> with the matching noun phrase. This noun phrase is then considered the answer. In the case of response formulation, only a few (or only one) template of good linguistic quality is produced. Then <person-name> is instantiated with the answer to produce a linguistically correct response.
I used formulation templates for a QA system that only answers factoid person questions [LAN05]. It is a relatively simple method that can only be used for short, factoid answers. As far as I know, the use of formulation templates for more extensive answer has not been investigated.
Mieke van Langen
5.1.2 Query-based
summarization
Query-based summarization aims at summarizing the part of a text relevant to the user's question. This method is used for answer extraction. For example, Cardie et al. [CNP00] investigated the use of a variant of the vector space model (primarily used by information retrieval systems) to generate a query-based summarization. With this method, the answer document is divided into chunks (e.g. sentences, paragraphs, 200-word passages), a vector representation is generated for the question and for each document chunk, then the similarity of each chunk to the question is determined, and the most similar chunk (up to a predetermined length) is taken to be the query-dependent summary. This summary is then used to extract possible answers to the question.
However, query-based summarizations also seem appropriate as response. According to Lin et al. [LQS03], the most natural response presentation style for a QA system is "focus-plus-context". This means the system returns the answer to the user's question, extended with the text surrounding the answer. Lin et al. investigated which context level is preferred by users: only the exact answer; the sentence from which the answer was extracted; the paragraph; or the entire document. They concluded that users prefer to receive the paragraph the answer was retrieved from.
Bosma [BOS05] investigated a more intelligent way of producing query-based summarizations. He used discourse annotations to determine which context sentences are most related to the sentence in which the answer is located. Only these sentences (which are not necessarily subsequent to each other in the original document) are included in the response, instead of the entire paragraph. This response was compared to a baseline consisting of the answer sentence extended with the preceding and the successive sentence. Users were asked to evaluate the query-based summarizations and the baseline on the extent to which their accurateness could be verified, on the usefulness with respect to the question, and on the amount of irrelevant information with respect to the question. It turned out that users thought Bosma's query-based summarizations were more verifiable, and contained less irrelevant information than the baseline response.
Actually, in their methods, Lin et al. and Bosma don't use the query to produce a query-based summarization at all. They only use the answer sentence as starting-point to determine which sentences to include in the response. Therefore, I call this method "answer extension", instead of "query-based summarization".
5.1.3 Sentence
Sentence fusion is used especially in multi-document summarization to summarize the information common to all documents. As an example Barzilay [BAR03] presents the sentences:
IDF Spokeswoman did not confirm this, but said the Palestinians fired an anti-tank missile at a bulldozer.
The clash erupted when Palestinian militants fired machine-guns and anti-tank missiles at a bulldozer that was building an embankment in the area to better protect Israelian forces.
These sentences would be fused into the sentence:
Palestinians fired an anti-tank missile at a bulldozer.
Sentence fusion could also be used in QA systems for response formulation. Marsi and Krahmer [MK05] investigate a method for Dutch to integrate similar answers retrieved from different documents into one sentence that is more specific or more general (depending on the goal of the system) than the original answer sentences. For example, the answers
Question answering for general practitioners
RSI can be caused by repeating the same sequence of movements many times an hour or day.
RSI is generally caused by a mixture of poor ergonomics, stress and poor posture.
Might be fused into a more specific response, like:
RSI can be caused by a mixture of poor ergonomics, stress, poor posture and by repeating the same sequence of movements many times an hour or day.
The method developed by Marsi and Krahmer comprises three stages: alignment, merging, and generation. During alignment, words and phrases in the different sentences that are related to each other are aligned, and each alignment is labeled with the semantic relation holding between the aligned phrases (e.g. equals, restates, specifies). In the merging stage it is decided which information from either sentence should be preserved. Finally, a grammatically correct surface representation is generated for the fused sentence. Marsi and Krahmer haven't investigated their method of sentence fusion for QA yet. They only evaluated it on parallel corpora.
5.2 GIPS
To get an idea of the possible answers received as input for GIPS, ten example questions about
RSI were submitted to the question answering module "qadr.qa". This module was used
because it provides the dependency structures of the question, the answer sentences, and the
sentences in their context, which can be useful for subsequent natural language processing.
More information on dependency structures is provided later in this chapter. The example
questions and their answers are presented in detail in Appendix D.
On inspection of the answers there appear to be two different types of answers. The first type can be interpreted without knowing the original context of the answer sentence. I call these "autonomous answers". For example, on the first question in Appendix D:
Wat is RSI? (What is RSI?)
one of the answers is
RSI is een verzamelnaam voor zeer uiteenlopende vormen van overbelasting in het gebied van nek, schouders, armen en ellebogen. (RSI is a collective term for very different forms of overload in the area of neck, shoulders, arms, and elbows.)
This answer can be interpreted easily, because it simply gives a definition.
However, there is a second type of answers that can hardly be interpreted without knowing their context. I call these "dependent answers". Many of these answers are not relevant for the question, but seem to have a strong relation with a sentence in their context that is necessary to correctly interpret the answer and would be very relevant for the question. For example, on the third question in Appendix D:
Welke spieren zijn betrokken bij RSI? (Which muscles are affected by RSI?)
one of the answers is
Uit onderzoek is bekend dat deze het vaakst betrokken is bij RSI. (It is known from research that this one is affected by RSI most often.)
This answer cannot be entirely interpreted, because it contains a referring expression "deze" ("this one"). This expression probably refers to a muscle. It is expected that the previous
Mieke van Langen
sentence contains the name of this referred muscle, which would be very relevant for the question.
Because there are two different types of answers, also two different response formulation strategies are proposed. Autonomous answer sentences retrieved from the same document could be integrated into a single sentence using some sentence fusion method. This is to be extended with sentences from their original context, using some answer extension method, to make them interpretable and
5.3 Answer
integration
Autonomous answers could be integrated into one sentence to make the response more concise and fluent. However, GIPS should only integrate answers originating from the same document into one sentence, because the general practitioner receiving the response must be able to check its source. Besides, it is hypothesized that general practitioners will more readily accept and trust a response accompanied by a link to its source, than a computer generated response that cannot be easily verified.
Within NLP, two different approaches for integrating sentences have been investigated: aggregation and sentence fusion. Aggregation is used in natural language generation (NLG) to make generated text more coherent by reducing redundant information and introducing connectives [RM99, SHA02]. NLG is defined as "the process of constructing natural language outputs from non-linguistic inputs" [JM00], which is also called concept-to-text generation [LAP03]. In concept-to-text generation, information that is represented in a knowledge base or other logical representation is transformed into natural language. For example, the logical proposition "likes(John, Mary) AND likes(Mary, John)" could be transformed into the natural language phrase "John and Mary like each other.".
Sentence fusion is used in text-to-text generation to create a concise and fluent fusion of two or more sentences [BAR03]. Text-to-text generation methods take information that is represented in natural language as input, and transform it into a new natural language representation satisfying certain constraints. For example, in automatic summarization, sentences could be deleted or fused to represent only the most important information. Sentence fusion is used especially in multi-document summarization to summarize the information common to all documents.
The response formulation task investigated in this chapter concerns text-to-text generation, because it starts with text snippets (answer sentences) retrieved by the question answering modules and aims at integrating them into a single coherent text (response). Therefore, sentence fusion (being a text-to-text generation task) seems the most appropriate approach for answer integration. Actually, the use of sentence fusion for the IMIX demonstrator is investigated at the University of Tilburg by Marsi and Krahmer [MK05]. However, I think that the concept-to-text generation field of aggregation could also provide useful insights for the response formulation task, because it has been investigated more thoroughly. Whereas sentence fusion primarily concerns the fusion of similar sentences (like in multi-document summarization or fusion of parallel corpora), aggregation concerns the integration of sentences connected by all sorts of relations. Therefore, I investigated the possibilities to use aggregation for response formulation.
The next subsection provides a general description of aggregation. Then inare drawn with respect to the aggregation types that can be used for the response formulation hm is presented for answer integration.
Question answering for general practitioners
5.3.1 Aggregation Reape and Mellish [RM99] conducted a literature survey to investigate what aggregation is. They concluded that aggregation is "the combination of two or more linguistic structures into a single linguistic structure which contributes to sentence structuring and construction". Aggregation roughly consists of two stages [SHA02]. In the first stage, it is decided which linguistic structures might be aggregated. Mostly, linguistic structures are aggregated when they have a rhetorical relation or when they show some similarity. In the second stage, transformations are applied to these structures. For this purpose linguistic constructions are used, such as conjunction, adjective phrase attachment, quantification, and gapping. Examples of these constructions ar
Aggregation is usually used to make computer generated text more concise, cohesive, and/or fluent and (consequently) to generate more complex sentences [SHA02]. For the sake of fluency, the linear ordering of aggregated constituents is also important. For example, the sentence "Mary is a sweet little girl." is more fluent than "Mary is a little sweet girl.", while both sentences are grammatical. Therefore, when two sentences are aggregated, a linear ordering of their constituents should also be specified.
There are several types of aggregation. Each author makes a different classification of aggregation types. Reape and Mellish [RM99] tried to integrate these classifications by introducing a classification based on levels of linguistic representation. They discern six different aggregation associated with these levels of linguistic representation.
Linguistic construction
Sue invited John. Sue invited Mary. → Sue invited John and Mary.
Adjective phrase attachment
Mary is a girl. Mary is sweet. → Mary is a sweet girl.
John was invited. Mary was invited. John and Mary are the only children. → All children were invited.
Gapping (deletion of a second verb) John hit Mary.
Phil hit Sue. → John hit Mary and Phil ∅ Sue.
Right-node-raising
John likes Mary. Phil hates Mary. → John likes ∅ and Phil hates Mary.
Table 8 Examples of linguistic constructions
5.3.2 Conceptual
aggregation
The first level, the conceptual or inferential level, concerns non-linguistic, language-independent, domain knowledge contained in some sort of knowledge base. Conceptual aggregation concerns the use of user modeling, domain knowledge, and common sense reasoning to reduce the number of concepts [SHA98]. This type of aggregation typically
Mieke van Langen
reduces the number of propositions in the message while increasing the complexity of the value of some conceptual role. For example, the two sentences "John hit Mary" and "Mary kicked John" might result in the aggregated sentence "John and Mary fought."
Aggregation type
Description
Conceptual aggregation
Reduction of the number of concepts
Discourse aggregation
Reduction of the complexity of the rhetorical structure
Semantic aggregation
Combination of two or more semantic entities
Syntactic aggregation
Combination of sentences using syntactic constructions
Lexical aggregation
Reduction of the number of lexical predicates and/or lexemes
Referential aggregation
Referring expression generation
Table 9 Types of aggregation
5.3.3 Discourse
aggregation
The second level, discourse, concerns the coherence of a text. Text coherence implicates that utterances in a text are linguistically as well as non-linguistically connected to each other [HAA02a]. Besides, the utterances must contribute to the rhetorical structure of the text. A theory that describes the different rhetorical relations between sentences is the Rhetorical Structure Theory (RST) developed by Mann and Thompson [MT87]. Most RST relations are asymmetric. In that case one of the sentences is considered the nucleus (conveying the most essential information) and the other the satellite. However, there are also multi-nuclear relations in which there is no distinction between nucleus RST relations [JM00]. RST relations are hierarchical, therefore the rhetorical structure of a text
Relation
Description
Elaboration The satellite presents some additional detail concerning the nucleus.
e.g. John likes girls. He likes Mary most.
The nuclei present things that are different in some relevant way. e.g. John likes girls. He doesn't like Sue.
Sequence The nuclei are realized in succession.
e.g. John invites Mary. John invites Sue.
The satellite presents the goal of performing the activity presented in the nucleus. e.g. John invites Mary. He wants her to come to his party.
The situation presented in the satellite results from the one presented in the nucleus. e.g. John invites Mary. She comes to his party.
Table 10 Examples of RST relations
Discourse or rhetorical aggregation is defined as "any operation that applies to a discourse structure, rhetorical structure or text plan and maps it to a better structure or plan" [RM99]. An example of discourse aggregation might be the mapping of the rhetorical structure tree
E(nuc(E(nuc(n),sat(p1))),sat(p2)) (for example: "John like girls. He likes Mary most. He also likes Sue.")
Question answering for general practitioners
E(nuc(n),sat(and(p1,p2))) (for example: "John likes girls. He likes Mary most and he also likes Sue.")
where "E" is the elaboration relation, "nuc" is the nucleus, and "sat" is the satellite. These trees and respectively. As this example illustrates, discourse aggregation typically reduces the complexity of the rhetorical structure while increasing the complexity of one of its propositional leaves.
Figure 12 Rhetorical structure tree E(nuc(E(nuc(n),sat(p1))),sat(p2))
Figure 13 Rhetorical structure tree E(nuc(n),sat(and(p1,p2)))
5.3.4 Semantic
aggregation
The semantic level concerns linguistic, language-dependent representations of meaning. In contrast to the conceptual level, the information at this level is domain-independent [RM99]. Semantic aggregation is defined as the combination of two or more semantic entities into one entity. Methods of semantic aggregation are semantic grouping and logical transformations. Semantic grouping is the ordering and bracketing of semantic content. Logical transformations concern the mapping of semantic predicates into fewer or just different predicates. For example, the meanings of "Jamie is Chris's sister" and "Chris is Jamie's brother" might be mapped to the meaning of "Chris and Jamie are brother and sister".
The distinction between conceptual and semantic aggregation is difficult. Actually, other authors, like Shaw [SHA02], don't distinguish between these types of aggregation. Instead, Shaw considers all aggregation operations that make no use of syntactic knowledge or lexicon (except for referential aggregation) interpretive aggregation. According to Shaw, interpretive aggregation operators perform inferences over conceptions and relations across propositions. Reape and Mellish [RM99] also admit that they couldn't find any clear examples of semantic
Mieke van Langen
aggregation "which couldn't alternatively be classified as either conceptual, syntactic or lexical aggregation". This type of aggregation is thus a bit doubtful.
5.3.5 Syntactic
aggregation
Syntax refers to the way words are arranged together [JM00]. Words belong to different word classes, implying restrictions on the way they can be used in a sentence. Syntactic aggregation is the most common form of aggregation [RM99]. It combines propositions using syntactic constructions, like conjunction, gapping, etc. [SHA02]. Syntactic aggregation can be paratactic or hypotactic. In paratactic aggregation the aggregated sentences are of equal syntactic status. The main paratactic aggregation operator is the coordinating conjunction, a linguistic construction that uses a coordinator (like "and", "or", "but") to link linguistic units of equal syntactic status. For example the sentences "John likes school." and "Mary likes school." can be aggregated as "John and Mary like school.". The coordinating conjunction can be used to combine propositions that have an addition, sequence, or non-volitional result rhetorical relation.
The clauses in hypotactic constructions have unequal syntactic status [SHA02]. For example, when two propositions have an elaboration relation, the proposition in satellite position can be transformed into a modifying construction, such as an adjectival phrase, a prepositional phrase, or a relative clause, like the transformation of "Mary is sweet." into an adjective in "Mary is a sweet girl.". Lexical information is used to determine if the result of hypotactic aggregation doesn't violate any syntactic or lexical constraints. For example, when the sentences "Mary is a girl." and "Mary is sweet." are aggregated as "Mary is a sweet girl.", "girl" must be realizable using an adjective, and "sweet" must be realizable as a pronominal modifier. Such restrictions are coded in a lexicon.
Rhetorical relations are very important in choosing a linguistic construction for syntactic aggregation. Shaw [SHA02] gives the following example to show how different rhetorical relations lead to very different aggregated sentences.
Consider the following two sentences:
1. John abused the duck.
2. The duck buzzed John.
When the main rhetorical relation connecting the nucleus and satellite is elaboration, the sentences might be aggregated by a relative clause, resulting in the following aggregated sentences (depending on which of the sentences was the nucleus):
a. John abused the duck that had buzzed him.
b. The duck buzzed John who had abused it.
By using the past perfect tense in the relative clauses, the nucleus and satellite also have a sequence relation, therefore sentences a en b describe very different situations. In the first, John was the victim first before he became an aggressor, while in the second the duck was the victim first.
When the main rhetorical relation connecting the nucleus and satellite is sequence, non-volitional result, or addition, the sentences might be aggregated by conjunction, resulting in the sentences c and d.
c. The duck buzzed John and he abused it.
d. John abused the duck and it buzzed him.
Thus, at least four different aggregated sentences can result from two sentences, depending on their rhetorical relation.
Question answering for general practitioners
5.3.6 Lexical
aggregation
The lexical level represents the concatenation of morphemes making up a word [JM00]. Lexical aggregation combines multiple lexical items to express them more concisely [SHA02]. This operation is related to paraphrasing. Compared with hypotactic aggregation, lexical aggregation operators use more detailed lexical information. For example, the phrase "a dog used by the police" might be transformed into "a police dog", transforming the reduced relative clause into a prenominal modifier. Another type of lexical aggregation is the combination of multiple lexemes into one, like the transformation of the phrase "rise sharply" into "shoot".
5.3.7 Referential
aggregation
The last type of aggregation, referential aggregation, is usually associated with referring expression generation [RM99]. Referring expression generation concerns the linking of words by introducing pronouns, demonstratives, and other types of reference [JM00]. Another type of referential aggregation, however, is quantification [SHA02]. Quantification replaces a set of entities with a reference to their type (based on ontology) as restricted by a quantifier. For example, when it is known that John and Mary are the only students, the sentences "John likes school" and "Mary likes school" could be transformed into "all students like school". For this type of aggregation an ontology is needed that provides information on instance-class relations, inheritance relations, and part-of relations between different entities.
However, referring expressions and quantification can introduce ambiguity, when applied incorrectly. For example, it might be unclear to which entity a referring expression refers [JM00], or when multiple quantifiers are synthesized in the same sentence, the scope of the quantifiers could be ambiguous [SHA02]. Thus, referential aggregation should be used cautiously.
5.3.8 Conclusions Because the task of response formulation is a text-to-text generation process instead of a concept-to-text generation process, not all levels of aggregation are relevant for the response formulation task. Conceptual, discourse, and semantic aggregation suggest an underlying knowledge and rhetorical structure presentation of the text that is not available to GIPS. Transforming the answers or the entire original document into a knowledge representation would extremely increase the complexity of the aggregation task. Therefore, conceptual, discourse, and semantic aggregation will not be used.
Syntactic aggregation is especially useful for the response formulation task, because it concerns using linguistic constructions to integrate sentences, while preserving their grammatical and lexical correctness. However, a specific syntactic construction can only be used to aggregate sentences having a certain rhetorical relation. Therefore, the rhetorical relations between the answers should be determined. This is very difficult, because the answers to be integrated are autonomous and usually not subsequent in the original text. They are all suggested to present an answer on the same question, however. Their rhetorical relation could thus be suggested to be addition. The primary syntactic construction used to aggregate sentences having an addition relation is the coordinating conjunction. This construction could thus be used to aggregate different autonomous answers retrieved from the same document.
For lexical aggregation a lot of morphological and semantic knowledge about Dutch words is needed, and it primarily concerns aggregation within sentences. This type of aggregation is probably more useful for concept-to-text generation processes in which lexical choices still have to be made. In the case of GIPS, answers already have their lexical representation. Therefore, this type of aggregation will not be used for this research. However, some lexical knowledge might be needed to correctly execute syntactic aggregation.
Finally, referential aggregation would be very useful to make a text more coherent, but it can introduce ambiguity when applied incorrectly. In an application for general practitioners
Mieke van Langen
ambiguity must be avoided as much as possible. Therefore, this type of aggregation will also be omitted.
5.3.9 Answer integration algorithm In the previous section it was concluded that answers should be integrated using a coordinating conjunction construction. Shaw [SHA02] presents a conjunction algorithm for an NLG system that incorporates different types of ellipsis, like gapping and right-node-raising. With some minor adjustments, this algorithm could also be very useful for the answer integration task. Shaw's algorithm consists of the following steps:
1. group propositions and order them according to their similarities while satisfying pragmatic
and contextual constraints;
2. determine recurring elements in the ordered propositions being combined; 3. create a sentence boundary when the combined clause reaches a-priori thresholds; 4. decide which recurring elements are redundant and should be deleted.
The first step, the grouping and ordering of answers, could be done by grouping the answers per source and ordering them according to the order of the original document to preserve the discourse structures of the original document. Then, based on the number of answers per source and their similarity, it will be determined which answers should be aggregated. To avoid generating too complex sentences, no more then two answers should be integrated into a single sentence. When there are more then two answers, the similarity of all subsequent answers could be computed to determine which couples could best be integrated.
When two answers are to be integrated, the second step is to determine recurring elements. Recurring elements could be deleted in the fourth step. Therefore, they need to be exactly identical. To check whether two constituents are identical, Shaw proposes two equivalence tests: the alphabet equivalence test; and the sense equivalence test. Alphabet equivalence concerns the surface form of a constituent. This can be easily checked by comparing the respective strings. Sense equivalence concerns "the identity of the indexicals". Shaw says that for nouns this means that their entity identifiers should be tested, and for verbs and adjectives their lexical senses should be tested. However, this is only possible in concept-to-text generation. In the case of the answers returned by the question answering modules, the entire original text should be analyzed to determine the senses of the nouns, verbs and adjectives. This would extremely complicate the algorithm. Instead, the dependency structures with which the answers are annotated could be used as an extra equivalence test. Dependency relations have a tree structure. They are described in detail in the next section. In short, identical constituents should have the same relation, POS and root tags in the case of a single word, or the same relation, cat tags and child nodes in the case of a phrase. Otherwise, they are not identical.
The third step, creating a sentence boundary, can be omitted, because the number of answers to be integrated was already restricted to two.
Finally, in the fourth step the sentences are joined using the coordinator "and", and recurring elements are deleted. When a recurring element, or a group of recurring elements, is realized at the end of a sentence, it should be deleted backward, meaning that the first occurrence of the identical constituent is deleted. Otherwise, it should be deleted forward, thus deleting the second occurrence of the identical constituent. According to Shaw, this directionality is a universal phenomenon, also valid for Dutch. For example, consider the following sentences:
a. Mary eats an apple in the morning.
eats a banana in the morning.
c. Mary eats an apple, and Sue a banana in the morning.
Question answering for general practitioners
In sentences a and b two recurring elements can be identified: "eats", and "in the morning". The first element, "eats", is realized in the middle of the sentence and should thus be deleted forward. The other is realized at the end of the sentence and is thus deleted backward. The deleted elements are underlined in sentences a and b. Sentence c is the aggregated sentence.
Not all sentences can be aggregated that easily, however. Shaw [SHA02] identifies several additional constraints on the conjunction algorithm dealing with scope ambiguities and morphology. Scope ambiguities can occur with modifiers, negation, and quantifiers. For example, when the phrases "tall men" and "women" are aggregated as "tall men and women", it is not clear whether the women are also tall. To avoid these kinds of ambiguity, the elements should be reordered to make the scope clear, like in "women and tall men". Morphological problems can occur when, for example, number agreement rules are harmed, like in "Mary and Sue eats an apple.".
Compared to the sentence fusion algorithm described by Marsi and Krahmer [MK05], this conjunction algorithm is more rigid. In this algorithm only identical constituents are deleted, whereas in the sentence fusion algorithm also constituents having a specification relation or restatements can be fused. Consider the exam
RSI can be caused by repeating the same sequence of movements many times an hour or day.
RSI is generally caused by a mixture of poor ergonomics, stress and poor posture.
The sentence fusion algorithm aligns the phrases "can be caused by" and "is generally caused by" and labels them as restatements of each other. Then one of these phrases is arbitrarily chosen to be preserved in the fused sentence. Instead, the conjunction algorithm would sign the words "caused" and "by" as recurring elements, deleting them forward, which would result in the ungrammatical sentence:
RSI can be caused by repeating the same sequence of movements many times an hour or day, and is generally a mixture of poor ergonomics, stress and poor posture.
Thus, the sentence fusion algorithm is more complex to implement, but it seems to be a more natural solution for answer integration then this conjunction algorithm. Sentence fusion appears to incorporate coordinating conjunction constructions, as well as lexical aggregation by aligning not only identical but also similar phrases.
5.4 Answer
extension
When an answer cannot be interpreted without knowing its context, it should be extended with the sentences most related to this answer to formulate a coherent response. Therefore, the rhetorical relations between the sentences in the context of the answer and the answer itself should be determined. In a natural language generation system, the rhetorical relation between two sentences should be specified by a component called a content planner [SHA02] or discourse planner [JM00]. However, the response formulation task described in this chapter is a text-to-text generation process. The rhetorical relations must thus be inferred from the text. Bosma [BOS05] used RST annotations for his answer extension algorithm. However, there aren't any automated RST analysis tools available for Dutch yet. Therefore, for GIPS a simple algorithm has been developed that detects sentences that are strongly related to a given answer.
It is assumed that strongly related sentences have been aggregated on some level by the author. Conceptual, semantic, syntactic, and lexical aggregation are primarily used to aggregate clauses or phrases within sentences or to integrate multiple sentences or propositions into one.
Mieke van Langen
Aggregation operators that are used to express a relation between two sentences (instead of within a sentence) are discourse and referential aggregation. Discourse and referential aggregation operators can be recognized by certain linguistic markers, like cue phrases document structure ibed that is used to select the sentences that should be included in the answer.
5.4.1 Cue
In discourse aggregation, linguistic devices are used to signal rhetorical relations explicitly [KS98]. These devices are called cue phrases, or discourse connectives. Power et al. [PSB03] discern three different types of discourse connectives: subordinating conjunctions, coordinating conjunctions, and conjunctive adverbs. Subordinating conjunctions (like "although", "because") connect a nucleus and satellite that must be expressed within the same sentence. The conjunction can be located either in the first or in the second clause. Coordinating conjunctions ("and", "or", "but") connect two nuclei either occurring in the same sentence or in different sentences. The conjunction always occurs in the second span. In Dutch there are five coordinating conjunctions [KS04]: "en" ("and"), "maar" ("but"), "want" ("for"), "dus" ("so"), and "of" ("or"). Conjunctive adverbs ("however", "moreover") always connect text spans occurring in different sentences. The adverb is located in the second sentence.
In this research only cue phrases that indicate a relation between two sentences are relevant, because they are used to determine whether a rhetorical relation exists between these sentences. Therefore, only coordinating conjunctions connecting text spans expressed in different sentences, and conjunctive adverbs are relevant for this research. These cue phrases always occur in the second of the two related sentences.
Figure 14 Knott and Sanders' Dutch cue phrase taxonomy
Knott and Sanders [KS98] constructed a cue phrase taxonomy expressing relationships This taxonomy doesn't describe the total set of Dutch cue phrases, however, because it was primarily used for "a first theory-driven systematic
Question answering for general practitioners
and cross-linguistic cue phrase study" in which the use of Dutch cue phrases was compared to that of English cue phrases. Besides, it also includes subordinating conjunctions like "omdat" ("because"), which are not relevant for this research.
To be able to recognize the relevant cue phrases, a list of cue phrases that signal a rhetorical relation between two sentences has been constructed. Therefore, a small Dutch textcorpus (about 10,000 words) containing paragraphs from the IMIX document collection and NHG patient education documents [NHGp], has been analyzed. Firstly, all conjunctions and adverbs that signal a rhetorical relation between the sentence they occur in and the previous sentence were marked manually. Then, it was investigated how these phrases could be recognized as connecting two sentences, because coordinating conjunctions could also connect text spans within a sentence, and because in Dutch some conjunctions and conjunctive adverbs are ambiguous. For example, the additive cue phrase "ook" identified by Knott and Sanders is a conjunctive adverb (meaning "also") in:
Jantje houdt van zwemmen. Ook Pietje zwemt graag. (John likes swimming. Peter also likes to swim.)
but it is an elliptic device (meaning "too") connecting two phrases within a sentence in:
Jantje houdt van zwemmen en Pietje ook. (John likes swimming and Peter too.)
Therefore, all occurrences of the marked words in the text were examined manually to identify under which circumstances they could be determined to be a cue phrase connecting two sentences. For this analysis, dependency trees were used.
Figure 15 Example dependency tree
Dependency trees make explicit the dependency relations between constituents in a sentence [BNM01]. Each non-terminal node in a dependency tree is connected with a head-daughter and one or more non-head daughters, whose dependency relations to the head are specified in a relation tag. For he dependency tree of the sentence "Jantje houdt van zwemmen en Pietje ook." is shown. The "top" node of this tree is connected with three leaf nodes (belonging to the words "en Pietje ook") and a non-terminal node "main". The head-
Mieke van Langen
daughter of the "main" node is the verb "houdt". Besides, the main node has two non-head daughters: one is the subject of the head ("Jantje"), the other is a prepositional complement. This prepositional complement in turn has a head-daughter (the preposition "van") and a non-head daughter (the verb "zwemmen"), which is the object of the head.
One of the question answering modules of the IMIX demonstrator ("qadr.qa") provides dependency structures of the question, the answer sentences, and the sentences in the context of the answers. These dependency structures are generated by the Alpino parser [BNM01]. In the next version of the IMIX demonstrator this parser will be available for all IMIX modules, enabling the output generation module to procure dependency structures also for the answers returned by the other question answering module ("rolaquad.qa"). Next to the relation to the head, the Alpino parser provides a POS-tag, the root, and the original word for each leaf node in the dependency tree.
With the analysis described above, three coordinating conjunctions were identfied: "en", "maar", and "of". These conjunctions were found to only signal a relation between two sentences, when they are the first word of the second sentence. The other two Dutch coordinating conjunctions, "dus" and "want", didn't occur as the first word of a sentence in this corpus. However, because all coordinating conjunctions are used in the same way, these conjunctions would also be proper cue phrases when they are the first word of a sentence.
The coordinating conjunctions "of" and "dus" could also be used in another role, however [KS04]. "Of" could also be a subordinating conjunction ("whether"). In that case it would definitely not be a cue phrase. When a sentence starts with a coordinating conjunction, this conjunction is attached directly to the top node of the corresponding dependency tree. When a sentence starts with a subordinating conjunction, however, this conjunction has a parent node labeled with the POS-tag "cp". Therefore, when a sentence starts with "of" it should firstly be checked whether this phrase's parent node is the top node, before marking it as a cue phrase.
The coordinating conjunction "dus" could also be an adverb, but in that case it could still be a cue phrase (especially when it is the first word of a sentence). Therefore, no extra checking is needed for this phrase. The Dutch coordinating conjunctions together with the conditions they should satisfy to be determined a cue phrase. Some of these conjunctions ("want" and "maar") are also Dutch nouns. However, in that case they will probably not be the first word of a sentence. Therefore, the POS-tags of these words do not need to be checked.
Coordinating conjunction Constraints
Only when it is the first word of the sentence.
Only when it is the first word of the sentence.
Only when it is the first word of the sentence and its parent node is the top node.
Only when it is the first word of the sentence.
Only when it is the first word of the sentence.
Table 11 Coordinating conjunctions for Dutch
Next to these coordinating conjunctions, several different conjunctive adverbs were identified. Although according to Power et al. [PSB03] conjunctive adverbs always connect text spans occurring in different sentences, in the analyzed corpus the adverbs are also sometimes used to connect clauses within a sentence. Besides, they are also used as other types of adverbs, or even other parts of speech.
Based on the analysis of the conjunctive adverbs, some regularities were discovered
Question answering for general practitioners
. Firstly, the POS-tag of the adverb should of course be "adv". Unfortunately, some of the conjunctive adverbs are labeled as adjectives or prepositional phrases by the Alpino parser. Those should still be considered cue phrases, however.
Secondly, it was found that a conjunctive adverb probably doesn't connect the sentence with the previous sentence in the following cases:
when it is embraced by brackets; when the relation of one of its parent nodes in the dependency tree is modifier (labeled by
the Alpino parser with the relation tag "mod");
when the sentence starts with a subordinated clause (when the first word of the sentence
has a parent node labeled with the relation tag "mod" and the POS-tag "cp");
when it is positioned in a clause following a semi-colon or colon; when it is positioned after a conjunction or pronoun (labeled by the Alpino parser with the
POS-tags "vg" or "pron") whose direct parent node is also a parent node of the adverb, unless this conjunction or pronoun is the first word of the sentence;
when it is positioned in the second part of a dependency tree whose top node has only two
daughters, both labeled with the relation tag "dp".
The first rule concerns the use of brackets. When a cue phrase is positioned in a bracketed phrase, like "bijvoorbeeld" ("like") in:
Over de bijdrage van persoonsgebonden risicofactoren (bijvoorbeeld lichaamsbouw, het omgaan met stress) aan de kans op het krijgen van RSI is nog vrijwel niets bekend. (Little is known of the influence of personal risk factors (like body structure, or dealing with stress) on the chance of getting RSI.)
it probably indicates the relation of the bracketed part to the rest of the sentence. Therefore bracketed parts should be discarded.
The second rule concerns modifiers. All cue phrases are normally modifiers themselves. However, when a cue phrase is located inside a larger modifier, it probably does not relate to the previous sentence but to the head of the modifier it is part of. For example, in:
Met een normale bloedsuiker wordt de kans op bijvoorbeeld hart- en vaatziekten kleiner. (A normal blood suger reduces the chance on for example heart and vascular diseases.)
the phrase "op bijvoorbeeld hart- en vaatziekten" ("on for example heart and vascular diseases") is a modifier of the noun "kans" ("chance"). In this case the cue phrase "bijvoorbeeld" ("for example") clearly does not signal a relation with the previous sentence. Therefore, cue phrases that have a parent node labeled "mod" are discarded.
The third rule concerns subordinated clauses. When a sentence starts with a subordinated clause, like:
Hoewel daarbij vrijwel continu bewogen en dus afgewisseld wordt, leidt typen er toch toe dat steeds dezelfde spiergroepen gespannen zijn. (Although this involves almost continuous movement and thus variation, typing still causes the same muscle groups to be constantly tense.)
a cue phrase positioned after this subordinated clause, like "toch" ("still"), probably relates the second clause with the subordinated clause. A cue phrase that is positioned within a subordinated clause could signal a relation with the previous sentence as well as with the next clause. Therefore, it was decided that sentences starting with a subordinating clause had better not be searched for cue phrases.
The fourth rule concerns colons and semicolons. When a cue phrase is positioned after a colon or semicolon, it probably relates to the part preceding the colon or semicolon. Thus when a
Mieke van Langen
sentence contains one of these punctuation marks, only the part preceding it will be searched for cue phrases.
The fifth rule concerns conjunctions and pronouns. When a cue phrase is positioned after a conjunction or pronoun, it probably doesn't refer to the previous sentence, but to something within the sentence preceding the conjunction or pronoun. However, this is only the case when the cue phrase is contained within the scope of the conjunction or pronoun. This means that the direct parent node of the conjunction or pronoun in the dependency tree is also a parent of the cue phrase. For example, in:
Hoewel daarbij vrijwel continu bewogen en dus afgewisseld wordt, leidt typen er toch toe dat steeds dezelfde spiergroepen gespannen zijn. (Although this involves almost continuous movement and thus variation, typing still causes the same muscle groups to be constantly tense.)
the cue phrase "dus" ("thus") occurs after and within the scope of the conjunction "en" ("and"). It doesn't signal a relation with the previous sentence. On the contrary, in:
Het risico op hart- en vaatziekten wordt echter niet alleen door de bloeddruk bepaald. (The risk of heart and vascular diseases is not only determined by blood pressure, however.)
the cue phrase "echter" ("however") occurs after but outside the scope of the conjunction "en" ("and"). This cue phrase does signal a relation with the previous sentence. An exception to this rule occurs when the conjunction or pronoun is the first word of the sentence. In that case cue phrases occurring within the scope of the conjunction or cue phrase probably still refer to the previous sentence, like "toch" ("yet") in:
En toch ontstaan vaak klachten, zelfs veel meer dan met die zware typmachines van vroeger. (And yet a lot of complaints arise, even more than with those ancient typing machines.)
Finally, the sixth rule concerns sentences consisting of two parts that might as well have been two different sentences. Cue phrases located in the second part of such sentences probably signal a relation with the first part instead of with the previous sentence. Therefore those second parts should also be discarded.
The above mentioned rules are not universal. For example, cue phrases positioned after a colon might still connect this sentence with the previous sentence. However, it is far more likely that it relates the second part of the sentence to its first part.
For some of the conjunctive adverbs additional cconjunctive adverbs are listed, along with their constraints. Of course, the corpus used to extract these cue phrases is relatively small. Probably, in a larger corpus more cue phrases and more precise constraints could be detected, but this list provides a good starting point for the algorithm used in this research.
Specific constraints were identified for "bijvoorbeeld" ("like", "for example") and "wel" ("still", "however"). "Bijvoorbeeld" is frequently used in appositive constructions. Like in
Deskundigen (bijvoorbeeld artsen) stimuleren het gebruik van pauzesoftware. (Experts (like physicians) stimulate the use of break programs.)
Deskundigen, bijvoorbeeld artsen, stimuleren het gebruik van pauzesoftware. (Experts, like physicians, stimulate the use of break programs.)
Question answering for general practitioners
In that case, it does not connect two sentences, but is just a modifier of the phrase it is in apposition with. Frequently, this use of "bijvoorbeeld" can be recognized because one of the general constraints (like the use of brackets, or a parent with relation "modifier") is met. However, this is not always the case. A relatively easy way to recognize those occurrences not embraced by brackets or contained in a modifier, is by looking at commas: when "bijvoorbeeld" is directly preceded by a comma, it is probably part of an appositive. Only when it is also directly followed by a comma, like in:
Deskundigen, bijvoorbeeld, stimuleren het gebruik van pauzesoftware. (Experts, for example, stimulate the use of break programs.)
it is surely not part of an appositive and, if no other constraints are harmed, it might well be a conjunctive adverb connecting two sentences.
Conjunctive adverb Constraints
Not when it occurs directly after a comma, unless it is also directly followed by a comma.
This phrase is labeled with the POS-tag "pp" by the Alpino parser.
This phrase is labeled with the POS-tag "pp" by the Alpino parser.
This cue phrase poses the same conditions as its constituent adverbs "dan" and "ook". It will therefore not be discerned as a separate cue phrase, though its use is different from that of "dan" and "ook".
This phrase is labeled with the POS-tag "adj" by the Alpino parser.
Only when it is the first word of the sentence.
Table 12 Conjunctive adverbs for Dutch
The adverb "wel" has a lot of different senses ("well", "indeed", "rather"), only some of which are conjunctive adverbs. The only occurrences of "wel" in the text corpus that were considered a conjunctive adverb connecting two sentences, were positioned at the start of a sentence. And when "wel" was the first word of the sentence, it always connected two sentences. Therefore, it was decided that "wel" should only be considered a cue phrase when it is the first word of the sentence.
5.4.2 Anaphoric referring expressions There are a lot of different types of referring expressions. Anaphoric referring expressions refer to an entity previously mentioned to the reader or hearer [HAA02b]. The most common are pronouns and demonstratives [KS04]. They usually refer to an entity mentioned at most two
Mieke van Langen
sentences ago [JM00]. Pronouns could for example be "he", "she", "it", "they" or possessive pronouns like "his", "her", "its", "their". Demonstratives, or demonstrative pronouns, are "this", "that", "these", and "those". In Dutch, there are also four demonstratives: "dit", "dat", "deze", and "die". Besides, in Dutch there is a possessive demonstrative "diens" [KS04]. These demonstratives can be used as a noun phrase or adjectively. When they are used in the place of a noun phrase combined with a preposition, their form is modified [HOU00]. For example, "met dit" ("with this") becomes "hiermee", and "dit … mee" like in:
# Dit kun je mee zwemmen. (You can swim with this.)
becomes "hier … mee":
Hier kun je mee zwemmen. (You can swim with this.)
(The # in front of the first sentence indicates that this sentence is not grammatical.) In this way, under influence of a preposition, the demonstratives "dit" and "deze" are transformed into "hier", and "dat" and "die" are transformed into "daar". When there aren't any words between "hier" or "daar" and the preposition, they are integrated into one word ("hierdoor", "hierop", daarvan", etc.). In the same way, the pronoun "het" ("it") is transformed into "er" under influence of a preposition.
Next to demonstrative pronouns, there are also demonstrative adverbs [KS04]. These refer to a place, time, or manner. In Dutch, there are two demonstrative adverbs referring to place: "hier" ("here") and "daar" ("there"); two demonstrative adverbs referring to time: "toen" and "dan" ("then"); and one demonstrative adverb referring to manner: "zo" ("in this way"). These adverbs were not categorized as conjunctive adverbs in the previous subsection, because they don't signal a rhetorical relation between two sentences, but refer to something mentioned previously.
Another type of anaphoric referring expression is a definite noun phrase [JM00]. A definite noun phrase consists of a definite determiner ("the") and a noun phrase mentioned previously or paraphrasing something mentioned previously. However, definite noun phrases are also used non-anaphorically to refer to an entity that is contained in the hearer's set of beliefs about the world, or an entity of which the uniqueness is implied by the description itself. A definite noun phrase thus not always refers to an entity introduced in the previous sentence. It is therefore not very suitable as a linguistic marker of a coherence relation between two sentences.
To construct a list of anaphoric referring expressions referring to an entity, place, time, or manner mentioned in the previous sentence, the Dutch corpus that was also used for extracting cue phrases has again been analyzed. Firstly, all pronouns and demonstratives that refer to something introduced or referred to in the previous sentence were marked manually. Besides, also adjectives referring explicitly to something mentioned in the previous sentence, like "andere" ("other") were marked. Then, it was investigated how these expressions could be recognized as referring to the previous sentence, because anaphoric expressions could also refer to an entity in the same sentence, or for example to the entire text (like in "This document is about RSI."). Besides, in Dutch some pronouns and demonstratives are ambiguous. For example, the Dutch word "het" can be a pronoun ("it"), but it could also be a determiner ("the"), in which case it is not a referring expression. Therefore, in the same way as with cue phrases, all occurrences of the marked words in the text were examined manually to identify under which circumstances they could be determined to be anaphoric expressions referring to an entity in the previous sentence.
Based on the analysis, for anaphoric expressions also some general regularities were discovered, analogous to but slightly different from those for cue phrases. It was found that a referring expression probably doesn't refer to an entity in the previous sentence in the following cases:
when it is embraced by brackets; when it is positioned in a clause following a subordinated clause (labeled with the relation
tag "mod" and the POS-tag "cp");
Question answering for general practitioners
when it is positioned in a clause following a semi-colon; when the sentence contains a colon; when it is positioned after a conjunction or pronoun (labeled with the POS-tags "vg" or
"pron") whose direct parent node is also a parent node of the referring expression, unless this conjunction or pronoun is the first word of the sentence;
when it is positioned in the second part of a dependency tree whose top node has only two
daughters, both labeled with the relation tag "dp".
These regularities are also not universal. For example, anaphoric expressions positioned in the second part of a sentence might still refer to something in the previous sentence. However, it is far more likely that it refers to something mentioned in the first part of the sentence. Even so, an anaphoric expression that is the first word of the sentence not necessarily refers to something in the previous sentence. It might also refer to another sentence or another text level.
There are three differences between the regularities for anaphoric expressions, and the ones for cue phrases. Firstly, whereas cue phrases probably do not refer to the previous sentence when the relation of one of its parent nodes in the dependency tree is modifier, this restriction doesn't hold for anaphoric expressions. Modifiers have some relation to the phrase they modify: the head. Therefore, when a cue phrase is part of a modifier, it probably expresses the rhetoric relation this modifier has to the head. However, when an anaphoric expression is located within a modifier, it could still refer to something in the previous sentence, especially when the sentence starts with this modifier, like in:
In dat stadium is het noodzakelijk ook arbeidsgebonden psychosociale en persoonsgebonden aspecten in beschouwing te nemen. (In that stage it is necessaru to also take into account psychosocial and personal aspects.)
In this sentence the phrase "in dat stadium" ("in that stage") is a modifier, but the anaphoric expression "dat" ("that") definitely refers to something mentioned in a previous sentence.
The second difference has to do with subordinated clauses. When a sentence starts with a subordinated clause, like:
Hoewel daarbij vrijwel continu bewogen en dus afgewisseld wordt, leidt typen er toch toe dat steeds dezelfde spiergroepen gespannen zijn. (Although this involves almost continuous movement and thus variation, typing still causes the same muscle groups to be constantly tense.)
cue phrases or anaphoric expressions located after the subordinated clause, like "toch" ("still"), usually refer to (something within) the preceding subordinated clause. Cue phrases located within this clause itself, might refer to the previous sentence, but they might also express the relation with the rest of the sentence. Therefore, cue phrases located within a sentence starting with a subordinated clause are never recognized as connecting two sentences. However, referring expressions within a subordinated clause, like "daarbij" ("with this") in the example sentence, were more often used anaphorically (referring backwards) than cataphorically (referring forwards) in the examined text corpus. Therefore, anaphoric expressions are only discarded when they are located after a subordinated clause.
Thirdly, when a sentence contains a colon, cue phrases are only discarded when they are located after the colon, because in that case they probably refer to the part preceding the colon. Referring expressions preceding a colon however, are frequently used cataphorically instead of anaphorically, like "andere" ("other") in:
Pauzesoftware mag niet de aandacht afleiden van risicofactoren van andere aard: werkplek en werkorganisatie. (Break programs should not distract from other types of risk factors: workspace and work organization.)
Therefore, when a sentence contains a colon, all referring expressions are discarded.
Mieke van Langen
Referring
Constraints
expression
Not in "onder andere" or "geen andere", or in "ene-andere" or "sommige-andere" constructions. Not after a "van-naar" construction.
Not in "if-then" constructions, recognizable by the word "als" ("if") occurring previously in the sentence or by a parent node with the relation tag "nucl" and the POS-tag "smain".
When it is the first word of the sentence.
Not in indications of the current period like "deze week", "deze maand", "deze eeuw".
Not in indications of the current period like "dit weekend", "dit jaar".
Only when it is followed by a particle (labeled "part") or preposition (labeled "prep") that has the same direct parent.
When it is labeled with the relation tag "mod", and directly followed by an adjective labeled "hd". Not in "as-as" constructions, recognizable by the word "als" ("as") occurring later in the sentence.
When it is the first word of the sentence.
laatstgenoemde adj
Only when it is the first word of the sentence. Not in "as-as" or "as-as possible" constructions, recognizable by the words "als" ("as") or "mogelijk" ("as possible") occurring later in the sentence.
Table 13 Anaphoric referring expressions for Dutch
Question answering for general practitioners
For some of the anaphoric expressions additional constraints were specified, because these expressions could also be used non-anaphorically. For example, the demonstrative adverb "zo" ("in this way") is also used in comparisons in the sense of "as", like in:
Eet daarom zo min mogelijk. (Therefore, eat as few as possible.)
along with their constraints and the POS-tags they should have to be considered an anaphoric expression.
One of the anaphoric expressions found in the corpus is not mentioned in this list: "het". As explained above, "het" could be a determiner ("the") or a pronoun ("it"). Only when it is a pronoun, it could be a referring expression. However, in that case it is still very difficult to determine whether it really is a referring expression. For example, "het" is an anaphoric expression, probably referring to some disease mentioned in the previous sentence, in:
In sommige families komt het meer voor dan in andere. (In some families it is more prevalent than in others.)
but it is not a referring expression in:
In sommige gebieden regent het vaker dan in andere. (In some areas it is raining more often than in others.)
Both sentences have very similar dependency trees. More complex linguistic knowledge would thus be needed to detect the difference between both occurrences of "het". Paice and Husk [PH87] investigated the different uses of "it" in English text. They identified seven different types of what they call structural "it" (as opposed to referential "it"). These seven types could also be identified for the Dutch word "het" (next to the use of "het" as a determiner). For each type different rules would be needed to recognize it. Most of them could be reliably detected using limited word lists and dependency trees, but for some of them, like expressions of time and ambience like "it is twelve o'clock" or "it is raining", more complex lexical or semantic knowledge would be needed. Because of the complexity of differentiating between structural and referential uses of "het", it was considered better to leave "het" out of the list of anaphoric expressions for this research.
Pronouns other than "het" ("it") were not found, except for one occurrence of "ze" ("they"), but this was also too difficult to distinguish from other occurrences not referring to an entity in the previous sentence. Probably this corpus doesn't contain much pronouns like "hij" ("he") or "zij" ("she") referring to an entity in the previous sentence, because the texts all deal with medical information. Other text categories, like for example newspaper articles, are expected to contain much more pronouns, because they more often deal with people instead of diseases.
All demonstrative pronouns were found in the corpus. Besides, a lot of occurrences of "hier" and "daar" in combination with a proposition were retrieved. They all satisfied the same constraints. Therefore, the list could be extended with all possible combinations of "hier" and "daar" with propositions. in the Dutch dictionary [STE94] are presented. These referring expressions could have the POS-tags "pp" or "adv".
Again, the corpus used to extract all these referring expressions is relatively small. Probably, in a larger corpus more expressions and more precise constraints could be detected, but the lists presented arting point for the algorithm used in this research.
Mieke van Langen
Table 14 Prepositional anaphoric expressions for Dutch
5.4.3 Document
structure
Next to cue phrases and anaphoric expressions, an answer may contain other signs of relatedness to another sentence or graphical element, like punctuation marks and captions. These signs deal with document structure. According to Power et al. [PSB03] document structure describes the organization of a document into graphical constituents like sections, paragraphs, sentences, bulleted lists, tables, and figures. Besides, document structure covers some features within sentences like quotation. Some of these graphical constituents are made explicit in mark-up languages, such as HTML.
Chapters, sections, paragraphs, sentences, clauses, and phrases are all levels of document structure [PSB03] (in descending order of abstractness). A response should contain at least one sentence, and at most an entire paragraph (for concerns of conciseness). A sentence starts with a capital letter and ends in a full stop (a dot, question mark, or exclamation mark). When the answer returned by the question answering module is a sentence, it could suffice as a response. However, if it ends with a question mark, it is probably a question. In that case, the next sentence is expected to be the answer on this question, and should thus also be included in the response. However, if the question is a link in an HTML-document (enclosed by the tags <a> and </a>), it would be better to include the linked document in the response.
When the answer returned by the question answering module is not a sentence, it could for example be a clause, a caption, or a heading. In the case of a clause, ending for example in a semicolon or colon, the next clause should also be included to make the sentence complete. When the answer ends with a colon, it might also be followed by an image, table, or list instead of a second clause. In that case, this image, table, or list should be included in the response. If the answer doesn't finish with a punctuation mark at all, the sentence is probably not finished. This may be because it crosses a page break in the original document. In that case, the rest of the sentence should be retrieved from the original document. When the answer is a caption belonging to a figure or table, the corresponding figure or table should also be retrieved from the original document and included in the response. Captions can be easily recognized, because they start with "Table" or "Figure". When the answer is a heading (recognizable by a heading number, or the heading tags in an HTML-document) at least the first sentence of the headed section should also be included in the response.
Question answering for general practitioners
A paragraph begins on a new line. Because the response should contain at most an entire paragraph, it should not cross a paragraph boundary in the original document. In HTML-documents paragraphs are separated by the tags <br> or <p>. In other documents, paragraph boundaries may be harder to detect. For example, in PDF-documents, all lines start with a new line to preserve the lay-out. New lines thus not always indicate a paragraph boundary.
Lists and quotations are examples of indented structures [PSB03]. Indented structures are of a certain level of document structure and are contained by an element possibly being of another level. For example, the elements of a list may be paragraphs, while the list itself is contained by a sentence. When the answer returned by the question answering module is part or contains part of an indented structure, the entire structure should be included in the response. The sentence containing or preceding the indented structure should also be included in the response, because it indicates the context of the structure. Vertical lists can be recognized by bullets (in the case of a bulleted list) or numbers (in the case of an enumerated list). In an HTML-document lists are enclosed by the tags <ul>, <ol>, or <dl>, and each element is preceded by the tag <li>. Quotations or other types of comments can be recognized by the enclosing single or double quotation marks, or brackets.
Horizontal lists or enumerations are not indented structures, because they are described in plain text. In this case the elements of the list are simply phrases within a clause, sentences within a paragraph, or paragraphs within a section, etc. When the list is not contained within one sentence, it might be recognized by cue phrases like "firstly … secondly … finally". However, horizontal lists may also be indicated with more subtle cue phrases or be implicit in the text. They will therefore be ignored in this research.
Finally, answers can be captions belonging to a table or figure, but they can also refer to a table or figure. Normally this is done by explicitly mentioning the word "table" or "figure" followed by a number. In that case the referred table or figure should also be included in the response.
5.4.4 Answer extension algorithm In the previous sections three types of linguistic markers that signal a strong relation between two sentences have been identified: cue phrases, anaphoric expressions, and document structure. In this section an algorithm is described that can be used to extend an answer sentence based on these linguistic markers. The input for the algorithm is a QA document generated by the question answering module "qadr.qa". This document contains up to five different answers that have already been annotated with their dependency structures. First of all these answers are grouped per source. Then each group of answers originating from the same source is extended.
To generate a coherent response by extending a group of answers, the answer sentences are firstly ordered according to the order of the original document. Then the first answer is extended. When this answer contains any cue phrases or anaphoric expressions that satisfy the relevant constraints, the previous sentence is included in the response. When this previous sentence also contains a cue phrase or anaphoric expression, its predecessor is also included. This procedure is repeated until the latest added sentence doesn't contain any cue phrases or anaphoric expressions, or when a paragraph boundary is reached. Then, the first sentence following the answer could be considered. If it contains any cue phrases or anaphoric expressions, it could also be included in the response. However, to prevent the response from getting too long, this procedure is only repeated if the answer hasn't already been extended with three or more sentences. Thus, when a sentence included in the response contains any cue phrases or anaphoric expressions, its previous sentence is always included, because otherwise the response could not fully be interpreted. But sentences following the response containing a cue phrase and/or anaphoric expression are only included if the response would not grow too large.
Mieke van Langen
When there are other answers retrieved from the same document, it is firstly checked whether they have already been included in the response. If they haven't, the same procedure is used to extend these answers.
Finally, the complete response is checked for any linguistic markers of document structure. For example, when the last sentence ends with a colon or semicolon, the next sentence or other graphical constituent is also included, and if there are any unfinished quotations, sentences are added until they are finished (if they can't be finished, they are completely omitted). Possibly also figures or tables are included in the response.
For example, on the third question in Appendix D:
Welke spieren zijn betrokken bij RSI? (Which muscles are affected by RSI?)
four different answers were retrieved from the same document (in the order of occurrence in the original document):
1. Gezien de neiging van RSI zich uit te breiden, blijkt dat na verloop van tijd vaak veel spieren
betrokken zijn bij het proces. (In view of the tendency of RSI to spread, after some time often a lot of muscles seem to be affected by the process.)
2. Toch zijn er wel een aantal spieren die opvallend vaak zijn aangedaan bij RSI. (However,
there are a number of muscles that are affected by RSI notably often.)
3. Uit onderzoek is bekend dat deze het vaakst betrokken is bij RSI. (It is known from research
that this one is affected by RSI most often.)
4. Ademhalingsspieren Andere spieren die betrokken kunnen zijn bij RSI zijn de scaleni.
(Respiratory muscles Other muscles that could be affected by RSI are the scaleni.)
The first answer doesn't contain any linguistic markers of a relation with the previous sentence. However, the sentence following the answer
Het aantal spieren dat bij verschillende RSI patiënten mee kan doen, is dan ook groot.
contains a cue phrase ("dan ook"). The subsequent sentence
Een beschrijving daarvan zou haast neerkomen op het dupliceren van een anatomische atlas.
contains the anaphoric expression "daarvan", and the following sentence
Toch zijn er wel een aantal spieren die opvallend vaak zijn aangedaan bij RSI.
contains the cue phrase "toch". Actually, this last sentence is the second answer. The second answer is thus automatically included in the response by extending the first answer.
When all answers have been extended, the following response results (answers are bold, and linguistic markers have been marked):
Gezien de neiging van RSI zich uit te breiden, blijkt dat na verloop van tijd
vaak veel spieren betrokken zijn bij het proces. Het aantal spieren dat bij
verschillende RSI patiënten mee kan doen, is dan ook groot. Een beschrijving
daarvan zou haast neerkomen op het dupliceren van een anatomische atlas. Toch
zijn er wel een aantal spieren die opvallend vaak zijn aangedaan bij RSI. De
meest beruchte spier is de monnikskapspier (trapezius). Uit onderzoek is bekend
dat deze het vaakst betrokken is bij RSI. Dat is ook niet zo verwonderlijk, want
deze spier zorgt voor het optillen en stabiliseren van de schouders. Zodra de
armen worden opgetild, zoals bij typen en telefoneren, neemt de spanning in deze
spier fors toe.
Question answering for general practitioners
Ademhalingsspieren Andere spieren die betrokken kunnen zijn bij RSI zijn de
scaleni. Dit is een groepje spieren die vast zit aan de halswervelkolom en aan de
bovenste ribben. Bij diep ademhalen spannen deze spieren aan, maar bij normale
activiteiten nauwelijks. Vandaar dat deze spieren ook wel hulpademhalingsspieren
genoemde worden, ze heffen de ribben bij speciale omstandigheden zoals niezen,
zuchten en hoesten.
This response consists of two paragraphs, separated in the original document by several pages. The response is very relevant for the question, but it is quite long. Actually, the original document contains three pages of information relevant for the question. This may indicate that a concise response is not possible on this question. On the other hand, GIPS is required to return only concise responses. In the case of the example presented above, the response could be restricted to the first paragraph, because that contains three out of four answers. It is not clear, however, how often a response would grow this large and how relevant they would be. A decision on how to prevent responses from getting too long has therefore been deferred to the evaluation stage (described in the next chapter).
5.5 Implementation
The response formulation algorithms described in this chapter are incorporated by the output
generation module of GIPS. A prototype of this module has been implemented in Java version
1.5.0. This prototype consists of a class "GipsGen" which provides a method "generate". This
method takes a QA document generated by the question answering module "qadr.qa" as input
(other types of input cannot be processed by this prototype). It reads the question sentence, the
answer sentences, their context sentences, and the annotations associated with these
sentences. Then it generates a P-ml document according to the answer extension algorithm
described above. It uses some subclasses to accomplish this. For example, a class "Node" has
been implemented to be able to generate dependency trees. Besides, a class "GipsTest" has
been implemented to provide "GipsGen" with example input and collect the P-ml files it
generates.
The answer integration algorithm described inthis would take too much time. Besides, although the sentence fusion algorithm developed at the University of Tilburg [MK05] aims at integrating answers retrieved from different documents, it also seems a good solution for integrating autonomous answers retrieved from the same document. The sentence fusion algorithm would be more flexible than the answer integration the sentence fusion algorithm is not implemented for question answering yet. Autonomous answers are therefore simply treated as extended answers (extended with only zero sentences) by this prototype.
A second prototype has been constructed that generates a baseline response. The class "BaselineGen" used for this purpose is very similar to "GipsGen". It groups the answers per source and puts them in the original order, just as is done with the answer extension algorithm. However, "BaselineGen" doesn't use the lists of cue phrases and anaphoric expressions. Nor does it generate any dependency trees. It simply includes the preceding and the successive sentence for each answer sentence.
5.5.1 Limitations The GIPS prototype has some limitations compared to the answer extension algorithm. First of all, because it doesn't have access to the IMIX document collection and the Alpino parser, answers could only be extended with sentences provided as context sentences in the QA document. Ideally, this context consists of the entire paragraph the answer was retrieved from. However, this is not always the case. Especially sentences retrieved from pdf-documents are frequently poor. For example, they are not finished because they are interrupted by a page break, or they contain a page header, like the document title or page number. Besides, it was also not possible to retrieve referred pictures or tables.
Mieke van Langen
Secondly, in the sentences returned by "qadr.qa", all words and punctuation marks are separated by white spaces. Therefore, it was very difficult to determine whether a quotation mark signaled the start or the end of a quotation, and thus to determine whether extra sentences should be included to finish the quotation or not. Fortunately, quotations don't occur frequently in the text category used for this research. Bracketed parts did not have this problem, because the opening bracket "(" and finishing bracket ")" could easily be distinguished.
Finally, vertical lists are also not recognized by the GIPS prototype. Therefore, they cannot be finished when part of it is included in the response.
All these limitations could be solved when the system has access to the original documents (and preferably also the HTML-tags) of the answers, and to the Alpino parser. This could have been accomplished during this research, but that would have taken very much extra effort, while the extra value for this research would have been minimal.
5.5.2 Code The Java code for the prototype and the baseline, including the example input and output files used for evaluation of the system, can be retrieved from my website:
Question answering for general practitioners
6 Evaluation
Based on literature research and interviews with general practitioners, an information
presentation module for a medical QA system (GIPS) has been designed. In this chapter the
design for GIPS in general and the response formulation algorithm developed for GIPS in
particular are evaluated. The evaluation of the entire design for GIPS is described in section
The evaluation of the answer extension algorithm used for the response formulation
Evaluation of the entire design
To evaluate the entire design for GIPS, a prototype of the information portal for general practitioners (including a prototype of the graphical user interface of GIPS) has been ypes have not been constructed to test the functionality of the corresponding systems, but to make the design tangible for users. The method used to evaluate this design is described inthe results and conclusions of this evaluation are presented.
6.1.1 Evaluation
As stated previously, this research did not aim to develop a system that would be equally appreciated by all Dutch general practitioners. It was expected that general practitioners who already use the Internet to search for answers on their medical questions are more likely to appreciate and use a QA system than other general practitioners. Therefore, to evaluate this research, the design for GIPS and the information portal resulting from this research have been evaluated only with the two general practitioners who indicated in the previous interviews that they already used the Internet to search for answers on their medical questions. Again, qualitative interviews have been used.
The interviews had the same semi-structure as the previous ones. They covered the topics of information needs, information sources and computer use. A general outline of the interviews is shown in Appendix E. First of all, the general practitioners were confronted with the prototypes of the information portal and GIPS. Then they were asked how often they think they would use such systems and whether they would use them during consultations and patient visits. Secondly, it was explained to the general practitioners what kind of resources GIPS would use and they were asked what they think of these resources. Thirdly, the general practitioners were asked a few questions about whether they think the information portal and GIPS could be an improvement for their work and what they think of the functionality of the systems.
6.1.2 Evaluation
In this paragraph, the results of the interviews are discussed with respect to the general practitioners' information needs, information sources, and computer use.
Information needs The general practitioners both thought they would use a system like GIPS to pursue their information needs if it were entirely functional and working properly. One of them thought she would use it multiple times a day, the other thought he would use it a few times per week. Example questions they would have liked to submit to the system are "How often do RSI and the chronic fatigue syndrome coincide?" or "What kind of exercises could be done when … ?". The general practitioners would both use the system during the consultation in order to be able to print its responses and give them to the patient, just as they do now with NHG patient letters that are provided by the general practitioner information system they use. Patient letters are not available for all topics, however. Therefore they would like any additional information GIPS could provide.
On the use of the information portal and mobile computing, the general practitioners were less unanimous. The information portal provides access to three existing systems (the NHG
Mieke van Langen
guidelines, Artsennet.nl, and PubMed) and three future systems (GIPS, image retrieval, and a social map). One of the general practitioners was primarily enthusiastic about the ease with which the three existing systems could be accessed via the information portal. The other one thought he would rather access these existing systems directly, just as he is doing now. He really liked the idea of the three future systems, however.
Mobile computing is still science fiction for both general practitioners. One of them would really like it. She is looking forward to having access to patient data and information systems like GIPS during patient visits. The other general practitioner hasn't got any problems with the way he is working now and wouldn't consider purchasing a laptop just to have access to his data during patient visits.
Information sources Both general practitioners indicated that it is very hard to determine the reliability of electronic sources. One of them said he once read about an anesthesiologist (whose name he can't remember) who made an overview of reliable medical websites. He thinks such an overview (constructed by a medical professional) could be used as document collection for GIPS. The other general practitioner said she would trust the information provided by the NHG, but from patient organizations she would only use practical information. She thinks these sources do not provide reliable theoretical information.
Computer use One of the general practitioners really thought the information portal would save her time compared with the way she now accessed the search engines Artsennet.nl and PubMed. She even asked whether the prototype of the information portal would remain online.
The general practitioners both appreciated the way the information is presented by GIPS and the length of the responses. They especially liked the printing function. However, one of the general practitioners indicated his computer wasn't connected to a printer yet. He would like to have a printer on his desktop to be able to print the information during the consultation without having to leave the room. This will be realized in the near future.
Finally, both general practitioners thought GIPS and the other future systems (image retrieval and the social map) could be an improvement for their work. Besides, one of the general practitioners said he appreciated GIPS, because he could enter an entire question instead of only a few keywords, like in traditional search engines. However, he indicated he found it hard to identify any negative aspects now, but when all possible questions could be entered in GIPS, he would firstly like to experiment with it for a while to determine the quality of the responses.
6.1.3 Conclusions A question answering system that answers questions for patient education would really be suitable for use by general practitioners, especially for the ones who already use Internet to look up information on patient care. The design that has been made during this research (GIPS) seems to be appreciated by general practitioners. However, special attention should be paid to selecting the right information sources. Besides, the automation level of general practices is currently not always sufficient for optimal use of such a question answering system. For example, general practitioners should have access both to the Internet and to a printer in their consulting rooms.
Next to question answering technology, image retrieval technology for retrieving dermatological images, and information extraction technology for retrieving address information of medical professionals and organizations (a social map) would also be very useful for general practitioners. The information portal designed during this research seems a good way of providing general practitioners with an overview of the kinds of information they can find on the Internet.
Question answering for general practitioners
Evaluation of the answer extension algorithm
An answer extension algorithm was developed for GIPS that extends an answer sentence (returned by a question answering module) with the sentences most related to it. In this section it is investigated whether these sentences add any relevant information and whether including them makes the response more coherent. The prototype used for this evaluation was described used to evaluate the algorithm is deresults and conclusions of this evaluation are presented.
6.2.1 Evaluation
To evaluate the answer extension algorithm, responses have been generated for a set of test questions. This test set includes the ten example RSI questions presented in Appendix D and ten non RSI-related medical questions randomly selected from an IMIX document with example questions [IMIXv]. The paragraphs from which the answers on these questions were retrieved were not part of the text corpus used for the development of the answer extension algorithm. On manual inspection of the responses on the test questions, the algorithm seems to have correctly recognized the relevant cue phrases and anaphoric expressions.
The responses generated with the prototype were compared with the corresponding baseline responses (consisting of the answer sentence extended with the preceding and successive sentence). Then the paragraphs were selected that satisfied the following criteria:
the paragraph generated by the answer extension algorithm differs from the baseline; the paragraph at least covers the same topic as the corresponding question (to prevent the
evaluation from being influenced by the correctness of the answers generated by the question answering module "qadr.qa");
the paragraph doesn't refer to any figures, tables, or lists; if multiple paragraphs from a single response satisfy these criteria, only the first one is
In total, the twenty responses contained 53 paragraphs. Of these paragraphs 14 were identical to the baseline. This does not necessarily mean that GIPS extended all these 14 answers with the preceding and successive sentence, because sometimes the QA document doesn't provide any context sentences or for example provides only successive context sentences, in which cases GIPS as well as the baseline were not able to extend the answers properly. 16 other paragraphs were considered not dealing with the same topic as the question, and 5 others contained expressions referring to a figure, table, or list. The remaining 18 paragraphs belonged to 12 different responses. For each of these 12 responses the first suitable paragraph was selected. Any unfinished sentences in these paragraphs were finished manually by consulting the original document instead of the context provided by the QA document.
Naïve users were asked to evaluate these paragraphs. It was not necessary to ask general practitioners to evaluate the paragraphs, because the responses were not evaluated on medical correctness, but on linguistic characteristics. Besides, GIPS aims at answering questions for patient education. Patients should thus be able to understand the responses. Naïve users are all potential patients, they are thus very good subjects for the evaluation of the responses. However, because some linguistic feeling is needed to be able to evaluate the responses on the relevant variables, only higher (non-medically) educated users participated in this evaluation.
Twenty-nine users participated in this evaluation. Though all participants received some higher education, the group was very heterogeneous, consisting of men and women of different ages (from 19 to 50 years old) and from different disciplines. There were no medically educated participants, because the evaluation of the responses should not be influenced by judgments on the medical correctness of the responses.
The participants were randomly divided into two groups. Each group received a different questionnaire. The two questionnaires are shown in Appendix F. The first group had to evaluate the GIPS generated paragraphs for one half of the twelve questions, and the baseline
Mieke van Langen
responses for the other questions. For the second group, it was the other way round. The GIPS generated responses and baseline responses were randomly ordered.
The goal of this evaluation was to investigate whether the sentences included by the answer extension algorithm add any relevant information and whether including them makes the response more coherent. Therefore, the participants were asked to indicate for each response on a five-point scale how useful the response is with respect to the question, how much irrelevant information with respect to the question is contained in the response, and how coherent the response is. The participants were told that the usefulness of the response doesn't relate to the amount of useful information a response provides, but only to the presence of any useful information with respect to the question. The amount of irrelevant information was defined as the proportion of information that is irrelevant with respect to the question. A response can thus both be very useful and contain a lot of irrelevant information with respect to the question. Finally, the coherence was defined as the linguistic coherence of the response. If there are any referring expressions that could not be resolved, like "these things" in:
These things are caused by smoking.
the coherence is said to be low. However, if the response is a fluent text that can be fully interpreted, the coherence is high. Or, as one of the participants stated it: "would I understand the response if I had not read the question?".
This evaluation method is very similar to that used by Bosma for the evaluation of his answer extension algorithm [BOS05]. He used the same baseline and the same type of questionnaire. However, instead of the coherence of the responses, he investigated to what extent the participants were able to verify whether the responses were accurate. This "verifiability" is high when a participant is able to verify that the response is accurate, as well as when he is able to verify that the response is not accurate. It is only low when a response doesn't contain enough context to determine whether the response concerns the subject of the question or another subject. This variable was not used for the evaluation of the responses generated by GIPS, because in a pretest participants thought it very hard to distinguish this variable from the usefulness of the response. Besides, when they understood what was meant with the verifiability, they never evaluated it as low. Therefore, it was decided to prevent any further participants from evaluating this variable. Instead, because the answer extension algorithm developed for GIPS was intended to improve the coherence of a response, participants were asked whether they thought the paragraphs formed a coherent text.
6.2.2 Evaluation
Five-point scales were used to evaluate the responses. These scales ranged from "very low" to "very high". The middle score was "neutral". For each variable (usefulness, irrelevance, and coherence) the proportion of high scores on the GIPS generated responses was compared to the proportion of high scores on the baseline responses. Neutral scores were ignored, because users also used this score when they could not decide whether the score should be high or low. The proportion of high scores is thus defined as the number of high scores divided by the total number of high and low scores on a specific variable. Double-sided two-sample t-tests were used to determine whether the difference between the proportions of high scores on the GIPS generated responses and on the baseline responses was significantly larger than zero, at a significance level of 0.05.
Only three of the twelve answers (the first, fourth, and sixth) were extended by the extension algorithm of GIPS. The other answers were considered autonomous. To investigate whether the not extended answers were really autonomous and whether the extended answers were extended better than the baseline answers, the proportions of high scores were also analyzed separately for the three dependent answers and the nine autonomous answers.
Question answering for general practitioners
Usefulness on usefulness are shown for the GIPS generated responses and the baseline responses. On dependent answers the usefulness of the responses with respect to the question was equally well evaluated, but on autonomous answers the baseline seems to perform somewhat better than GIPS. This difference is not significant (at a significance level of 0.05), however.
There are three individual autonomous answers on which the baseline did score significantly higher than GIPS. In these cases GIPS apparently failed to include useful sentences that were included by the baseline. Relationships between these sentences were not signaled by cue phrases or anaphoric expressions, however. The algorithm thus did work properly. For example, on the question:
Komt RSI in Nederland vaker voor dan in de rest van Europa? (Is the prevalence of RSI in the Netherlands higher than in the rest of Europe?)
the answer sentence was:
De geschiedenis leert dat RSI geen modeverschijnsel is. (History teaches us that RSI is not a trend.)
This sentence is indeed not useful with respect to the question at all. It doesn't have any cue phrases or anaphoric expressions either. However, the preceding sentence is very useful with respect to the question:
Nederland heeft niet meer RSI-klachten dan andere landen in Europa. (The Netherlands don't count more RSI complaints than other countries in Europe.)
The other two questions on which the baseline provided a significantly more useful reponse concerned answer sentences that were more useful than the one illustrated above. However, in these cases the preceding and successive sentences provided some additional information that was evaluated as useful by the participants.
Baseline
Significant
difference
Dependent answers
Autonomous answers
Table 15 Proportion of high scores on usefulness
Irrelevance The scores on the proportion of irrelevant information with respect to the question are shown in sponses was significantly lower than the irrelevance of the baseline responses, at a significance level of 0.05. This was not the case for the extended responses, however.
On the individual extended responses also no significant differences were measured concerning irrelevance. For only one of these responses, on the sixth question:
Wat zijn de verschijnselen van griep? (What are the symptoms of influenza?)
the baseline scored better (i.e. had a lower irrelevance) than GIPS. Possibly this was due to the sentence length of the GIPS generated response:
Mieke van Langen
Griep wordt veroorzaakt door het zogenaamde influenzavirus, met
verschijnselen van koorts, neusverkoudheid, hoesten, hoofdpijn,
spierpijn en vermoeidheid. Omdat het virus erg besmettelijk is kan
iedereen griep krijgen en zal je meestal ruim een week het bed moeten
houden. Gezonde mensen knappen daarna weer op door rust en veel
drinken, maar bij mensen met een chronische aandoening, patiënten met
een verminderde weerstand, bewoners van verpleeg-verzorgingshuizen en
ouderen boven de 65 jaar kan de ziekte ernstig verlopen. Zij worden dan ook
jaarlijks door hun huisarts gevaccineerd tegen griep, de zogenaamde
influenzavaccinatie, die voor 70-80% bescherming biedt tegen het krijgen
van griep (influenza).
The first two sentences of this response (in bold) were both answer sentences, retrieved from the same paragraph. The third and the fourth sentences were added, because they contained an anaphoric expression and a cue phrase respectively. The response thus counts only four sentences, but these sentences have a mean length of 25 words. As a comparison: another extended response counted seven sentences, but received a lower irrelevance score. Its mean sentence length was 15 words. The baseline for the response presented above didn't include the last sentence (counting 25 words). Instead the sentence preceding the first answer sentence was included. This sentence counted only 14 words.
Baseline
Significant
difference
Dependent answers
Autonomous answers
Table 16 Proportion of high scores on irrelevance
Coherence GIPS scores significantly higher on coherence than the baseline, at a significance level of 0.05. Again, this is not true for the extended responses, however.
On the individual extended responses no significant differences were measured concerning coherence either. But, probably for the same reason as hypothesized before, the GIPS generated response on the sixth question (the one presented above) scored lower than the baseline response. Another GIPS generated response also scored lower, but this difference was even less significant. The third extended response scored almost significantly better than the baseline response.
Baseline
Significant
difference
Dependent answers
Autonomous answers
Table 17 Proportion of high scores on coherence
Question answering for general practitioners
6.2.3 Conclusions Concerning the usefulness of the responses with respect to the question no great differences were found between GIPS and the baseline. There was only one question for which GIPS apparently failed to include the most useful sentence where the baseline did. Actually, this failure could also be attributed to the question answering module of the IMIX demonstrator, which should have marked this sentence as the answer sentence instead of the successive one.
Based on the evaluation of the proportion of the response that is irrelevant to the question, it might be concluded that GIPS succeeds in filtering a lot of irrelevant information that was included by the baseline. However, answers that are extended by GIPS should not grow too large. The algorithm used by GIPS only restricted the number of sentences included, but maybe it should also take into account the number of words.
Finally, on coherence the GIPS generated responses scored significantly higher than the baseline responses. It is suggested that restricting the number of words of extended responses might increase the coherence even more.
Especially answers that were not extended by GIPS (autonomous answers) contained significantly less irrelevant information and were judged significantly more coherent than the baseline responses. It might thus be concluded that these answers should indeed not be extended. For dependent answers (which were extended) no significant differences with the baseline responses were found. This is probably due to the small number of dependent answers.
Bosma [BOS05] developed a similar answer extension algorithm and executed a similar evaluation. He also concluded that the differences on usefulness between the responses generated by his algorithm (the query-based summarizations) and the baseline responses were not significant, and that the query-based summarizations contained less irrelevant information than the baseline responses. He did not evaluate the responses on coherence. For his algorithm an automatic RST-annotation tool for Dutch would be needed, which is not yet available. The algorithm developed for GIPS only needs a tool for generating dependency trees. For this task a Dutch tool is available: the Alpino parser. Therefore, the answer extension algorithm developed for GIPS currently seems a good alternative for the algorithm developed by Bosma.
Mieke van Langen
Question answering for general practitioners
7 Conclusions
The research question to be answered in this master's thesis was:
Which information needs of Dutch general practitioners can be satisfied by a question
answering (QA) system and how should the answers be presented?
Literature research and interviews with general practitioners were used to identify the different work roles and tasks, information needs, variables of awareness of information, and information sources of general practitioners.
Work roles and tasks It was concluded that a QA system could support the general practitioner primarily in his work role of service provider, during the phase of searching databases, in order to help the general practitioner providing explanations to the patient.
Information needs The information needs most suitable to pursue with a QA system, would be those concerning patient education and population statistics. Most Dutch general practitioners already use a general practitioner information system that incorporates the patient letters issued by the Dutch College of General Practitioners (NHG). These letters can be printed and handed out to the patient. However, not all topics are covered by these patient letters. Therefore, some of the interviewed general practitioners indicated they would like to use a question answering system to search for additional information to give to the patient.
Awareness of information Concerning the awareness of information, a QA system could improve the accessibility of electronic information sources for patient education, because the general practitioner only has to enter a question in Dutch. A dialogue between the system and the general practitioner might be used in order to specify the question when needed. The general practitioners participating in the interviews did not yet use the Internet to search for answers during medical consultations, because that would take too much time. However, when they want to hand out information to the patient, they should look up the information during the medical consultation. Therefore, the QA system's response time should be short enough to enable using the system during medical consultations. Besides, general practitioners must know which information they can find with the QA system. To provide an overview of the information that can be found with the QA system and the information types that can be retrieved with other information retrieval systems useful for general practitioners, an information portal has been designed. This portal was appreciated by the general practitioners participating in the evaluation.
Information sources The information sources used by the QA system should be suitable for patient education. Because the information must be up-to-date, these sources could best be retrieved from the Internet. However, it is hard to determine the reliability of sources on the Internet. Therefore, only information sources marked as reliable by medical professionals should be used.
Computer use The computer use of general practitioners was also investigated. Most general practitioners have a computer in their consulting rooms and make use of a general practitioner information system. The use of mobile devices in the general practice is far less common, however. Although it is expected that these devices would really improve the physician's work during patient visits, none of the interviewed general practitioners was planning to have one. Some of them even rather omit visiting patients for this reason.
It was concluded that the general practitioners most likely to appreciate a QA system are those who already search for answers on their medical questions on the Internet. Concerning the user interface, the system should accommodate both keyboard and mouse input. Speech input should only be possible when it is working perfectly. The system should be able to recognize
Mieke van Langen
ICPC coding and other medical slang in the question. There shouldn't be any timeouts and all functional options must be immediately visible on the graphical user interface. Further, general practitioners must have access to a computer with Internet connection and a printer in their consulting rooms to be able to use the QA system optimally. A lot of general practitioners already have these facilities, because they also need them for searching the Web and printing patient letters. It is expected that most other general practitioners will acquire these facilities in the near future. So when the QA system would become available on the Internet, general practitioners wouldn't have to make any extra costs to be able to use it. There only needs to be an organization that informs the general practitioners about the QA system, and selects the right information sources and keeps them up-to-date.
Information presentation The presentation of the answers a QA system returns should include a few important aspects for general practitioners. The answer selection component of a QA system (which provides the input for the response formulation component) returns a number of answers, which could all be correct. It was concluded that answers retrieved from different sources should not be integrated into a single answer, because general practitioners want to be able to check (the reliability of) the source of the answer and because they want to have the feeling that as medical professionals they are in control of the decision process, not the system. However, answers from different sources should be integrated into a single view, to enable general practitioners to select the most suitable sources or answers easily, and print this selection of answers. Therefore, each answer should be presented together with a link to its source, and a checkbox to indicate whether it should be printed or not.
Answers originating from the same source could be integrated into one concise answer, possibly extended with sentences from their context. An algorithm has been developed that determines whether an answer should be extended and with which sentences it should be extended. For this purpose, lists of cue phrases and anaphoric referring expressions were produced and rules were extracted that determine whether an occurrence of a cue phrase or anaphoric expression in a text signals a relation with the preceding sentence (in which case this preceding sentence should be included in the response).
The answer extension algorithm has been evaluated and the evaluation results have been compared to those of Bosma [BOS05] who developed a similar algorithm. It was concluded that the answer extension algorithm produced coherent responses that contain less irrelevant information than a baseline response consisting of the answer sentence extended with the preceding and successive sentence. These results were similar to those of Bosma. However, for his algorithm an automatic RST-annotation tool for Dutch would be needed, which is not yet available. Therefore, the answer extension algorithm developed during this research currently seems a good alternative for the algorithm developed by Bosma.
Question answering for general practitioners
8 Discussion
This research essentially consists of two parts. The first part concerns the information and
computer use by general practitioners. The second part deals with response formulation.
The research on the information and computer use by general practitioners concentrated on the possibility of using question answering (QA) technology to improve general practitioners' work. It was concluded that QA systems would primarily be suitable to answer questions for patient education. However, during the interviews with general practitioners, other types of information needs that might be pursued with intelligent information retrieval technology also became apparent. For example, when confronted with dermatological diseases, general practitioners frequently have to look up images in a dermatology book. Systems for dermatological image retrieval might help general practitioners by reducing the time to find the relevant picture. Another information type needed by general practitioners is a "social map" that provides an overview of regional health professionals and medical organizations. A lot of these organizations can be found on the Internet. Information extraction technology might be used to enable general practitioners to retrieve contact information of these organizations quickly. It is therefore strongly recommended that the applicability of image retrieval and information extraction technology for general practitioners is investigated in future research.
Another question arising from this research is whether a QA system that answers questions for patient education could also be used by patients themselves instead of general practitioners. I think this depends on the state of the art of the QA technology. When the system always returns responses that make sense, it would be very useful for patients, because the system will use a document collection consisting of reliable sources aimed at patient education. Actually, the IMIX demonstrator (the QA system that was the starting-point of this research) is targeted towards these naïve users. However, currently the system also returns a lot of answers that are not even dealing with the same subject as the question. This information could be very misleading to the patient. General practitioners could serve as an intermediary to filter these answers and give the patient only those answers useful for him.
Further, a design was made for an information portal for general practitioners. A very simple prototype has been implemented to illustrate this design. More work could be dedicated to improve this portal, for example by enabling personalization. When general practitioners are able to add and remove systems themselves, they will possibly be more likely to appreciate and use this portal.
In the second part of this research, dealing with response formulation, two algorithms were developed: an answer integration algorithm, and an answer extension algorithm. The answer integration algorithm has not been implemented because of time constraints and because it was expected that a Dutch sentence fusion algorithm which is already being investigated by Marsi and Krahmer [MK05] would achieve better results. Their algorithm has not yet been implemented and evaluated for application in a QA system, however. Therefore, the comparison of (the complexity and results of) these algorithms is left for future research.
The answer extension algorithm has been implemented and evaluated. Lists of Dutch cue phrases and anaphoric expressions were constructed for this purpose, and rules were extracted that determine whether an occurrence of a cue phrase or anaphoric expression in a text signals a relation with the preceding sentence. These lists and rules were based on a rather small text corpus covering only the medical domain. Therefore, they are probably not complete. Future research is needed to construct completer lists, especially when the algorithm would be used for other domains. However, although the lists were not complete, the evaluation results of the answer extension algorithm are promising.
The answer extension algorithm restricts the number of sentences the answer sentence could be extended with. Sentences occurring after the answer sentence are only added when the total number of added sentences doesn't exceed three. However, based on the evaluation results, it
Mieke van Langen
was hypothesized that this number should depend on the sentence length. For example, a second test could be added that checks whether the number of words added doesn't exceed twenty before adding an extra sentence. Extra research would be needed to investigate what numbers of words and sentences the response should maximally contain to optimize both the relevance and coherence of the response.
Further, it was concluded that answers that were not extended by the answer extension algorithm, were more coherent and contained significantly less irrelevant information with respect to the question than a baseline response consisting of the answer sentence extended with the preceding and successive sentence, while no significant differences with respect to the usefulness of the responses were found. It thus seems that the answer extension algorithm is very useful to determine whether an answer should be extended or not. However, no significant differences were found between the extended responses and the baseline responses. This is probably due to the small number of extended responses. A more thorough evaluation would be needed to investigate the usefulness, irrelevance, and coherence of extended responses. For this purpose, it would be better to integrate the answer extension algorithm with the IMIX demonstrator to enable automatic generation of example responses, and speed up the process of finding extended answers suitable for evaluation.
It was also concluded that, for the time being, the answer extension algorithm would be a good alternative for the algorithm developed by Bosma [BOS05], because for his algorithm an automatic RST-annotation tool for Dutch would be needed, which is not yet available. However, when such a tool becomes available, it would be useful to compare the performance of both algorithms on the same set of example responses to determine which algorithm performs best on usefulness, irrelevance, and coherence.
Finally, in this research, response formulation was restricted to selecting and integrating sentences from the document an answer was retrieved from. More intelligent technology would be needed to perform some reasoning with the retrieved information, and formulate responses like "Yes, it is", or "No, but …" on verification questions, or "100,000" on quantity questions. However, it is expected that with the current state of the art, general practitioners would not very much appreciate and trust systems interpreting texts for them. This issue was therefore not dealt with in this research.
Question answering for general practitioners
References
[APO] Apotheek.nl.
Geneesmiddelen. http://www.apotheek.nl
[BAR03] Barzilay,
Information fusion for multidocument summarization: Paraphrasing and
generation. PhD Thesis, Columbia University, 2003.
Best evidence [database on cd-rom]. Philadelphia: American College of Physicians, 1996.
British Medical Journal. Clinical Evidence. London: BMJ Publishing Group Limited. http://www.clinicalevidence.com
[BNM01] Bouma, G., Noord, G. van, and Malouf, R. Alpino: Wide-coverage computational
analysis of Dutch. 2001. http://www.let.rug.nl/ vannoord/papers/alpino.pdf
[BOO03] Boonstra,
Interpretative perspectives on the acceptance of an optional
information system: Lessons from the introduction of an electronic prescription system for general practitioners. University of Groningen: Research Institute SOM, 2003. http://www.ub.rug.nl/eldoc/som/a/03A08/03A08.pdf
[BOS05] Bosma, W. Extending answers using discourse structure. Submitted to Crossing
Barriers in Text Summarization Research. Workshop to be held in conjunction with RANLP, 2005.
Question answering for Dutch using dependency relations. September
2003, Groningen, the Netherlands. http://odur.let.rug.nl/ gosse/Imix/ project_description.pdf
QADR output specification. 2004. IMIX Internal Project Page (restricted):
Barrie, A.R. and Ward, A.M. Questioning behaviour in general practice: A pragmatic study. In British Medical Journal 315, 1997. pp. 1512-1515.
[CBD05] Canisius, S., Bosch, A. van den, and Daelemans, W. IMIX Rolaquad: XML output
specification. 2005. IMIX Internal Project Page (restricted): http://imix.uvt.nl/Demonstrator/Integration/rolaquad_qa/Rolaquad-XML-specification.pdf
Cochrane Library. http://www.cochrane.org
Centre for Evidence-Based Medicine. Focusing clinical questions. http://www.cebm.net/focus_quest.asp
Coumou, H. and Meijman, F. Hoe zoekt de huisarts literatuurgegevens bij problemen van patiënten? In Huisarts en Wetenschap 46, 2003. pp. 359-63.
[CNP00] Cardie, C., Ng, V., Pierce, D., and Buckley, C. Examining the role of statistical and
linguistic knowledge sources in a general-knowledge question-answering system. In Proceedings of the 6th Conference on Applied Natural Language Processing. 2000. pp. 180-187.
[COX00] Cox, D. Uitslag enquête LHV en NHG. Huisartsen surfen thuis! In Huisarts en
Wetenschap 43, 2000. pp. 408-409.
[DHP98] Dupuits, F.M.H.M., Hasman, A., and Pop, P. Computer-based assistance in family
medicine. In Computer Methods and Programs in Biomedicine 55, 1998. pp. 39-50.
Dennis, A. and Newman, W. Supporting doctor-patient interaction: Using a surrogate application as a basis for evaluation. In Proceedings of the CHI '96 Conference
Mieke van Langen
Companion on Human Factors in Computing Systems: Common Ground, April 13-18, 1996, Vancouver, BC, Canada. ACM. pp. 223-224.
Detmer, W.M. and Shortliffe, E.H. Using the Internet to improve knowledge diffusion in medicine. In Communications of the ACM 40 (8), 1997. pp. 101-108.
[EOE02] Ely, J.W., Osheroff, J.A., Ebell, M.H., Lee Chambliss, M., Vinson, D.C., Stevermer,
J.J., and Pifer, E.A. Obstacles to answering doctors' questions about patient care with evidence: qualitative study. In British Medical Journal 324, 2002. pp. 710—722.
[EOE99] Ely, J.W., Osheroff, J.A., Ebell, M.H., Bergus, G.R., Levy, B.T., Lee Chambliss, M.,
and Evans, E.R. Analysis of questions asked by family doctors regarding patient care. In British Medical Journal 319, 1999. pp. 358-361.
[GAW94] Gorman, P.N., Ash, J., and Wykoff, L. Can primary care physicians' questions be
answered using the medical journal literature? In Bulletin of the Medical Library Association 82 (2), 1994. pp. 140-146.
Gorman, P.N. and Helfand, M. Information seeking in primary care: how physicians choose which clinical questions to pursue and which to leave unanswered. In Medical Decision Making 15 (2), 1995. pp. 113-119.
http://www.google.nl
[GOR95] Gorman, P.N. Information needs of physicians. In Journal of the American Society
for Information Science 46 (10), 1995. pp. 729-736.
[HAA02a] Haan, S. de. Discourse. In Appel, R., Baker, A., Hengeveld, K., Kuiken, F., and
Muysken, P. (eds.). Taal en taalwetenschap. Oxford: Blackwell Publishers, 2002. pp. 71-86.
[HAA02b] Haan, S. de. Zinsbetekenis. In Appel, R., Baker, A., Hengeveld, K., Kuiken, F., and
Muysken, P. (eds.). Taal en taalwetenschap. Oxford: Blackwell Publishers, 2002. pp. 163-181.
Multiplatform Testbed: A tutorial. Nijmegen, 2003. IMIX Internal Project
Prisma handboek van de Nederlandse taal. First edition. Utrecht: Het
Architecture. IMIX Internal Project Page (restricted):
Module imix.gui. IMIX Internal Project Page (restricted):
P-ml presentation format. IMIX Internal Project Page (restricted):
Stevin vragen. IMIX Internal Project Page (restricted):
Jurafsky, D. and Martin, J.H. Speech and language processing: An introduction to natural language processing, computational linguistics, and speech recognition. Prentice Hall, 2000.
[JMR03] Jijkoun, V., Mishne, G. and Rijke, M. de. Building Infrastructure for Dutch Question
Answering. In: A.P. de Vries (ed.), Proceedings DIR 2003, 2003.
Question answering for general practitioners
Koninklijke Nederlandsche Maatschappij tot bevordering der Geneeskunst. Artsennet. http://www.artsennet.nl
[KPG03] Kosseim, L., Plamondon, L., and Guillemette, L. Answer formulation for question-
answering. In Proceedings of The Sixteenth Conference of the Canadian Society for Computational Studies of Intelligence, Canada, June 2003. pp. 24–34.
Knott, A. and Sanders, T. The classification of coherence relations and their linguistic markers: An exploration of two languages. In Journal of Pragmatics, 30 (2). 1998. pp. 135-175.
Koenen, L. and Smits, R. Handboek Nederlands. First edition. Utrecht: Bijleveld, 2004.
Langen, M.C.G. van. Building a system for answering Dutch person questions. In 2nd Twente Student Conference on IT. Enschede, 2005.
Lapata, M. Probabilistic text structuring: Experiments with sentence ordering. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics. July 2003. pp. 545-552.
Landelijk Informatie Netwerk Huisartsenzorg: Feiten en cijfers over huisartsenzorg in Nederland. http://www.linh.nl
Lin, J., Quan, D., Sinha, V., Bakshi, K., Huynh, D., Katz, B., and Karger, D.R. What makes a good answer? The role of context in question answering. In Proceedings of the Ninth IFIP TC13 International Conference on Human-Computer Interaction. Zurich, Switzerland, 2003.
Lagendijk, P.J.B., Schuring, R.W. and Spil, T.A.M. Het Elektronisch Voorschrijf Systeem: Van kwaal tot medicijn. Enschede: Universiteit Twente, 2001.
Leckie, G.J., Pettigrew, K.E., and Sylvain, C. Modeling the information seeking of professionals: A general model derived from research on engineers, health care professionals, and lawyers. In Library Quarterly 66 (2). 1996. pp. 161-193.
[MCW05] Magrabi, F., Coiera, E.W., Westbrook, J.I., Gosling, A.S. and Vickland, V. General
practitioners' use of online evidence during consultations. In International Journal of Medical Informatics 74 (1), January 2005. pp. 1-12.
Marsi, E. and Krahmer, E. Explorations to sentence fusion. Submitted to ENLG '05. 2005.
Moldovan, D. and Surdeanu, M. On the role of information retrieval and information extraction in question answering systems. In M. T. Pazienza (ed.). SCIE 2002. July 2002, pp. 129-147.
Mann, W.C. and Thompson, S.A. Rhetorical structure theory: A theory of text organization. Technical report RS-87-190. University of Southern California, Information Sciences Institute. 1987.
National Guideline Clearinghouse. http://www.guideline.gov
Nederlands Huisartsen Genootschap. NHG-Formularium. http://nhg.artsennet.nl
Nederlands Huisartsen Genootschap. Patiëntenvoorlichting. http://nhg.artsennet.nl
Nederlands Huisartsen Genootschap. NHG-Standaarden. http://nhg.artsennet.nl
National Health Service. NLH Question-Answering Service. http://www.clinicalanswers.nhs.uk
Mieke van Langen
Nederlandse Organisatie voor Wetenschappelijk Onderzoek. Interactieve Multimodale Informatie Extractie. http://www.nwo.nl/imix
Nederlandse Organisatie voor Wetenschappelijk Onderzoek. IMOGEN: Interactive Multimodal Output Generation. http://www.nwo.nl/nwohome.nsf/pages/NWOP_653H7L
Os, E. den (ed.). Functional specification IMIX demonstrator. 2004. IMIX Internal Project Page (restricted): http://imix.uvt.nl/Demonstrator/Specification/functional_specification_1_0.doc
Paice, C.D. and Husk, G.D. Towards the automatic recognition of anaphoric features in English text: The impersonal pronoun "it". In Computer Speech and Language 2. 1987. pp. 109-132.
Power, R., Scott, D., and Bouayad-Agha, N. Document structure. In Computational Linguistics 29 (4). 2003. pp. 211-260.
[RHF92] Rector, A.L., Horan, B., Fitter, M., Kay, S., Newton, P.D., Nowlan, W.A., Robinson,
D., and Wilson, A. User centered development of a general practice medical workstation: The PEN&PAD experience. In Proceedings of the SIGCHI conference on Human factors in computing systems, June 1992, Monterey, California, United States. ACM. pp. 447-453.
Reape, M. and Mellish, C. Just what is aggregation anyway? In Proceedings of the 7th European Workshop on Natural Language Generation. Toulouse, France. 1999. pp. 20-29.
[SHA98] Shaw, J.C. Clause aggregation using linguistic knowledge. In Proceedings of the 9th
International Workshop on Natural Language Generation. Canada, 1998. pp. 138-147.
Clause aggregation: An approach to generating concise text. PhD thesis,
Columbia University, 2002.
Sterkenburg, P.G.J. van. Van Dale handwoordenboek van hedendaags Nederlands. Second edition. Utrecht: Van Dale Lexicografie, 1994.
QA XML output (shared part). 2005. IMIX Internal Project Page
Text Retrieval Conference. Question answering collections: http://trec.nist.gov/data/qa.html
[VBM95] Verhoeven, A.A.H., Boerma, E.J., and Meyboom-de Jong, B. Use of information
sources by family physicians: a literature survey. In Bulletin of the Medical Library Association 83, 1995. pp. 85-90.
[VER99] Verhoeven,
Information-seeking by general practitioners. PhD Thesis,
Rijksuniversiteit Groningen. Groningen: Van Denderen, 1999.
[VNB99] Verhoeven, A.A.H., Noort, C.P. van, Bosveld, H.E.P., Boerma, E.J., and Meyboom-
de Jong, B. Information use and needs: a survey among Dutch general practitioners. In Verhoeven, A.A.H. Information-seeking by general practitioners. PhD Thesis, Rijksuniversiteit Groningen. Groningen: Van Denderen, 1999.
Vidiam and Paradime. Functional specification IMIX dialogue system: Version 1. 2005. http://wwwhome.cs.utwente.nl/ schooten/vidiam/funcspec2/funcspec-22apr2005.pdf
Verhoeven, A.A.H. and Schuling, J. Op zoek naar bewijs: een vraag- en antwoorddienst voor de huisarts. In Huisarts en Wetenschap 46, 2003. pp. 12-17.
Question answering for general practitioners
[WHB02] Wolters, I., Hoogen, H. van den, and Bakker, D. de. Evaluatie invoering Elektronisch
Voorschrijf Systeem Monitoringfase: de situatie in 2001. Utrecht: NIVEL, 2002.
Westberg, E.E. and Miller, R.A. The basis for using the Internet to support the information needs of primary care. In Journal of the American Medical Informatics Association 6 (1), 1999. pp. 6-25.
Mieke van Langen
Question answering for general practitioners
Appendix A: Questions
This Appendix contains a sample of typical questions asked by general practitioners. These
questions were collected in Oregon studies of information needs [GOR95, GAW94].
1. In a patient with refractory headaches, now benefiting from a calcium channel blocker, is
there a specific drug or dose that has been shown to work? Is there a study showing this?
2. After 2 courses of antibiotics in a physician's daughter with bronchitis, what treatment is
appropriate for persistent symptoms?
3. In an octogenarian with anemia, angina, and a history of transient ischemic attacks, with a
normal creatinine, iron, and mean corpuscular volume, who refuses a bone marrow exam, what diagnostic and therapeutic options are there?
4. Is it safe to use ibuprofen in a 50-year-old man with a history of colon cancer, now reporting
dysuria, who has cellular casts in his urine?
5. Does Norpace cause fatigue? 6. What are the cost, risk, and usefulness of dipyridamole thallium scanning in a patient with
chronic obstructive lung disease, claudication, and angina pectoris?
7. In a woman with sclerosing adenosis on breast biopsy and family history of breast cancer,
who requires estrogen therapy to control symptoms, how can the risk of breast cancer be lowered?
8. In an 88-year-old woman with dysphagia due to past laryngeal cancer, now in respiratory
failure due to aspiration, what is the physician's role in aggressiveness of care decisions when the patient's family has unrealistic expectations?
9. For a child with exacerbation of steroid dependent asthma and varicella exposure, how do
you give varicella immune globulin and where do you get it?
10. Is meclizine effective for labyrinthitis? 11. In a man with vague intermittent abdominal and back pain, what additional information will
be most useful and what is the complete differential diagnosis?
12. Can aspirin or an antiplatelet agent be used as prophylaxis against pulmonary embolism
(PE) in an elderly woman with unexplained oxygen desaturation and no clinical risk factors for PE (none that warrant transport 100 miles for diagnostic tests)?
13. In a woman with history of delivering at 33 weeks, now having Braxton-Hicks contractions at
32 weeks, on terbutaline and bed rest, in breech position, is c-section indicated if labor cannot be stopped?
14. How can I distinguish and manage chest pain in an older woman with known coronary
disease, status post angioplasty of the left anterior descending coronary artery, arthritis which precludes treadmill testing, esophagitis, inadequate personality which complicates history, given that dipyridamole testing is 180 miles away?
15. In a patient with steroid dependent chronic obstructive lung disease, does the risk of renal
or gastrointestinal complications outweigh the benefit of non-steroidal anti-inflammatory therapy for degenerative joint disease?
16. Can an insulin-dependent diabetic be certified as a commercial driver? 17. At what age is screening prostate-specific antigen [testing] indicated in a low-risk patient? 18. What is the exact increase in risk of thrombotic events on oral contraceptives in a woman
with family history of myocardial infarction (her grandmother at age forty-nine) and of deep-vein thrombosis?
19. Are nonacetylated salicylates really safer (and how much safer) in patients with NSAID GI
intolerance (who benefit from anti-inflammatory effect)?
20. For diagnosis of deep-vein thrombosis, how good is ultrasound; does it obviate the need for
venogram (can it exclude the diagnosis)?
21. Is amoxicillin safe for use in a lactating woman? 22. What is [sic] the sensitivity and specificity of arterial ultrasound exam of the lower
23. Is hypothyroidism associated with high cholesterol or low? 24. What is the dose of Imferon?
Mieke van Langen
25. At what point is endoscopy indicated in patients with esophagitis who remain symptomatic
26. Where can I send this patient for education about his alcoholism: more education than
Alcoholics Anonymous provides, less expense than inpatient treatment?
Question answering for general practitioners
Appendix B: Interview general practitioners
Information needs
The following questions deal with the medical questions you are confronted with during patient care.
1. How frequently are you confronted with questions from patient care?
2. Do you always search for answers on such questions? Include answers which you look up
in the Pharmacotherapeutic Directory (Farmacotherapeutisch Kompas), or which you obtain from consulting a colleague.
3. At what moments do you search for information?
4. Do you also search for information when you are visiting patients?
5. Are the questions you are confronted with during patient visits different from those you are
confronted with in the consulting room?
Information sources
6. Which information sources do you use when you are looking for the answer to a medical
7. Do you use any electronic sources?
8. Do you also use English information sources?
9. What do you think of your possibilities for finding information for patient care?
10. Are any improvements needed for finding information for patient care?
Computer use
11. Do you own any of the following items at work?
Computer CD-ROM player Software to search the medical literature CD or disk with medical knowledge Subscription to Internet
12. Which computer applications do you use at work?
The following questions deal with information retrieval and question answering systems. I will
13. Do you ever use information retrieval systems during clinical practice?
Yes, namely ……………………………….
14. Are you generally able to find what you need with these systems?
15. Which features of these systems do you like?
16. Are there any negative aspects of these systems?
17. Do you think a question answering system could be an improvement for your work?
18. In which information sources would you like a question answering system to search?
Mieke van Langen
19. What kind of answers do you prefer when you search for an answer to a medical question?
Complete articles Only relevant paragraphs A concise answer Other, namely: ……………………………….
20. Would this be the same when you were searching for information during a patient visit?
21. If you had a system that provides you with only paragraphs or concise answers to your
medical questions, what additional information would you like to have?
22. When you use a computer at work, do you generally prefer mouse or keyboard input?
23. Would you like to use speech input?
24. Can you show me a user interface of a medical information system that you really
Figure 16 A common information retrieval user interface [GOO]
Question answering for general practitioners
Figure 17 A medical information retrieval user interface [EPM]
Figure 18 An example of a question answering user interface
Mieke van Langen
Question answering for general practitioners
Appendix C: Screenshots
ype of the web portal for general practitioners.
the prototype of GIPS, which can be accessed through the
web portal.
Figure 19 Screenshot of the information portal for general practitioners
Figure 20 Screenshot of the prototype of GIPS
Mieke van Langen
Question answering for general practitioners
Appendix D: Questions and answers
In this appendix ten example RSI questions and their answers as generated by the "qadr.qa"
question answering module of the IMIX demonstrator are presented (in Dutch). The answers
are grouped per source.
RSI is een verzamelnaam voor zeer uiteenlopende vormen van
overbelasting in het gebied van nek , schouders , armen en
Beroepsziekte : met 2600 mensen per jaar in de WAO is RSI de
meest gesignaleerde beroepsziekte in 2001 .
Twee jaar geleden hebben we een grote beeldschermwerk-dag
georganiseerd , sinds die tijd is RSI gelukkig wel een belangrijk onderwerp in Nederland . "
RSI is geen verkoudheid waar je na een paar weken weer vanaf
bent , zoveel is wel duidelijk .
RSI is een overkoepelende term voor aandoeningen aan nek ,
schouder , arm en hand .
2. Waardoor kan RSI ontstaan?
RSI wordt veroorzaakt door een combinatie van risicofactoren .
Sommige ayurvedische therapeuten gaan ervan uit dat RSI wordt
veroorzaakt door een stoornis in de stofwisseling .
Caissicres , kappers , musici en lopende band-medewerkers : er
is nog een groot aantal andere beroepen die tot RSI leiden .
3. Welke spieren zijn betrokken bij RSI?
Review_RSI_Bulthuis_Elkhuizen :
Gezien de neiging van RSI zich uit te breiden , blijkt dat na
verloop van tijd vaak veel spieren betrokken zijn bij het proces
Ademhalingsspieren Andere spieren die betrokken kunnen zijn bij
RSI zijn de scaleni .
Uit onderzoek is bekend dat deze het vaakst betrokken is bij RSI
Toch zijn er wel een aantal spieren die opvallend vaak zijn
aangedaan bij RSI .
Mieke van Langen
4. Welke beroepen worden getroffen door RSI?
Tabel 4.4 Proportie RSI bij werknemers met 13 weken verzuim :
onderverdeling naar beroepsgroep
Verdeling naar beroepsgroep Aangezien beroepsgroepen en
bedrijfstakken voor een groot gedeelte overeen komen is het niet
verrassend dat er relatief veel verzuimende werknemers met ambachtelijke en industriële beroepen RSI hebben .
39 verzuim door RSI gevonden werden zijn ambachtelijke en
industriële beroepen en dienstverlenende beroepen .
In meer recent onderzoek is , behalve bevestiging van hoge
prevalenties in sommige van de genoemde beroepen , ook een hoge
prevalentie bij echografisten en in visverwerkende bedrijven gevonden ( Ohl94 , Smi97 ) .
Uit een overzicht van buitenlandse onderzoeken naar een relatie
tussen arbeid en diverse klachten en aandoeningen die onder RSI
gerekend worden , komt een aantal beroepen met zeer hoge prevalenties naar voren ( Hag95 ) .
5. Hoe is opkomende RSI te herkennen?
RSI uit zich in spier- , pees- , en zenuwklachten .
6. Welke oefeningen kan ik op mijn werkplek uitvoeren om RSI te
De Interne Arbodienst van de Universiteit Leiden geeft de
volgende aanwijzingen :
Zo blijven mensen achter de pc in vorm en kan RSI mogelijk
worden voorkomen .
Beroepsziekte : met 2600 mensen per jaar in de WAO is RSI de
meest gesignaleerde beroepsziekte in 2001 .
7. Hoe kan ik mijn werkplek het beste inrichten om RSI te voorkomen?
De Interne Arbodienst van de Universiteit Leiden geeft de
volgende aanwijzingen :
Een goed ingerichte werkplek is de eerste stap om RSI klachten
Question answering for general practitioners
8. Helpt pauzesoftware bij de bestrijding van RSI?
Het RSI-Kenniscentrum richt zich op kennis van effectiviteit van
therapeutische interventies , hulpmiddelen en voorlichting rond de preventie en bestrijding van RSI .
Stichting RSI Nederland wil o.a. met deze website een
substantiële bijdrage leveren aan de informatieverstrekking over
RSI , en daarmee helpen bij de preventie en de bestrijding van de muisarm .
9. Kan je door RSI in de WAO komen?
Wel is van het aantal personen tussen 35 en 55 dat in de WAO
terechtkomt een relatief groter percentage door RSI in de WAO
gekomen ( 4,2% ) dan in de jongere ( 2,7% ) en oudere leeftijdsgroepen ( 3,0% ) 3 .
Wel komen van het aantal werkende vrouwen er bijna twee keer
zoveel in de WAO ( door RSI ) als van het aantal werkende man-
Wel is van het aantal personen tussen 35 en 55 dat in de WAO
terechtkomt een relatief iets groter percentage door RSI in de WAO gekomen dan in de jongere en oudere leeftijdsgroepen3 .
Wel komen van het aantal werkende personen bijna twee keer
zoveel vrouwen in de WAO ( door RSI ) dan mannen .
Bedrijfssectoren waarin men een hoog risico loopt door RSI in de
WAO te komen zijn de reinigingsindustrie , de textielindustrie
en de steen- , cement- , glas- , en keramische industrie .
Komt RSI in Nederland vaker voor dan in de rest van Europa?
Nederland is geen koploper wat betreft RSI klachten in Europa . Nederland is in Europa de laatste jaren koploper wat betreft
computergebruik ( Paoli , 1992 ; Paoli , 1997 ; Paoli & Merllié , 2000 ; Andries e.a. , 2002 ) .
Ik wil nu overgaan tot de vragen van alledag die je vaak over
RSI hoort , en ik wil met u onderzoeken welke antwoorden het wetenschappelijk onderzoek al heeft .
De geschiedenis leert dat RSI geen een modeverschijnsel is .
De ziekte komt veel in Europa voor en er zijn ook gevallen
bekend in de voormalige Sovjetunie , China , Japan en Australië .
Mieke van Langen
Question answering for general practitioners
Appendix E: Evaluation interview general practitioners
The general practitioner is confronted with the prototypes of the information portal for general
practitioners and the graphical user interface of GIPS developed during this research. It is
explained that, eventually, GIPS will also be able to hold a dialogue to specify the question,
ICPC coding will be recognized, and keyboard input will be accommodated. Then the general
practitioner is asked to answer the following questions assuming both systems are working
perfectly.
Information needs
1. How frequently do you think you would use this portal or GIPS?
2. Could you remind any questions you met recently which you would have liked to enter in
one of these systems? If yes, what were they?
3. Would you use it during patient care or after the consultation?
4. Could you also imagine yourself using it during patient visits?
Information sources
GIPS will retrieve its answers from online sources for patient education published by the NHG, patient organizations, etc.
5. Do you think these sources are appropriate?
6. Are there any other sources you would like the system to search in?
Computer use
7. Do you think using this portal would save time compared to using the search engines you
8. Do you think you would use the printing option provided by GIPS?
9. Do you like the presentation of the answers?
10. Are there any negative aspects of the portal or GIPS?
11. Do you think these systems could be an improvement for your work?
Mieke van Langen
Question answering for general practitioners
Appendix F: Questionnaire response formulation
The participants to this questionnaire have been divided into two groups that completed
different questionnaires. Each questionnaire contains twelve question-answer pairs, half of
which have been answered by GIPS and the other half are the baseline answers. In the first
questionnaire, the first, second, fourth, seventh, ninth, and tenth answer are GIPS generated. In
the second questionnaire it is the other way round. For each answer the participants are asked
how useful the provided information is with respect to the question, how much irrelevant
information is contained in the answer, and how coherent the answer is.
First questionnaire
High Neut Low Very
1. Welke spieren zijn betrokken bij RSI?
Gezien de neiging van RSI zich uit te breiden, Usefulness
blijkt dat na verloop van tijd vaak veel spieren Irrelevance
betrokken zijn bij het proces. Het aantal spieren dat bij verschillende RSI patiënten mee kan doen, Coherence
is dan ook groot. Een beschrijving daarvan zou haast neerkomen op het dupliceren van een anatomische atlas. Toch zijn er wel een aantal spieren die opvallend vaak zijn aangedaan bij RSI. De meest beruchte spier is de monnikskapspier (trapezius). Uit onderzoek is bekend dat deze het vaakst betrokken is bij RSI. Dat is ook niet zo verwonderlijk, want deze spier zorgt voor het optillen en stabiliseren van de schouders.
2. Hoe voorkom je likdoorns?
Doordat likdoorns meestal ontstaan door slecht Usefulness
passende schoenen, kunnen deze verdwijnen Irrelevance
wanneer beter passend schoeisel wordt gedragen.
3. Komt RSI in Nederland vaker voor dan in de rest
van Europa?
Nederland heeft niet meer RSI-klachten dan Usefulness
andere landen in Europa. De geschiedenis leert Irrelevance
dat RSI geen modeverschijnsel is. In de vleesindustrie en andere beroepen met veel Coherence
repeterende armbewegingen en in kantoorwerk kwamen de klachten al voor voordat de term RSI bestond.
Mieke van Langen
Very High Neut Low Very
4. Wat is RSI?
RSI is een verzamelnaam voor zeer sefulness
uiteenlopende vormen van overbelasting in het Irrelevance
gebied van nek, schouders, armen en ellebogen. Het kan zijn dat de arts aanvullend onderzoek Coherence
laat verrichten naar andere ziektebeelden met soortgelijke symptomen.
5. Kan je door RSI in de WAO komen?
Specifieke risicoberoepen zijn horecapersoneel, Usefulness
conciërges en schoonmakers, gezinshulpen en Irrelevance
bejaardenverzorgers en 'overige ambachtelijke beroepen'. Bedrijfssectoren waarin men een hoog Coherence
risico loopt door RSI in de WAO te komen zijn de reinigingsindustrie, de textielindustrie en de steen-, cement-, glas-, en keramische industrie. Administratieve beroepen vormen in het huidige onderzoek geen risicogroep voor verzuim door RSI; tevens vormen de zakelijke dienstverlening en de overheid geen risicogroepen voor WAO-intrede door RSI.
6. Wat zijn de verschijnselen van griep?
Elke winter wordt 5 tot 20 % van de Nederlands Usefulness
bevolking getroffen door griep. Griep wordt Irrelevance
veroorzaakt door het zogenaamde influenzavirus, met verschijnselen van koorts, neusverkoudheid, Coherence
hoesten, hoofdpijn, spierpijn en vermoeidheid. Omdat het virus erg besmettelijk is kan iedereen griep krijgen en zal je meestal ruim een week het bed moeten houden. Gezonde mensen knappen daarna weer op door rust en veel drinken, maar bij mensen met een chronische aandoening, patiënten met een verminderde weerstand, bewoners van verpleeg-verzorgingshuizen en ouderen boven de 65 jaar kan de ziekte ernstig verlopen.
7. Hoeveel procent van de Nederlandse bevolking
heeft psoriasis?
De kans op psoriasis is gerelateerd aan het Usefulness
aantal familieleden dat deze aandoening heeft.
Question answering for general practitioners
High Neut Low Very
8. Hoe is opkomende RSI te herkennen?
RSI uit zich in spier-, pees-, en zenuwklachten. Irrelevance
Een combinatie van onderstaande mechanismen veroorzaakt de problemen.
9. Waardoor kan RSI ontstaan?
RSI wordt veroorzaakt door een combinatie van Usefulness
10. Wat is het verschil tussen een vlokkentest en een
Een vlokkentest kan in plaats van een Usefulness
vruchtwaterpunctie worden gedaan, tenzij voor Irrelevance
een onderzoek juist vruchtwater nodig is, bijvoorbeeld voor het bepalen van de concentratie Coherence
alfafoetoproteïne in het vruchtwater.
11. Wat is slapeloosheid?
Artsen classificeren slapeloosheid als primair of Usefulness
secundair. Primaire slapeloosheid is een lang Irrelevance
bestaande aandoening die weinig of geen verband lijkt te hebben met enige spanning of Coherence
bijzondere gebeurtenissen in het leven. De secundaire vorm wordt veroorzaakt door pijn, angst, geneesmiddelen, depressie of extreme spanningen.
12. Welke beroepen worden getroffen door RSI?
Beroepsgroepen waarin men een meer dan Usefulness
gemiddeld risico loopt op (kort) verzuim door RSI Irrelevance
zijn beroepen in de transportsector en dienstverlenende beroepen. Beroepsgroepen die Coherence
als risicogroep voor langdurig (meer dan 13 weken durend) verzuim door RSI gevonden werden zijn ambachtelijke en industriële beroepen en dienstverlenende beroepen. Specifieke risicoberoepen zijn horecapersoneel, conciërges en schoonmakers, gezinshulpen en bejaardenverzorgers en 'overige ambachtelijke beroepen'.
Mieke van Langen
Second questionnaire
High Neut Low Very
1. Welke spieren zijn betrokken bij RSI?
De één heeft vooral last in de schouders, bij de Usefulness
ander ontstaan de klachten in de pols of in de Irrelevance
arm. Gezien de neiging van RSI zich uit te breiden, blijkt dat na verloop van tijd vaak veel Coherence
spieren betrokken zijn bij het proces. Het aantal spieren dat bij verschillende RSI patiënten mee kan doen, is dan ook groot. Een beschrijving daarvan zou haast neerkomen op het dupliceren van een anatomische atlas. Toch zijn er wel een aantal spieren die opvallend vaak zijn aangedaan bij RSI. De meest beruchte spier is de monnikskapspier (trapezius). Uit onderzoek is bekend dat deze het vaakst betrokken is bij RSI. Dat is ook niet zo verwonderlijk, want deze spier zorgt voor het optillen en stabiliseren van de schouders.
2. Hoe voorkom je likdoorns?
Eeltplekken kunnen worden voorkomen door de Usefulness
irritatiebron weg te nemen of, als dit niet mogelijk Irrelevance
is, handschoenen, beschermende materialen, bijvoorbeeld ringen te dragen. Doordat likdoorns Coherence
meestal ontstaan door slecht passende schoenen, kunnen deze verdwijnen wanneer beter passend schoeisel wordt gedragen. Een middel dat de hoornlaag losweekt, bijvoorbeeld salicylzuur, kan likdoorns sneller doen verdwijnen.
3. Komt RSI in Nederland vaker voor dan in de rest
van Europa?
De geschiedenis leert dat RSI geen
modeverschijnsel is.
4. Wat is RSI?
Verwacht bij het eerste onderzoek geen Usefulness
definitieve diagnose. RSI is een verzamelnaam Irrelevance
voor zeer uiteenlopende vormen van overbelasting in het gebied van nek, schouders, Coherence
armen en ellebogen. Het kan zijn dat de arts aanvullend onderzoek laat verrichten naar andere ziektebeelden met soortgelijke symptomen.
Question answering for general practitioners
High Neut Low Very
5. Kan je door RSI in de WAO komen?
Bedrijfssectoren waarin men een hoog risico loopt Usefulness
door RSI in de WAO te komen zijn de Irrelevance
reinigingsindustrie, de textielindustrie en de steen-, cement-, glas-, en keramische industrie.
6. Wat zijn de verschijnselen van griep?
Griep wordt veroorzaakt door het zogenaamde Usefulness
influenzavirus, met verschijnselen van koorts, Irrelevance
neusverkoudheid, hoesten, hoofdpijn, spierpijn en vermoeidheid. Omdat het virus erg besmettelijk is Coherence
kan iedereen griep krijgen en zal je meestal ruim een week het bed moeten houden. Gezonde mensen knappen daarna weer op door rust en veel drinken, maar bij mensen met een chronische aandoening, patiënten met een verminderde weerstand, bewoners van verpleeg-verzorgingshuizen en ouderen boven de 65 jaar kan de ziekte ernstig verlopen. Zij worden dan ook jaarlijks door hun huisarts gevaccineerd tegen griep, de zogenaamde influenzavaccinatie, die voor 70-80% bescherming biedt tegen het krijgen van griep (influenza).
7. Hoeveel procent van de Nederlandse bevolking
heeft psoriasis?
De aandoening heeft een erfelijke component. De Usefulness
kans op psoriasis is gerelateerd aan het aantal Irrelevance
familieleden dat deze aandoening heeft. Psoriasis is niet te genezen, maar in de meeste gevallen Coherence
wel goed te behandelen.
8. Hoe is opkomende RSI te herkennen?
RSI uit zich in spier-, pees-, en zenuwklachten.
Mieke van Langen
High Neut Low Very
9. Waardoor kan RSI ontstaan?
Integrale aanpak
RSI wordt veroorzaakt door een combinatie van Irrelevance
risicofactoren. Gezien de grote verscheidenheid aan factoren ligt het eigenlijk voor de hand dat de Coherence
aanpak van de klachten zich dient te richten op alle aspecten die een rol hebben bij het ontstaan van klachten.
10. Wat is het verschil tussen een vlokkentest en een
Met een vlokkentest worden bepaalde afwijkingen Usefulness
van de foetus opgespoord, meestal tussen de Irrelevance
tiende en twaalfde week van de zwangerschap. Een vlokkentest kan in plaats van een Coherence
vruchtwaterpunctie worden gedaan, tenzij voor een onderzoek juist vruchtwater nodig is, bijvoorbeeld voor het bepalen van de concentratie alfafoetoproteïne in het vruchtwater. Vóór de test wordt met behulp van echografie vastgesteld of de foetus leeft, wat de leeftijd van de foetus is en wat de ligging van de placenta is.
11. Wat is slapeloosheid?
Primaire slapeloosheid is een lang bestaande Usefulness
aandoening die weinig of geen verband lijkt te Irrelevance
hebben met enige spanning of bijzondere gebeurtenissen in het leven.
12. Welke beroepen worden getroffen door RSI?
Beroepsgroepen die als risicogroep voor
langdurig (meer dan 13 weken durend) verzuim Irrelevance
door RSI gevonden werden zijn ambachtelijke en industriële beroepen en dienstverlenende Coherence
Source: http://hmi.ewi.utwente.nl/verslagen/afstudeer/LangenMiekevanFinalThesis.pdf
Héctor Morales Morgado Afroariqueños: configuraciones de un proceso histórico de presencia Giselle I. Duconge1 y Menara Lube Guizardi2 El presente artículo constituye un esfuerzo inicial por El objetivo del presente texto es recuperar y narrar los procesos his- comprender la organización comunitaria de los colecti- tóricos de la presencia afrodescendiente en la ciudad de Arica, norte
The Florida Journey Adapting FIMR to Address Maternal Mortality Karen R. Coon, ARNP, MSN Florida Department of Health Division of Community Health Promotion Infant, Maternal & Reproductive Health "You can't really understand another person's experience until you've walked a mile in their shoes." . The Florida Journey: Adapting FIMR to Address Maternal Mortality