Canadian Study team authorized us to use their assessment
instruments. The study was performed in three public teach-
ing hospitals in the State of Rio de Janeiro that provide acute
care and emergency care. Obstetric care is provided by two
of them. Teaching hospitals were selected because of their
willingness to participate and the comparatively higher quality
of their patient records.
The study population consisted of 27 350 patients admitted
in the year 2003. A random sample of patients was selected.
The sample frame excluded patients under 18 years old,
patients who stayed in the hospital less than 24 hr and cases
with psychiatric diagnoses. Unlike the Canadian study, obste-
tric cases were included in the sample due to the persistently
high maternal mortality rate in Brazil (73 maternal deaths per
100 000 live births in 2003). It must be noted that the great
majority of deliveries in Brazil occur in hospitals (97%). The
parameters used to define the sample size were based on
the Canadian study: a 10% expected incidence of patients
with adverse events (maximum absolute error 3%) and a
50% proportion of potential adverse events and, with a
significance level of 5%. The loss rate was estimated at 10%.
The rate of ineligible patients was estimated at 20%. The
final sample size was 1628 patients, with 1103 eligible for
the study.
Assessment of adverse events involved two phases. Phase
1 was an explicit patient record review by nurses to screen
for patients potentially with adverse events based on screen-
ing criteria (trigger tools). At least one screening criterion
determined that the record should be included in the second
phase review. Phase 2 was an implicit structured review by
physicians.
Reviewers were trained using standard patient records
specially selected for this purpose. Reviewers were only auth-
orized to perform fieldwork after reaching at least 80%
agreement with these patient records. All the physicians and
nurses had more than 20 years of professional experience.
The phases 1 and 2 review forms, developed by the
Canadian study, were translated and adapted to the context of
Brazilian hospitals. The forms were translated from English to
Portuguese by two different translators, followed by a compari-
son of the two versions. An expert panel decided by consensus
on the best translation of key terminology. The expert panel
consisted of nine physicians with clinical, surgical, intensive-
care, epidemiological and pharmaceutical backgrounds.
The expert panel also decided on the list of screening cri-
teria to be used, based on the Canadian study list of screen-
ing criteria. The expert panel suggested the exclusion of two,
the addition of one and the modification of five screening
criteria. Excluded criteria were: Criterion 1, ‘Unplanned hos-
pitalization (including readmission) as a result of any health
care provided during the 12 months prior to the index
admission’, and Criterion 2, ‘Unplanned admission to any
hospital during the 12 months following discharge from the
index admission’. These were judged inappropriate in the
Brazilian context, because of lack of systematic documen-
tation in the patient records about details of previous admis-
sions to the same hospital or to other hospitals. The added
criterion refers to increases in creatinine level during hospital
stay. This criterion was included to identify patients who
developed acute renal failure while in hospital. Other modifi-
cations were based on the writing for accuracy of under-
standing. The adapted forms were pre-tested and
back-translated [14]. Software was developed for data
collection.
The exclusion of the two screening criteria suggested by
the expert panel had an impact on both the incidence of
patients with adverse events and the proportion of preventa-
ble adverse events, but these differences were not statistically
significant (P . 0.05). The new criterion of ‘starting from a
normal creatinine in admission, the value increased to twice
the admission value during hospital stay’ was identified
17 times but did not change either the incidence or the pro-
portion of preventable adverse events. For purposes of com-
parison, the results presented in this article adhered strictly
to the Canadian list of screening criteria.
In the phase 2 review, the physicians first identified the
occurrence of unintentional injuries or harm. Then, they ana-
lyzed injuries or harm for any association with temporary or
permanent disability and/or prolonged hospital stay or death.
Finally, using the six-point scale from the Canadian study,
they determined whether the injury or harm was caused by
the care provided to the patient. This scale ranges from (1)
‘virtually no evidence’ to (6) ‘virtually certain evidence’. An
injury or harm is classified as an adverse event when it is
rated as 4 or more. The preventability of the adverse events
is also judged according to a six-point scale, with an adverse
event classified as preventable when rated as 4 or more.
The target patient safety measures included: incidence of
patients with adverse events (number of patients with at least
one adverse event/total number of patients); proportion of
preventable adverse events (number of patients with preven-
table adverse events/total number of patients with adverse
events), number of adverse events per 100 patients and inci-
dence density of adverse events per 100 patient-days
(number of adverse events/sum of hospital days across all
study patients).
Descriptive analysis included patient characteristics: sex
and age (age group: 18–20; 21–30; 31–40; 41–50; 51–60;
61–70 years; and 70 years and older); adverse event
classification—diagnostic error, surgical procedure, orthope-
dic care (fractures), medication, anesthesia, obstetrics, clinical
procedure, system (organization); type of error—omission
and commission; location of the adverse event—inside the
hospital (ward, surgery room, intensive care unit, emergency
room, delivery room, procedure room, service area and out-
patient service) and outside the hospital (home and other
places)—and time of occurrence and detection of the
adverse event.
Reliability in screening criteria between nurse reviewers
was measured at a significance level of 5%, using simple
agreement statistics given the small number of comparisons.
After each 10 cases reviewed, the following case was also
reviewed by a second reviewer, previously assigned as the
first reviewer’s pair for purposes of comparison. Cases for
testing inter-rater agreement were automatically selected by
the software. Simple agreement between nurse reviewers in