Development and Validation of CA
Content
Osteoporos Int (2006) 17: 304–312
DOI 10.1007/s00198-004-1679-1
O R I GI N A L A R T IC L E
Development and validation of a food frequency questionnaire for
assessing dietary calcium intake in the general population
Faidon Magkos Æ Yannis Manios Æ Eirini Babaroutsi
Labros S. Sidossis
Received: 19 January 2004 / Accepted: 17 May 2004 / Published online: 10 September 2004
International Osteoporosis Foundation and National Osteoporosis Foundation 2004
Abstract The aim of this study was to develop and
evaluate a food frequency questionnaire (FFQ) for
assessing dietary calcium intake in the general population, since all available questionnaires at present are ageand/or gender-specific. A total of 1001 individuals
(including children, adults, and elderly people of both
genders) were randomly recruited throughout Greece.
Estimates of calcium intake from the 30-item FFQ were
compared with those from a multi-pass 24-h recall. The
FFQ underestimated mean calcium intake compared to
the 24-h recall by (mean±SD) –133±333 mg/day or –
5.4±47.6% (P<0.001). The two methods were strongly
correlated (r=0.639, P<0.001), but the 95% limits of
agreement for individual assessment were rather wide, as
the FFQ could provide estimates of calcium intake from
533 mg/day above to 799 mg/day below the 24-h recall.
Actual values for surrogate FFQ quartiles manifested a
progressive increase, with significant differences between
mean calcium intakes (P<0.001). The FFQ could
identify individuals who consumed less calcium than
800 mg/day or less than the age-specific adequate intake
with a relatively high sensitivity (82.8 and 95.5%,
respectively), but low specificity (54.9 and 34.1%,
respectively). Cross-classification analysis indicated that
only 17 subjects (1.7%) were grossly misclassified (lowest quartile for one method and highest quartile for the
other), while 827 subjects (82.6%) were correctly classified (into the same or adjacent quartiles). The FFQ
could be used in population-based epidemiological
studies or screening programs involving individuals of
all ages and both genders, where the discrimination of
subjects with relatively low (<500 mg/day) and
relatively high (>1000 mg/day) calcium intakes is of
F. Magkos Æ Y. Manios Æ E. Babaroutsi Æ L.S. Sidossis (&)
Laboratory of Nutrition and Clinical Dietetics,
Department of Nutrition and Dietetics,
Harokopio University, 70 El. Venizelou Avenue,
17671 Athens, Greece
E-mail: [email protected]
Tel.: +30-210-9549154
Fax: +30-210-9549141
primary interest. Results, however, do not support its
use for the quantitative assessment of individual calcium
intakes.
Keywords Age Æ Calcium Æ FFQ Æ Gender Æ Methods Æ
Osteoporosis Æ Questionnaire
Introduction
Calcium is an important nutrient for skeletal health
throughout the lifespan, and assumes a critical role in
the pathogenesis of osteoporosis [1]. In recent years,
considerable evidence has emerged with respect to the
effects of dietary calcium on bone health in all age
groups [2], and osteoporosis is no longer considered ageor gender-dependent [3]. These lines of reasoning
emphasise the need for lifelong adequate calcium intake
in both males and females [4]. Optimal calcium intake
refers to the levels of consumption that are necessary for
maximization of peak bone mass during childhood and
adolescence, maintenance of bone mass during adult
years, and minimization of bone loss later in life [5].
From the several macro- and micronutrients postulated
as putative determinants of bone mass and osteoporosis
risk, however, calcium is probably the most likely to be
inadequate in terms of dietary intake [6]. Assessing
dietary calcium intake could therefore be useful in both
clinical practice and research, and the need for developing cost-effective methods for identifying individuals
at all ages with insufficient calcium intakes from food
has been highlighted [5].
There is a wide range of methods that could be used
for the dietary assessment of individuals or groups of
people, and each one has advantages and disadvantages
that make it suitable for use in different settings and for
specific purposes [7]. In general, the choice of method
depends on the objectives of the study, the foods or
nutrients of primary interest, the need for group versus
individual data, the need for absolute versus relative
305
intake estimations, the characteristics of the population
(e.g. age, gender, literacy, motivation, socio-cultural
diversity), the time-frame of interest, the level of specificity needed, and available resources [8]. For instance,
estimation of ‘‘true’’ calcium intake for an individual
with a reasonable precision would ideally need 74–88
days of diet recording on average, ranging from 30 to
168 days depending on the magnitude of intra-individual
variability in daily food intake [9]. Unfortunately, this
would be impractical and unsuitable for most clinical
and research settings. Instead, food frequency questionnaires (FFQs) provide a practical, low-cost, and
self-administered tool for assessing usual consumption
patterns in large groups of people, and are widely used
for ranking individual nutrient intake, identifying individuals at the extremes of intake and, conditionally, for
providing quantitative information on individual intakes
as well [7].
In this respect, FFQs may be considered quite convenient for the assessment of dietary calcium intake. A
number of FFQs have been specifically designed for
calcium and validated for use in some population subgroups, most commonly in adult and elderly women
[10,11,12,13,14,15,16,17,18,19,20,21,22,23], and less frequently in children [24,25]. At present, therefore, not
only are there no relevant studies in men, but also, two
or three such questionnaires would have to be used
concurrently in large population-based epidemiological
surveys or in clinical settings, where individuals of all
ages and both genders would attend. This study, therefore, was designed to examine the feasibility of developing a calcium-specific FFQ for assessing dietary
calcium intake in the general population, without age or
gender discrimination.
Materials and methods
enrollment, children’s parents or guardians, as well as
the adults and the elderly participants, were fully informed about the objectives and methods of the study
and signed a written consent. The children provided
their verbal assent. Approval to conduct the survey was
granted by the Bioethics Committee of Harokopio
University, Athens, Greece.
Development of the FFQ
The calcium-specific FFQ used in the present study was
developed on the basis of a general semi-quantitative
questionnaire validated previously for use in Greek
adults [27]. From the initial 190-item FFQ, only those
foods identified as potentially major sources of dietary
calcium were included. A first draft of the questionnaire
was pilot-tested on approximately 140 unselected subjects, and adjustments were made on both items and
questions aiming at quantitative estimate of consumed
food. The final 30-item FFQ probed for ten dairy
products (milk, yogurt, and eight types of soft and hard
cheeses), four types of pie, two cereal products, two
types of nuts, four vegetable products, legumes, four fish
products, eggs, as well as ice cream and chocolate
(Appendix A). Although not all food items were likely to
contribute significant calcium to the diets of all subjects,
the questionnaire was designed as such to suit the needs
of both genders and all age groups. Respondents were
asked about the frequency (never or rarely, or times per
month/week/day, as appropriate) and amount (natural
units or standard quantities, but not actual weights) of
consumption of these foods during the previous 12
months. No visual aids or food models for estimating
portion sizes were provided, in order to make the process as simple and rapid as possible. The FFQ was selfadministered and completed by most participants within
approximately 5 min.
Sample
A total of 1060 individuals were randomly selected from
a larger cohort of subjects participating in a nationwide
survey of osteoporosis risk factors in Greece [26].
Apparently healthy children (10–15 years old, n=360),
adults (26–33 years old, n=300), and elderly people (60–
75 years old, n=400) were recruited. Among those asked
to participate, 37 declined the invitation, while 22 collected questionnaires were characterized as incomplete,
unreadable or misreported, and were excluded from the
analysis (overall participation rate was 94.4%). The final
sample (n=1001), therefore, consisted of 351 children
(189 girls and 162 boys, aged 11.9±1.2 years), 260
adults (192 women and 68 men, aged 29.6±2.7 years),
and 390 elderly individuals (317 women and 73 men,
aged 68.6±4.6 years). All subjects were healthy, did not
use any medications or dietary supplements (including
calcium), and had maintained stable body weights and
dietary habits for the previous 6 months (the stable body
weight criterion was not applied for children). Prior to
The ‘‘reference’’ method
The results from the FFQ were evaluated against those
from a multi-pass 24-h recall [28]. All interviews, lasting
approximately 30 min each, were carried out by three
trained dietitians and were held during morning hours.
Respondents were asked to recall the type and amount
of any food and beverage consumed during the previous
day in a chronological order, i.e. from the time they
woke up in the morning to the same time the following
day. Thereafter, the interviewer went through the food
list again to clarify entries and add possible forgotten
items. To improve the accuracy of food descriptions,
standard household measures (cups, tablespoons, etc.)
and picture food models (Dairy Food Council, USA)
were used during interviews to define amounts when
appropriate. Food intake data was analyzed by the
Nutritionist V diet analysis software (First DataBank
Inc., San Bruno, Calif., USA), amended to include
306
traditional Greek recipes as described in the Food
Composition Tables and Composition of Greek Cooked
Food and Dishes [29]. In addition, information on
processed foods was obtained from food companies and
national fast food chains, in order to enter in the
Nutritionist V database actual weights of processed
foods, as well as nutrient content, when available. The
same database was used to derive the calcium content of
the food items included in the FFQ (Appendix B).
Administration of the questionnaire always preceded the
24-h recall interview, to avoid a possible memory effect
from the use of picture food models during the latter.
Statistical analysis
Results are reported as means±SD, or as percentages
(%) and 95% confidence intervals (CI), unless otherwise
stated. To examine the quantitative efficiency of the
FFQ, daily calcium intakes from the two methods were
estimated and compared by the Student’s paired t-test.
Pearson’s correlation and linear regression analyzes were
also performed. The degree of agreement between the
FFQ and the 24-h recall for an individual was assessed
by computing the mean±2 SD (i.e. 95% CI) of the
difference [30]. The discriminative power of the FFQ was
evaluated by ascribing actual values for surrogate categories [17]. Briefly, subjects were first grouped into
quartiles or quintiles on the basis of their FFQ-calculated calcium intake. Then, the ‘‘true’’ calcium intake
derived from the 24-h recall was assigned to each of
these categories and compared by one-way analysis of
variance (ANOVA), followed by Tukey’s post hoc tests.
It should be clarified that ‘‘true’’ intakes in this case do
not represent 24-h recall quartiles or quintiles; rather,
they merely reflect actual intakes for each FFQ surrogate category. Cross-classification analysis was carried
out to identify the proportion of subjects correctly
classified (within one quartile) and grossly misclassified
(lowest quartile for one method and highest quartile for
the other) by the FFQ [25]. Sensitivity, specificity, positive and negative predictive values (PPV and NPV,
respectively) were also determined. For this purpose, the
identification of a subject with calcium intake less than a
specified cut-off level by the FFQ, who also fell below
this level on the basis of the 24-h recall, was considered a
‘‘true positive’’ finding. All analyzes were carried out
using SPSS 10.0.5 for Windows (SPSS Inc., Chicago, Ill.,
USA). Statistical significance was set at P<0.05.
133 mg/day equals, in fact, 15.4% of 861 mg/day) was
due to the uneven distribution of underestimation across
intakes (see below). The Pearson correlation coefficient
between calcium intake derived from the FFQ and the
24-h recall was moderately strong (r=0.639) and highly
significant (P<0.001) but, at the individual level, the
two methods showed poor agreement. The latter is
illustrated by the Bland-Altman plot (Fig. 1), where the
95% limits of agreement are shown. Apparently, the
FFQ could provide estimates of calcium intake from
533 mg/day above to 799 mg/day below the 24-h recall.
Such a range (more than 1300 mg/day) cannot be considered satisfactory or acceptable for the quantitative
assessment of individual calcium intake.
It is worth noting that a statistically significant inverse correlation was observed between ‘‘true’’ calcium
intake derived from the 24-h recall and the magnitude of
underestimation by the FFQ (r=–0.553, P<0.001), the
latter defined as the difference in calcium intake by
the FFQ minus that by the 24-h recall. This means that
Table 1 Calcium intakes by the FFQ and the 24-h recall (n=1001)
Calcium by FFQ (mg/day)
Calcium by 24-h recall (mg/day)
Difference (mg/day)
Absolute difference (mg/day)
Difference (%)
Absolute difference (%)
Mean±SD
95% CI
728±361*
861±415
–133±333
284±219
–5.4±47.6
36.9±30.5
705, 750
835, 887
–154, –112
271, 298
–8.4, –2.5
35.1, 38.8
*P<0.001 vs calcium intake by the 24-h recall, by paired t-test
P<0.001 vs zero, by one-sample t-test
Results
Calcium intakes by both the FFQ and the 24-h
recall were normally distributed, as revealed by the
Kolmogorov-Smirnov test. The FFQ significantly
underestimated mean calcium intake compared to the
24-h recall by approximately 133 mg/day or 5.4%
(P<0.001; Table 1). This apparent discrepancy (i.e.
Fig. 1 Bland-Altman plot of agreement between the FFQ and the
24-h recall. ‘‘Calcium difference’’ (y-axis) is the difference in
calcium intake by the FFQ minus that by the 24-h recall, while
‘‘calcium mean’’ (x-axis) is the mean of calcium intake by the two
methods. The mean (solid line) and the 95% CI (broken lines) of the
difference are shown
307
the higher the intake, the higher the underestimation.
However, the linear regression of the difference between
the two methods on calcium intake by the 24-h recall
was also significant, albeit of low predictive power
(F=439.4, SEE=278 mg/day, r2=0.305, P<0.0001):
difference in calcium intake (mg/day)=249 (±20)–0.444
(±0.021)·calcium intake by 24-h recall (mg/day). This
equation (coefficients are shown with standard error in
parenthesis) yields a zero value for the difference (i.e. no
underestimation) when calcium intake by the 24-h recall
equals to approximately 561 mg/day. For intakes less
than that, positive values are derived, i.e. overestimation
occurs. The phenomenon of overestimation at low intakes and underestimation at high intakes is a rather
common feature of FFQs, known as the ‘‘flat-slope
syndrome’’ (Fig. 2) [7].
Consequent to that, however, when differences between the two methods were expressed as percentages
(dividing by the 24-h recall intake), relative overestimations at low intakes more than counterbalanced relative underestimations at high intakes. This is illustrated
in Fig. 3, where the percentage difference is plotted
against the 24-h recall intake. Again, a significant negative linear correlation was observed (r=–0.437,
P<0.001), but the regression equation more closely
describing this relationship was an inverse function of
‘‘true’’ intake (F=363.2, SEE=40.8%, r2=0.267,
P<0.0001): Difference in calcium intake (%)=–39.16
(±2.19)+21452 (±1126)/calcium intake by 24-h recall
(mg/day). The zero intercept here is at approximately
548 mg/day. Apparently, therefore, percentage differences could range from +¥ (for intakes close to zero) to
–39.16% (for intakes close to infinite), but not lower.
Fig. 2 The ‘‘flat-slope syndrome’’ of the FFQ. ‘‘Calcium difference’’ (y-axis) is the difference in calcium intake by the FFQ minus
that by the 24-h recall, while ‘‘calcium by 24-h recall’’ (x-axis) is the
calcium intake by the 24-h recall. The linear regression line (solid
line) and the zero line (broken line) are shown. The zero intercept is
at approximately 561 mg/day
Fig. 3 Relative differences as a function of ‘true’ intake. ‘‘Calcium
difference’’ (y-axis) is the difference in calcium intake by the FFQ
minus that by the 24-h recall, expressed as percentage of the latter,
while ‘‘calcium by 24-h recall’’ (x-axis) is the calcium intake by the
24-h recall
Hence at low ‘‘true’’ intakes, e.g. 100 mg/day, overestimation was approximately +175% (corresponding to
+175 mg/day), whereas at high intakes, e.g. 1000 mg/
day, underestimation was only –17.7% (corresponding
to –177 mg/day). While absolute differences would
more or less cancel out (+175–177=–2 mg/day),
relative differences would certainly not (+175–
17.7=+157.3%). Overall, this is why the ratio of the
means (i.e. –133/861=–15.4%) was so much different
from the mean of ratios (i.e. –5.4%). Because the mean
of a distribution that spans zero substantially underestimates the absolute error (i.e. regardless of sign), which
is important when dealing with individuals, mean
absolute differences between the FFQ and the 24-h recall
are also shown in Table 1. Whether expressed in mg/day
or as percentages of ‘‘true’’ intake, absolute differences
between the two methods were significantly different
from zero (P<0.001), indicating the inability of the
FFQ to accurately estimate individual calcium intakes.
Actual values for surrogate FFQ quartiles manifested
a progressive increase with significant differences
between mean calcium intakes (F=158.9, P=0.001;
Table 2). The latter suggests that the FFQ could reliably
distinguish between categories at any level of intake. It is
worth mentioning that, individuals whose calcium intake was overestimated by the FFQ (due to the flat-slope
syndrome referred to above) were more likely to have
been classified into the second quartile, rather than the
first. When five surrogate categories were assigned, the
FFQ lost its discriminatory ability between the second
and the third quintile (P=0.142), but it was still highly
discriminative across all the remaining categories
(F=126.8, P<0.001; Table 2). Following a stepwise
increase in the number of surrogate categories, it was
observed that the FFQ could readily distinguish between
308
Table 2 Discriminatory power of the FFQ. For each FFQ quartile or quintile, ‘‘true’’ values for calcium intake (mean±SD) by the 24-h
recall have been calculated. These latter intakes do not represent 24-h recall quartiles or quintiles
FFQ
quartiles
n
Calcium range
by FFQ (mg/day)
Calcium intake
by 24-h recall
(mg/day)
FFQ
quintiles
n
Calcium range
by FFQ (mg/day)
Calcium intake
by 24-h recall
(mg/day)
1
2
3
4
–
250
251
250
250
–
<473
473–676
676–951
>951
–
576±292a
760±326b
886±343c
1223±397d
–
1
2
3
4
5
200
200
201
200
200
<425
425–600
600–762
762–1025
>1025
551±283a
733±324b
811±334b
938±348c
1272±394d
a,b,c,d
Quartiles or quintiles not sharing the same letter are statistically significantly different from each other at P<0.001 (i.e. they are
discriminated), by one-way ANOVA and Tukey’s post hoc tests. Quintiles 2 and 3 share the same letter and are not different from each
other (i.e. they cannot be discriminated by the FFQ)
has a considerable influence on skeletal health [2], and
its adequate intake from food is of major importance
for preventing osteoporosis and reducing fracture risk
[4]. However, evidence suggests that many individuals
fail to meet their calcium needs [6,32], hence putting
themselves at increased risk. Taking all this information into consideration, and because the consequences
of osteoporosis are multidimensional (i.e. financial,
physical, and psychosocial) and affect both the individual as well as the family and the community [3], a
recommendation was made for developing cost-effective
methods to identify those with insufficient calcium intakes [5].
Along this line, several FFQs for assessing dietary
calcium consumption have been designed and validated
for use, most commonly among adult and elderly women [10,11,12,13,14,15,16,17,18,19,20,21,22,23]. Similar
studies in children are scarce [24,25], while absent in
men. At our present state of knowledge, therefore,
available FFQs for calcium are age- and/or genderspecific, and this may be of concern in large-scale
population-based epidemiological surveys or in clinical
practice, where the need to evaluate dietary calcium
intake of individuals at all ages and both genders
would arise. The present study was designed to fill in
this gap, by developing a calcium FFQ for the general
population, including male and female children, adults,
and elderly subjects. Although results are mainly relevant to the Greek population, as dietary assessment
methods need to be culturally sensitive with respect to
food intake patterns [33], they are also indicative of the
feasibility of developing such a FFQ, besides its practicality, and could be of some use for similar studies in
the future.
the lowest (<500 mg/day) and the highest (>1000 mg/
day) calcium intakes up until ten such categories were
assigned. In this case, the FFQ could discriminate the
first (473±242 mg/day), the six intermediate (second
through seventh), and the three higher (eighth,
1020±343 mg/day, through tenth) intake categories
(F=69.0, P<0.001), but was unable to distinguish the
intermediate ones (i.e. those falling between 600–
900 mg/day) from each other.
To enable comparison with similar studies, sensitivity, specificity, PPV and NPV have been calculated setting the cut-off level for calcium intake at 800 mg/day.
The FFQ had a moderate-to-high sensitivity (82.8%)
and a moderate-to-low specificity (54.9%), while PPV
(64.3%) and NPV (76.6%) were only moderate. Use of
the age-specific adequate intakes for calcium, i.e. 1300,
1000, and 1200 mg/day for children, adults, and the elderly, respectively [31], resulted in increased sensitivity
(95.5%) and PPV (84.0%), but also in decreased specificity (34.1%) and NPV (67.9%). The cross-classification
analysis indicated that only 17 subjects (1.7%) were
grossly misclassified, while 827 subjects (82.6%) were
correctly classified (Table 3).
Discussion
Although osteoporosis was once thought to be a natural part of aging among women, it is now widely
accepted that the disease may affect all individuals
regardless of age or gender [3]. Optimization of bone
health by appropriate dietary and lifestyle practices,
therefore, is a process that must occur throughout the
lifespan in both males and females [3]. Dietary calcium
Table 3 Cross-classification
analysis for the FFQ.
Frequencies and marginal
distributions are shown
24-h recall quartiles
FFQ quartiles
1
2
3
4
Total
1
2
3
4
Total
130 (52.0)
73 (29.1)
44 (17.6)
3 (1.2)
250
66 (26.4)
74 (29.5)
75 (30.0)
36 (14.4)
251
40 (16.0)
67 (26.7)
72 (28.8)
71 (28.4)
250
14 (5.6)
37 (14.7)
59 (23.6)
140 (56.0)
250
250
251
250
250
1001
309
Our findings indicate that the FFQ tended to
underestimate mean calcium intake by approximately –
133 mg/day or –5.4% compared to the 24-h recall (Table 1). Also, at the individual level, the questionnaire
could provide estimates of calcium intake from 533 mg/
day above to 799 mg/day below the 24-h recall, meaning
that its quantitative accuracy for an individual spanned
over approximately 1300 mg/day (Fig. 1). These results
are similar to those obtained for some of the previous
FFQs [15,19,25], but are certainly worse than others,
where no statistically significant differences in mean
calcium intakes derived from the FFQ and the reference
method were observed [10,12,14,16,17,20,21], and the
95% limits of agreement were tighter [17,21]. From the
quantitative perspective, therefore, this FFQ cannot be
considered appropriate for estimating actual calcium
intake of an individual.
On the other hand, the linear correlation between the
two methods (r=0.639, P<0.001) was in the range reported previously (r=0.5–0.8) [11,13,14,18,20,21,23,25],
although in one study, a coefficient equal to 0.9 was
obtained [17]. The FFQ had good discriminatory power
at all levels of calcium intake when quartiles were assigned, but it could not differentiate between the second
and the third quintile (Table 2), hence being more efficient than some [21,25] but less efficient than other [17]
questionnaires in this respect. Still, it maintained a fair
ability to discriminate the lower (<500 mg/day) and the
higher (>1000 mg/day) calcium intakes from each other
and from the intermediate ones (600–900 mg/day), up
until ten surrogate categories were assigned. This, in
combination with the weak tendency to overestimate
calcium intake at low intakes and underestimate it at
high intakes (Fig. 2), strengthens the clinical value of an
intake below 500 mg/day or above 1000 mg/day by the
FFQ, since the former will most likely represent an
overestimation and the latter an underestimation of
‘‘true’’ calcium intake. This property may prove
important, since it is now well established that calcium is
a threshold nutrient, in that benefit and risk are not
linearly related to intake. More specifically, risk of
fractures seems to increase only in those subjects whose
calcium intake is less than 500 mg/day [34], while no
apparent health benefit is observed at intakes above
1000–1500 mg/day [4].
The FFQ demonstrated a reasonable ability to classify individuals into quartiles of calcium intake, with
82.6% of the subjects being correctly classified into the
same or adjacent quartile, and only 1.7% being grossly
misclassified (Table 3). Respective proportions in the
literature vary from 0–1.2% [14,15] to 3.2–3.4% [21,25]
for gross misclassification, and from 81–84.1% [21,25] to
92.9–95.2% [14,15] for correct classification. Sensitivity
in identifying subjects with intakes less than 800 mg/day
was relatively high (82.8%), but specificity in identifying
those with intakes more than 800 mg/day fell short
(54.9%). Using higher cut-off levels for calcium intake,
such as the age-specific adequate intakes, resulted
in increased sensitivity and decreased specificity, in
accordance with previous observations [14]. These values might be expected keeping in mind the tendency of
the FFQ to underestimate calcium intake in general, and
can be considered moderate at best compared with other
questionnaires [14,17,21,25].
Some discussion on the validation method is also
justified. Virtually all of the available techniques have
been used previously as reference tools against which the
various calcium FFQs have been validated, including
modified diet history interviews [10], 4-day weighed
[11,19] or semi-weighed [22] food records, 3-day [14],
4-day [18,25], 7-day [20,21,23], or 14-day [17] estimated
food records, as well as 24-h diet recalls [15,16] and
full-length FFQs [12,13]. Depending on the more or less
burdensome nature of the reference method, sample
sizes have ranged from approximately 20–60
[10,11,15,18,19,23,25] to approximately 100 [13,20,21,22]
subjects, although in some studies greater numbers were
recruited, e.g. more than 200 [14,17] or 500 [12] individuals. Although our choice of the 24-h recall as a
reference tool may not be the best for validation purposes, nevertheless, this method is considered the most
suitable to get population means and distributions for
subjects aged 10 years and over with reasonable accuracy, especially when combined with visual aids for
estimating portion sizes [8]. Thus, the 24-h recall was
preferred over other methods under these circumstances,
as a practical, cost-effective, and fairly accurate technique. Also, because of the large number of subjects
surveyed, inter-individual variability in daily food intake
would be expected to decrease [7].
Another limitation that should be acknowledged is
the lack of information on the reproducibility (reliability) of the FFQ. This is usually assessed by administering the questionnaire at two (or more) points in time to
the same group of people. Generally, however, less than
half of all FFQs being validated are tested for reproducibility [35, 36]. In the case of Ca calcium-specific
questionnaires, we could identify only three studies that
actually did that [12,13,21]. Bearing in mind that the
present FFQ was self-administered, the two major concerns regarding reproducibility, namely, intra- and interrater reliability, would automatically cancel out, as no
interviewer was involved [35]. Therefore, not testing for
reproducibility might not be as much of a problem for
this FFQ as it would, for example, for those questionnaires administered by trained dietitians (e.g. [17]). What
is more, in order to assess reproducibility, one must allow for sufficient time before re-administrating the
questionnaire, so as to avoid a possible memory effect on
the respondents’ behalf. Because the present FFQ was
quite short (30 items), and hence the responses were easy
to remember, the time interval before re-administration
would have to be longer. In this instance, however, true
changes in dietary intake might have occurred (as they
usually do), contributing to falsely low reproducibility
[37]. For all these reasons, we feel that absence of any
information on the reliability of the FFQ is probably not
as critical, though indeed an omission.
310
Fig. 4 The 30-item FFQ developed and used for estimating usual calcium intake in the Greek population
In summary, the present study extended on previous
ones by developing an age- and gender-independent
FFQ for assessing dietary calcium intake in the general
population. Evaluation has been conducted in subjects
with various levels of education, different socioeconomic
statuses, and degrees of cooperation, and not merely in
311
highly motivated individuals, hence strengthening its
applicability in practice. The FFQ could be used in
large-scale epidemiological surveys or in clinical settings
as a rapid method for ranking calcium intake, as well as
for discriminating relatively low and probably insufficient (<500 mg/day) from relatively high and probably
sufficient (>1000 mg/day) calcium intakes. Quantitative
estimations at the individual level, however, should be
treated with prudence.
Appendix A
Figure 4 shows the 30-item FFQ that was developed and
used for estimating usual calcium intake in the Greek
population. The questionnaire was self-administered by
the subjects and then returned for analysis. The final two
questions were included for cross-checking purposes (i.e.
subjects who answered ‘‘yes’’ in either one were excluded
from subsequent analysis; see Materials and methods).
Appendix B
Table 4 lists the calcium content of each FFQ item.
Table 4 Spreadsheet showing
the calcium content of each
FFQ item
Acknowledgements This study was part of a nationwide project on
osteoporosis, supported by Friesland Hellas. The authors would
like to thank Maria Bletsa, Maria Rammata, and Anastasia
Doulgeri, dietitians, Silia Sidossis, research assistant, and Antigoni
Tsiafitsa, technician, for their valuable help in data collection and
processing.
References
1. Nordin BE (1997) Calcium and osteoporosis. Nutrition 13:664–
686
2. European Commission (1998) Report on Osteoporosis in the
European Community: action for prevention. Office for Official
Publications for the European Commission, Luxembourg
3. National Institutes of Health (2000) Osteoporosis prevention,
diagnosis, and therapy. NIH Consensus Statement 17:1–45
4. Heaney RP (2002) The importance of calcium intake for lifelong skeletal health. Calcif Tissue Int 70:70–73
5. National Institutes of Health (1994) Optimal calcium intake.
NIH Consensus Statement 12:1–31
6. Weaver CM (2000) The growing years and prevention of
osteoporosis in later life. Proc Nutr Soc 59:303–306
7. Magkos F, Yannakoulia M (2003) Methodology of dietary
assessment in athletes: concepts and pitfalls. Curr Opin Clin
Nutr Metab Care 6:539–549
8. Biro G, Hulshof KF, Ovesen L, Amorim Cruz JA (2002)
Selection of methodology to assess food intake. Eur J Clin Nutr
56:S25–32
Food item
FFQ quantity
Weight equivalent
(g)
Calcium content
(mg)
Milk
Yogurt
Feta cheese
Graviera cheese
Kaseri cheese
Mozzarella
Emmenthal, cheddar, etc.
Parmesan or kefalotiri cheese
Anthotiro cheese
Mashed cheese
Cheese pie
Cream pie
Leafy vegetables pie
Spinach pie with cheese
Bread and similar
Cereals
Peanuts or almonds
Other nuts
Spinach
Salad vegetables
Green vegetables
Potatoes
Legumes
Sardines
Scallops
Shrimps
White fish or salmon
Eggs
Ice cream
Chocolate
1 glass
1 pot
1 matchbox
1 matchbox
1 matchbox
1 matchbox
1 matchbox
1 tablespoon
1 tablespoon
1 tablespoon
1 serving
1 serving
1 serving
1 serving
1 slice or 1 piece
1/2 cup
1 handful
1 handful
1 cup
1/2 cup
1/2 cup
1 medium
1 cup
10 small
1 serving
1 serving
1 serving
1 egg
2 scoops
5 squares
250
225
20
20
20
20
20
15
20
15
100
100
100
100
30
100
25
25
240
120
120
100
240
60
100
100
100
40
100
25
250
280
65
180
150
88
160
120
50
56
300
150
50
200
25
20
60
30
216
60
60
10
96
200
100
200
70
25
151
50
312
9. Basiotis PP, Welsh SO, Cronin FJ, Kelsay JL, Mertz W (1987)
Number of days of food intake records required to estimate
individual and group nutrient intakes with defined confidence. J
Nutr 117:1638–1641
10. Angbratt M, Moller M (1999) Questionnaire about calcium
intake: can we trust the answers? Osteoporos Int 9:220–225
11. Angus RM, Sambrook PN, Pocock NA, Eisman JA (1989) A
simple method for assessing calcium intake in Caucasian women. J Am Diet Assoc 89:209–214
12. Blalock SJ, Currey SS, DeVellis RF, Anderson JJ, Gold DT,
Dooley MA (1998) Using a short food frequency questionnaire
to estimate dietary calcium consumption: a tool for patient
education. Arthr Care Res 11:479–484
13. Brown JL, Griebler R (1993) Reliability of a short and long
version of the Block food frequency form for assessing changes
in calcium intake. J Am Diet Assoc 93:784–789
14. Chee WS, Suriah AR, Zaitun Y, Chan SP, Yap SL, Chan YM
(2002) Dietary calcium intake in postmenopausal Malaysian
women: comparison between the food frequency questionnaire
and three-day food records. Asia Pac J Clin Nutr 11:142–146
15. Green JH, Booth CL, Bunning RL (2002) Assessment of a
rapid method for assessing adequacy of calcium intake. Asia
Pac J Clin Nutr 11:147–150
16. Haines CJ, Chung TK, Leung PC, Leung DH, Wong MY, Lam
LL (1994) Dietary calcium intake in postmenopausal Chinese
women. Eur J Clin Nutr 48:591–594
17. Montomoli M, Gonnelli S, Giacchi M, Mattei R, Cuda C,
Rossi S, Gennari C (2002) Validation of a food frequency
questionnaire for nutritional calcium intake assessment in
Italian women. Eur J Clin Nutr 56:21–30
18. Musgrave KO, Giambalvo L, Leclerc HL, Cook RA, Rosen CJ
(1989) Validation of a quantitative food frequency questionnaire for rapid assessment of dietary calcium intake. J Am Diet
Assoc 89:1484–1488
19. Pasco JA, Sanders KM, Henry MJ, Nicholson GC, Seeman E,
Kotowicz MA (2000) Calcium intakes among Australian women: Geelong Osteoporosis Study. Aust N Z J Med 30:21–27
20. Taitano RT, Novotny R, Davis JW, Ross PD, Wasnich RD
(1995) Validity of a food frequency questionnaire for estimating calcium intake among Japanese and white women. J Am
Diet Assoc 95:804–806
21. Wilson P, Horwath C (1996) Validation of a short food frequency questionnaire for assessment of dietary calcium intake
in women. Eur J Clin Nutr 50:220–228
22. Xu L, Porteous JE, Phillips MR, Zheng S (2000) Development
and validation of a calcium intake questionnaire for postmenopausal women in China. Ann Epidemiol 10:169–175
23. Cummings SR, Block G, McHenry K, Baron RB (1987)
Evaluation of two food frequency methods of measuring dietary calcium intake. Am J Epidemiol 126:796–802
24. Bellu R, Riva E, Ortisi MT, De Notaris R, Santini I, Banderali
G, Giovannini M (1995) Calcium intakes in a sample of 35,000
Italian schoolchildren. J Int Med Res 23:191–199
25. Taylor RW, Goulding A (1998) Validation of a short food
frequency questionnaire to assess calcium intake in children
aged 3 to 6 years. Eur J Clin Nutr 52:464–465
26. Magkos F, Manios Y, Babaroutsi E, Sidossis LS (2004)
Quantitative ultrasound calcaneus measurements: normative
data for the Greek population. Osteoporos Int (in press)
27. Katsouyanni K, Rimm EB, Gnardellis C, Trichopoulos D,
Polychronopoulos E, Trichopoulou A (1997) Reproducibility
and relative validity of an extensive semi-quantitative food
frequency questionnaire using dietary records and biochemical
markers among Greek schoolteachers. Int J Epidemiol
26:S118–127
28. Thompson FE, Byers T (1994) Dietary assessment resource
manual. J Nutr 124:2245S–317S
29. Trichopoulou A (1992) Food composition tables and composition of Greek cooked food and dishes. School of Public
Health, Athens
30. Bland JM, Altman DG (1986) Statistical methods for assessing
agreement between two methods of clinical measurement.
Lancet 1:307–310
31. Institute of Medicine (1997) Dietary reference intakes for calcium, phosphorus, magnesium, vitamin D, and fluoride. National Academy Press, Washington D.C.
32. Nicklas TA (2003) Calcium intake trends and health consequences from childhood through adulthood. J Am Coll Nutr
22:340–356
33. Hankin JH, Wilkens LR (1994) Development and validation of
dietary assessment methods for culturally diverse populations.
Am J Clin Nutr 59:198S–200S
34. Lau EM, Cooper C (2001) Risk factors for osteoporosis in
Europe. J Bone Miner Metab 19:142–145
35. Cade J, Thompson R, Burley V, Warm D (2002) Development,
validation and utilisation of food-frequency questionnaires—a
review. Public Health Nutr 5:567–587
36. Cade JE, Burley VJ, Warm DL, Thompson RL, Margetts BM
(2004) Food-frequency questionnaires: a review of their design,
validation and utilisation. Nutr Res Rev 17:5–23
37. Tsubono Y, Nishino Y, Fukao A, Hisamichi S, Tsugane S
(1995) Temporal change in the reproducibility of a selfadministered food frequency questionnaire. Am J Epidemiol
142:1231–1235
DOI 10.1007/s00198-004-1679-1
O R I GI N A L A R T IC L E
Development and validation of a food frequency questionnaire for
assessing dietary calcium intake in the general population
Faidon Magkos Æ Yannis Manios Æ Eirini Babaroutsi
Labros S. Sidossis
Received: 19 January 2004 / Accepted: 17 May 2004 / Published online: 10 September 2004
International Osteoporosis Foundation and National Osteoporosis Foundation 2004
Abstract The aim of this study was to develop and
evaluate a food frequency questionnaire (FFQ) for
assessing dietary calcium intake in the general population, since all available questionnaires at present are ageand/or gender-specific. A total of 1001 individuals
(including children, adults, and elderly people of both
genders) were randomly recruited throughout Greece.
Estimates of calcium intake from the 30-item FFQ were
compared with those from a multi-pass 24-h recall. The
FFQ underestimated mean calcium intake compared to
the 24-h recall by (mean±SD) –133±333 mg/day or –
5.4±47.6% (P<0.001). The two methods were strongly
correlated (r=0.639, P<0.001), but the 95% limits of
agreement for individual assessment were rather wide, as
the FFQ could provide estimates of calcium intake from
533 mg/day above to 799 mg/day below the 24-h recall.
Actual values for surrogate FFQ quartiles manifested a
progressive increase, with significant differences between
mean calcium intakes (P<0.001). The FFQ could
identify individuals who consumed less calcium than
800 mg/day or less than the age-specific adequate intake
with a relatively high sensitivity (82.8 and 95.5%,
respectively), but low specificity (54.9 and 34.1%,
respectively). Cross-classification analysis indicated that
only 17 subjects (1.7%) were grossly misclassified (lowest quartile for one method and highest quartile for the
other), while 827 subjects (82.6%) were correctly classified (into the same or adjacent quartiles). The FFQ
could be used in population-based epidemiological
studies or screening programs involving individuals of
all ages and both genders, where the discrimination of
subjects with relatively low (<500 mg/day) and
relatively high (>1000 mg/day) calcium intakes is of
F. Magkos Æ Y. Manios Æ E. Babaroutsi Æ L.S. Sidossis (&)
Laboratory of Nutrition and Clinical Dietetics,
Department of Nutrition and Dietetics,
Harokopio University, 70 El. Venizelou Avenue,
17671 Athens, Greece
E-mail: [email protected]
Tel.: +30-210-9549154
Fax: +30-210-9549141
primary interest. Results, however, do not support its
use for the quantitative assessment of individual calcium
intakes.
Keywords Age Æ Calcium Æ FFQ Æ Gender Æ Methods Æ
Osteoporosis Æ Questionnaire
Introduction
Calcium is an important nutrient for skeletal health
throughout the lifespan, and assumes a critical role in
the pathogenesis of osteoporosis [1]. In recent years,
considerable evidence has emerged with respect to the
effects of dietary calcium on bone health in all age
groups [2], and osteoporosis is no longer considered ageor gender-dependent [3]. These lines of reasoning
emphasise the need for lifelong adequate calcium intake
in both males and females [4]. Optimal calcium intake
refers to the levels of consumption that are necessary for
maximization of peak bone mass during childhood and
adolescence, maintenance of bone mass during adult
years, and minimization of bone loss later in life [5].
From the several macro- and micronutrients postulated
as putative determinants of bone mass and osteoporosis
risk, however, calcium is probably the most likely to be
inadequate in terms of dietary intake [6]. Assessing
dietary calcium intake could therefore be useful in both
clinical practice and research, and the need for developing cost-effective methods for identifying individuals
at all ages with insufficient calcium intakes from food
has been highlighted [5].
There is a wide range of methods that could be used
for the dietary assessment of individuals or groups of
people, and each one has advantages and disadvantages
that make it suitable for use in different settings and for
specific purposes [7]. In general, the choice of method
depends on the objectives of the study, the foods or
nutrients of primary interest, the need for group versus
individual data, the need for absolute versus relative
305
intake estimations, the characteristics of the population
(e.g. age, gender, literacy, motivation, socio-cultural
diversity), the time-frame of interest, the level of specificity needed, and available resources [8]. For instance,
estimation of ‘‘true’’ calcium intake for an individual
with a reasonable precision would ideally need 74–88
days of diet recording on average, ranging from 30 to
168 days depending on the magnitude of intra-individual
variability in daily food intake [9]. Unfortunately, this
would be impractical and unsuitable for most clinical
and research settings. Instead, food frequency questionnaires (FFQs) provide a practical, low-cost, and
self-administered tool for assessing usual consumption
patterns in large groups of people, and are widely used
for ranking individual nutrient intake, identifying individuals at the extremes of intake and, conditionally, for
providing quantitative information on individual intakes
as well [7].
In this respect, FFQs may be considered quite convenient for the assessment of dietary calcium intake. A
number of FFQs have been specifically designed for
calcium and validated for use in some population subgroups, most commonly in adult and elderly women
[10,11,12,13,14,15,16,17,18,19,20,21,22,23], and less frequently in children [24,25]. At present, therefore, not
only are there no relevant studies in men, but also, two
or three such questionnaires would have to be used
concurrently in large population-based epidemiological
surveys or in clinical settings, where individuals of all
ages and both genders would attend. This study, therefore, was designed to examine the feasibility of developing a calcium-specific FFQ for assessing dietary
calcium intake in the general population, without age or
gender discrimination.
Materials and methods
enrollment, children’s parents or guardians, as well as
the adults and the elderly participants, were fully informed about the objectives and methods of the study
and signed a written consent. The children provided
their verbal assent. Approval to conduct the survey was
granted by the Bioethics Committee of Harokopio
University, Athens, Greece.
Development of the FFQ
The calcium-specific FFQ used in the present study was
developed on the basis of a general semi-quantitative
questionnaire validated previously for use in Greek
adults [27]. From the initial 190-item FFQ, only those
foods identified as potentially major sources of dietary
calcium were included. A first draft of the questionnaire
was pilot-tested on approximately 140 unselected subjects, and adjustments were made on both items and
questions aiming at quantitative estimate of consumed
food. The final 30-item FFQ probed for ten dairy
products (milk, yogurt, and eight types of soft and hard
cheeses), four types of pie, two cereal products, two
types of nuts, four vegetable products, legumes, four fish
products, eggs, as well as ice cream and chocolate
(Appendix A). Although not all food items were likely to
contribute significant calcium to the diets of all subjects,
the questionnaire was designed as such to suit the needs
of both genders and all age groups. Respondents were
asked about the frequency (never or rarely, or times per
month/week/day, as appropriate) and amount (natural
units or standard quantities, but not actual weights) of
consumption of these foods during the previous 12
months. No visual aids or food models for estimating
portion sizes were provided, in order to make the process as simple and rapid as possible. The FFQ was selfadministered and completed by most participants within
approximately 5 min.
Sample
A total of 1060 individuals were randomly selected from
a larger cohort of subjects participating in a nationwide
survey of osteoporosis risk factors in Greece [26].
Apparently healthy children (10–15 years old, n=360),
adults (26–33 years old, n=300), and elderly people (60–
75 years old, n=400) were recruited. Among those asked
to participate, 37 declined the invitation, while 22 collected questionnaires were characterized as incomplete,
unreadable or misreported, and were excluded from the
analysis (overall participation rate was 94.4%). The final
sample (n=1001), therefore, consisted of 351 children
(189 girls and 162 boys, aged 11.9±1.2 years), 260
adults (192 women and 68 men, aged 29.6±2.7 years),
and 390 elderly individuals (317 women and 73 men,
aged 68.6±4.6 years). All subjects were healthy, did not
use any medications or dietary supplements (including
calcium), and had maintained stable body weights and
dietary habits for the previous 6 months (the stable body
weight criterion was not applied for children). Prior to
The ‘‘reference’’ method
The results from the FFQ were evaluated against those
from a multi-pass 24-h recall [28]. All interviews, lasting
approximately 30 min each, were carried out by three
trained dietitians and were held during morning hours.
Respondents were asked to recall the type and amount
of any food and beverage consumed during the previous
day in a chronological order, i.e. from the time they
woke up in the morning to the same time the following
day. Thereafter, the interviewer went through the food
list again to clarify entries and add possible forgotten
items. To improve the accuracy of food descriptions,
standard household measures (cups, tablespoons, etc.)
and picture food models (Dairy Food Council, USA)
were used during interviews to define amounts when
appropriate. Food intake data was analyzed by the
Nutritionist V diet analysis software (First DataBank
Inc., San Bruno, Calif., USA), amended to include
306
traditional Greek recipes as described in the Food
Composition Tables and Composition of Greek Cooked
Food and Dishes [29]. In addition, information on
processed foods was obtained from food companies and
national fast food chains, in order to enter in the
Nutritionist V database actual weights of processed
foods, as well as nutrient content, when available. The
same database was used to derive the calcium content of
the food items included in the FFQ (Appendix B).
Administration of the questionnaire always preceded the
24-h recall interview, to avoid a possible memory effect
from the use of picture food models during the latter.
Statistical analysis
Results are reported as means±SD, or as percentages
(%) and 95% confidence intervals (CI), unless otherwise
stated. To examine the quantitative efficiency of the
FFQ, daily calcium intakes from the two methods were
estimated and compared by the Student’s paired t-test.
Pearson’s correlation and linear regression analyzes were
also performed. The degree of agreement between the
FFQ and the 24-h recall for an individual was assessed
by computing the mean±2 SD (i.e. 95% CI) of the
difference [30]. The discriminative power of the FFQ was
evaluated by ascribing actual values for surrogate categories [17]. Briefly, subjects were first grouped into
quartiles or quintiles on the basis of their FFQ-calculated calcium intake. Then, the ‘‘true’’ calcium intake
derived from the 24-h recall was assigned to each of
these categories and compared by one-way analysis of
variance (ANOVA), followed by Tukey’s post hoc tests.
It should be clarified that ‘‘true’’ intakes in this case do
not represent 24-h recall quartiles or quintiles; rather,
they merely reflect actual intakes for each FFQ surrogate category. Cross-classification analysis was carried
out to identify the proportion of subjects correctly
classified (within one quartile) and grossly misclassified
(lowest quartile for one method and highest quartile for
the other) by the FFQ [25]. Sensitivity, specificity, positive and negative predictive values (PPV and NPV,
respectively) were also determined. For this purpose, the
identification of a subject with calcium intake less than a
specified cut-off level by the FFQ, who also fell below
this level on the basis of the 24-h recall, was considered a
‘‘true positive’’ finding. All analyzes were carried out
using SPSS 10.0.5 for Windows (SPSS Inc., Chicago, Ill.,
USA). Statistical significance was set at P<0.05.
133 mg/day equals, in fact, 15.4% of 861 mg/day) was
due to the uneven distribution of underestimation across
intakes (see below). The Pearson correlation coefficient
between calcium intake derived from the FFQ and the
24-h recall was moderately strong (r=0.639) and highly
significant (P<0.001) but, at the individual level, the
two methods showed poor agreement. The latter is
illustrated by the Bland-Altman plot (Fig. 1), where the
95% limits of agreement are shown. Apparently, the
FFQ could provide estimates of calcium intake from
533 mg/day above to 799 mg/day below the 24-h recall.
Such a range (more than 1300 mg/day) cannot be considered satisfactory or acceptable for the quantitative
assessment of individual calcium intake.
It is worth noting that a statistically significant inverse correlation was observed between ‘‘true’’ calcium
intake derived from the 24-h recall and the magnitude of
underestimation by the FFQ (r=–0.553, P<0.001), the
latter defined as the difference in calcium intake by
the FFQ minus that by the 24-h recall. This means that
Table 1 Calcium intakes by the FFQ and the 24-h recall (n=1001)
Calcium by FFQ (mg/day)
Calcium by 24-h recall (mg/day)
Difference (mg/day)
Absolute difference (mg/day)
Difference (%)
Absolute difference (%)
Mean±SD
95% CI
728±361*
861±415
–133±333
284±219
–5.4±47.6
36.9±30.5
705, 750
835, 887
–154, –112
271, 298
–8.4, –2.5
35.1, 38.8
*P<0.001 vs calcium intake by the 24-h recall, by paired t-test
P<0.001 vs zero, by one-sample t-test
Results
Calcium intakes by both the FFQ and the 24-h
recall were normally distributed, as revealed by the
Kolmogorov-Smirnov test. The FFQ significantly
underestimated mean calcium intake compared to the
24-h recall by approximately 133 mg/day or 5.4%
(P<0.001; Table 1). This apparent discrepancy (i.e.
Fig. 1 Bland-Altman plot of agreement between the FFQ and the
24-h recall. ‘‘Calcium difference’’ (y-axis) is the difference in
calcium intake by the FFQ minus that by the 24-h recall, while
‘‘calcium mean’’ (x-axis) is the mean of calcium intake by the two
methods. The mean (solid line) and the 95% CI (broken lines) of the
difference are shown
307
the higher the intake, the higher the underestimation.
However, the linear regression of the difference between
the two methods on calcium intake by the 24-h recall
was also significant, albeit of low predictive power
(F=439.4, SEE=278 mg/day, r2=0.305, P<0.0001):
difference in calcium intake (mg/day)=249 (±20)–0.444
(±0.021)·calcium intake by 24-h recall (mg/day). This
equation (coefficients are shown with standard error in
parenthesis) yields a zero value for the difference (i.e. no
underestimation) when calcium intake by the 24-h recall
equals to approximately 561 mg/day. For intakes less
than that, positive values are derived, i.e. overestimation
occurs. The phenomenon of overestimation at low intakes and underestimation at high intakes is a rather
common feature of FFQs, known as the ‘‘flat-slope
syndrome’’ (Fig. 2) [7].
Consequent to that, however, when differences between the two methods were expressed as percentages
(dividing by the 24-h recall intake), relative overestimations at low intakes more than counterbalanced relative underestimations at high intakes. This is illustrated
in Fig. 3, where the percentage difference is plotted
against the 24-h recall intake. Again, a significant negative linear correlation was observed (r=–0.437,
P<0.001), but the regression equation more closely
describing this relationship was an inverse function of
‘‘true’’ intake (F=363.2, SEE=40.8%, r2=0.267,
P<0.0001): Difference in calcium intake (%)=–39.16
(±2.19)+21452 (±1126)/calcium intake by 24-h recall
(mg/day). The zero intercept here is at approximately
548 mg/day. Apparently, therefore, percentage differences could range from +¥ (for intakes close to zero) to
–39.16% (for intakes close to infinite), but not lower.
Fig. 2 The ‘‘flat-slope syndrome’’ of the FFQ. ‘‘Calcium difference’’ (y-axis) is the difference in calcium intake by the FFQ minus
that by the 24-h recall, while ‘‘calcium by 24-h recall’’ (x-axis) is the
calcium intake by the 24-h recall. The linear regression line (solid
line) and the zero line (broken line) are shown. The zero intercept is
at approximately 561 mg/day
Fig. 3 Relative differences as a function of ‘true’ intake. ‘‘Calcium
difference’’ (y-axis) is the difference in calcium intake by the FFQ
minus that by the 24-h recall, expressed as percentage of the latter,
while ‘‘calcium by 24-h recall’’ (x-axis) is the calcium intake by the
24-h recall
Hence at low ‘‘true’’ intakes, e.g. 100 mg/day, overestimation was approximately +175% (corresponding to
+175 mg/day), whereas at high intakes, e.g. 1000 mg/
day, underestimation was only –17.7% (corresponding
to –177 mg/day). While absolute differences would
more or less cancel out (+175–177=–2 mg/day),
relative differences would certainly not (+175–
17.7=+157.3%). Overall, this is why the ratio of the
means (i.e. –133/861=–15.4%) was so much different
from the mean of ratios (i.e. –5.4%). Because the mean
of a distribution that spans zero substantially underestimates the absolute error (i.e. regardless of sign), which
is important when dealing with individuals, mean
absolute differences between the FFQ and the 24-h recall
are also shown in Table 1. Whether expressed in mg/day
or as percentages of ‘‘true’’ intake, absolute differences
between the two methods were significantly different
from zero (P<0.001), indicating the inability of the
FFQ to accurately estimate individual calcium intakes.
Actual values for surrogate FFQ quartiles manifested
a progressive increase with significant differences
between mean calcium intakes (F=158.9, P=0.001;
Table 2). The latter suggests that the FFQ could reliably
distinguish between categories at any level of intake. It is
worth mentioning that, individuals whose calcium intake was overestimated by the FFQ (due to the flat-slope
syndrome referred to above) were more likely to have
been classified into the second quartile, rather than the
first. When five surrogate categories were assigned, the
FFQ lost its discriminatory ability between the second
and the third quintile (P=0.142), but it was still highly
discriminative across all the remaining categories
(F=126.8, P<0.001; Table 2). Following a stepwise
increase in the number of surrogate categories, it was
observed that the FFQ could readily distinguish between
308
Table 2 Discriminatory power of the FFQ. For each FFQ quartile or quintile, ‘‘true’’ values for calcium intake (mean±SD) by the 24-h
recall have been calculated. These latter intakes do not represent 24-h recall quartiles or quintiles
FFQ
quartiles
n
Calcium range
by FFQ (mg/day)
Calcium intake
by 24-h recall
(mg/day)
FFQ
quintiles
n
Calcium range
by FFQ (mg/day)
Calcium intake
by 24-h recall
(mg/day)
1
2
3
4
–
250
251
250
250
–
<473
473–676
676–951
>951
–
576±292a
760±326b
886±343c
1223±397d
–
1
2
3
4
5
200
200
201
200
200
<425
425–600
600–762
762–1025
>1025
551±283a
733±324b
811±334b
938±348c
1272±394d
a,b,c,d
Quartiles or quintiles not sharing the same letter are statistically significantly different from each other at P<0.001 (i.e. they are
discriminated), by one-way ANOVA and Tukey’s post hoc tests. Quintiles 2 and 3 share the same letter and are not different from each
other (i.e. they cannot be discriminated by the FFQ)
has a considerable influence on skeletal health [2], and
its adequate intake from food is of major importance
for preventing osteoporosis and reducing fracture risk
[4]. However, evidence suggests that many individuals
fail to meet their calcium needs [6,32], hence putting
themselves at increased risk. Taking all this information into consideration, and because the consequences
of osteoporosis are multidimensional (i.e. financial,
physical, and psychosocial) and affect both the individual as well as the family and the community [3], a
recommendation was made for developing cost-effective
methods to identify those with insufficient calcium intakes [5].
Along this line, several FFQs for assessing dietary
calcium consumption have been designed and validated
for use, most commonly among adult and elderly women [10,11,12,13,14,15,16,17,18,19,20,21,22,23]. Similar
studies in children are scarce [24,25], while absent in
men. At our present state of knowledge, therefore,
available FFQs for calcium are age- and/or genderspecific, and this may be of concern in large-scale
population-based epidemiological surveys or in clinical
practice, where the need to evaluate dietary calcium
intake of individuals at all ages and both genders
would arise. The present study was designed to fill in
this gap, by developing a calcium FFQ for the general
population, including male and female children, adults,
and elderly subjects. Although results are mainly relevant to the Greek population, as dietary assessment
methods need to be culturally sensitive with respect to
food intake patterns [33], they are also indicative of the
feasibility of developing such a FFQ, besides its practicality, and could be of some use for similar studies in
the future.
the lowest (<500 mg/day) and the highest (>1000 mg/
day) calcium intakes up until ten such categories were
assigned. In this case, the FFQ could discriminate the
first (473±242 mg/day), the six intermediate (second
through seventh), and the three higher (eighth,
1020±343 mg/day, through tenth) intake categories
(F=69.0, P<0.001), but was unable to distinguish the
intermediate ones (i.e. those falling between 600–
900 mg/day) from each other.
To enable comparison with similar studies, sensitivity, specificity, PPV and NPV have been calculated setting the cut-off level for calcium intake at 800 mg/day.
The FFQ had a moderate-to-high sensitivity (82.8%)
and a moderate-to-low specificity (54.9%), while PPV
(64.3%) and NPV (76.6%) were only moderate. Use of
the age-specific adequate intakes for calcium, i.e. 1300,
1000, and 1200 mg/day for children, adults, and the elderly, respectively [31], resulted in increased sensitivity
(95.5%) and PPV (84.0%), but also in decreased specificity (34.1%) and NPV (67.9%). The cross-classification
analysis indicated that only 17 subjects (1.7%) were
grossly misclassified, while 827 subjects (82.6%) were
correctly classified (Table 3).
Discussion
Although osteoporosis was once thought to be a natural part of aging among women, it is now widely
accepted that the disease may affect all individuals
regardless of age or gender [3]. Optimization of bone
health by appropriate dietary and lifestyle practices,
therefore, is a process that must occur throughout the
lifespan in both males and females [3]. Dietary calcium
Table 3 Cross-classification
analysis for the FFQ.
Frequencies and marginal
distributions are shown
24-h recall quartiles
FFQ quartiles
1
2
3
4
Total
1
2
3
4
Total
130 (52.0)
73 (29.1)
44 (17.6)
3 (1.2)
250
66 (26.4)
74 (29.5)
75 (30.0)
36 (14.4)
251
40 (16.0)
67 (26.7)
72 (28.8)
71 (28.4)
250
14 (5.6)
37 (14.7)
59 (23.6)
140 (56.0)
250
250
251
250
250
1001
309
Our findings indicate that the FFQ tended to
underestimate mean calcium intake by approximately –
133 mg/day or –5.4% compared to the 24-h recall (Table 1). Also, at the individual level, the questionnaire
could provide estimates of calcium intake from 533 mg/
day above to 799 mg/day below the 24-h recall, meaning
that its quantitative accuracy for an individual spanned
over approximately 1300 mg/day (Fig. 1). These results
are similar to those obtained for some of the previous
FFQs [15,19,25], but are certainly worse than others,
where no statistically significant differences in mean
calcium intakes derived from the FFQ and the reference
method were observed [10,12,14,16,17,20,21], and the
95% limits of agreement were tighter [17,21]. From the
quantitative perspective, therefore, this FFQ cannot be
considered appropriate for estimating actual calcium
intake of an individual.
On the other hand, the linear correlation between the
two methods (r=0.639, P<0.001) was in the range reported previously (r=0.5–0.8) [11,13,14,18,20,21,23,25],
although in one study, a coefficient equal to 0.9 was
obtained [17]. The FFQ had good discriminatory power
at all levels of calcium intake when quartiles were assigned, but it could not differentiate between the second
and the third quintile (Table 2), hence being more efficient than some [21,25] but less efficient than other [17]
questionnaires in this respect. Still, it maintained a fair
ability to discriminate the lower (<500 mg/day) and the
higher (>1000 mg/day) calcium intakes from each other
and from the intermediate ones (600–900 mg/day), up
until ten surrogate categories were assigned. This, in
combination with the weak tendency to overestimate
calcium intake at low intakes and underestimate it at
high intakes (Fig. 2), strengthens the clinical value of an
intake below 500 mg/day or above 1000 mg/day by the
FFQ, since the former will most likely represent an
overestimation and the latter an underestimation of
‘‘true’’ calcium intake. This property may prove
important, since it is now well established that calcium is
a threshold nutrient, in that benefit and risk are not
linearly related to intake. More specifically, risk of
fractures seems to increase only in those subjects whose
calcium intake is less than 500 mg/day [34], while no
apparent health benefit is observed at intakes above
1000–1500 mg/day [4].
The FFQ demonstrated a reasonable ability to classify individuals into quartiles of calcium intake, with
82.6% of the subjects being correctly classified into the
same or adjacent quartile, and only 1.7% being grossly
misclassified (Table 3). Respective proportions in the
literature vary from 0–1.2% [14,15] to 3.2–3.4% [21,25]
for gross misclassification, and from 81–84.1% [21,25] to
92.9–95.2% [14,15] for correct classification. Sensitivity
in identifying subjects with intakes less than 800 mg/day
was relatively high (82.8%), but specificity in identifying
those with intakes more than 800 mg/day fell short
(54.9%). Using higher cut-off levels for calcium intake,
such as the age-specific adequate intakes, resulted
in increased sensitivity and decreased specificity, in
accordance with previous observations [14]. These values might be expected keeping in mind the tendency of
the FFQ to underestimate calcium intake in general, and
can be considered moderate at best compared with other
questionnaires [14,17,21,25].
Some discussion on the validation method is also
justified. Virtually all of the available techniques have
been used previously as reference tools against which the
various calcium FFQs have been validated, including
modified diet history interviews [10], 4-day weighed
[11,19] or semi-weighed [22] food records, 3-day [14],
4-day [18,25], 7-day [20,21,23], or 14-day [17] estimated
food records, as well as 24-h diet recalls [15,16] and
full-length FFQs [12,13]. Depending on the more or less
burdensome nature of the reference method, sample
sizes have ranged from approximately 20–60
[10,11,15,18,19,23,25] to approximately 100 [13,20,21,22]
subjects, although in some studies greater numbers were
recruited, e.g. more than 200 [14,17] or 500 [12] individuals. Although our choice of the 24-h recall as a
reference tool may not be the best for validation purposes, nevertheless, this method is considered the most
suitable to get population means and distributions for
subjects aged 10 years and over with reasonable accuracy, especially when combined with visual aids for
estimating portion sizes [8]. Thus, the 24-h recall was
preferred over other methods under these circumstances,
as a practical, cost-effective, and fairly accurate technique. Also, because of the large number of subjects
surveyed, inter-individual variability in daily food intake
would be expected to decrease [7].
Another limitation that should be acknowledged is
the lack of information on the reproducibility (reliability) of the FFQ. This is usually assessed by administering the questionnaire at two (or more) points in time to
the same group of people. Generally, however, less than
half of all FFQs being validated are tested for reproducibility [35, 36]. In the case of Ca calcium-specific
questionnaires, we could identify only three studies that
actually did that [12,13,21]. Bearing in mind that the
present FFQ was self-administered, the two major concerns regarding reproducibility, namely, intra- and interrater reliability, would automatically cancel out, as no
interviewer was involved [35]. Therefore, not testing for
reproducibility might not be as much of a problem for
this FFQ as it would, for example, for those questionnaires administered by trained dietitians (e.g. [17]). What
is more, in order to assess reproducibility, one must allow for sufficient time before re-administrating the
questionnaire, so as to avoid a possible memory effect on
the respondents’ behalf. Because the present FFQ was
quite short (30 items), and hence the responses were easy
to remember, the time interval before re-administration
would have to be longer. In this instance, however, true
changes in dietary intake might have occurred (as they
usually do), contributing to falsely low reproducibility
[37]. For all these reasons, we feel that absence of any
information on the reliability of the FFQ is probably not
as critical, though indeed an omission.
310
Fig. 4 The 30-item FFQ developed and used for estimating usual calcium intake in the Greek population
In summary, the present study extended on previous
ones by developing an age- and gender-independent
FFQ for assessing dietary calcium intake in the general
population. Evaluation has been conducted in subjects
with various levels of education, different socioeconomic
statuses, and degrees of cooperation, and not merely in
311
highly motivated individuals, hence strengthening its
applicability in practice. The FFQ could be used in
large-scale epidemiological surveys or in clinical settings
as a rapid method for ranking calcium intake, as well as
for discriminating relatively low and probably insufficient (<500 mg/day) from relatively high and probably
sufficient (>1000 mg/day) calcium intakes. Quantitative
estimations at the individual level, however, should be
treated with prudence.
Appendix A
Figure 4 shows the 30-item FFQ that was developed and
used for estimating usual calcium intake in the Greek
population. The questionnaire was self-administered by
the subjects and then returned for analysis. The final two
questions were included for cross-checking purposes (i.e.
subjects who answered ‘‘yes’’ in either one were excluded
from subsequent analysis; see Materials and methods).
Appendix B
Table 4 lists the calcium content of each FFQ item.
Table 4 Spreadsheet showing
the calcium content of each
FFQ item
Acknowledgements This study was part of a nationwide project on
osteoporosis, supported by Friesland Hellas. The authors would
like to thank Maria Bletsa, Maria Rammata, and Anastasia
Doulgeri, dietitians, Silia Sidossis, research assistant, and Antigoni
Tsiafitsa, technician, for their valuable help in data collection and
processing.
References
1. Nordin BE (1997) Calcium and osteoporosis. Nutrition 13:664–
686
2. European Commission (1998) Report on Osteoporosis in the
European Community: action for prevention. Office for Official
Publications for the European Commission, Luxembourg
3. National Institutes of Health (2000) Osteoporosis prevention,
diagnosis, and therapy. NIH Consensus Statement 17:1–45
4. Heaney RP (2002) The importance of calcium intake for lifelong skeletal health. Calcif Tissue Int 70:70–73
5. National Institutes of Health (1994) Optimal calcium intake.
NIH Consensus Statement 12:1–31
6. Weaver CM (2000) The growing years and prevention of
osteoporosis in later life. Proc Nutr Soc 59:303–306
7. Magkos F, Yannakoulia M (2003) Methodology of dietary
assessment in athletes: concepts and pitfalls. Curr Opin Clin
Nutr Metab Care 6:539–549
8. Biro G, Hulshof KF, Ovesen L, Amorim Cruz JA (2002)
Selection of methodology to assess food intake. Eur J Clin Nutr
56:S25–32
Food item
FFQ quantity
Weight equivalent
(g)
Calcium content
(mg)
Milk
Yogurt
Feta cheese
Graviera cheese
Kaseri cheese
Mozzarella
Emmenthal, cheddar, etc.
Parmesan or kefalotiri cheese
Anthotiro cheese
Mashed cheese
Cheese pie
Cream pie
Leafy vegetables pie
Spinach pie with cheese
Bread and similar
Cereals
Peanuts or almonds
Other nuts
Spinach
Salad vegetables
Green vegetables
Potatoes
Legumes
Sardines
Scallops
Shrimps
White fish or salmon
Eggs
Ice cream
Chocolate
1 glass
1 pot
1 matchbox
1 matchbox
1 matchbox
1 matchbox
1 matchbox
1 tablespoon
1 tablespoon
1 tablespoon
1 serving
1 serving
1 serving
1 serving
1 slice or 1 piece
1/2 cup
1 handful
1 handful
1 cup
1/2 cup
1/2 cup
1 medium
1 cup
10 small
1 serving
1 serving
1 serving
1 egg
2 scoops
5 squares
250
225
20
20
20
20
20
15
20
15
100
100
100
100
30
100
25
25
240
120
120
100
240
60
100
100
100
40
100
25
250
280
65
180
150
88
160
120
50
56
300
150
50
200
25
20
60
30
216
60
60
10
96
200
100
200
70
25
151
50
312
9. Basiotis PP, Welsh SO, Cronin FJ, Kelsay JL, Mertz W (1987)
Number of days of food intake records required to estimate
individual and group nutrient intakes with defined confidence. J
Nutr 117:1638–1641
10. Angbratt M, Moller M (1999) Questionnaire about calcium
intake: can we trust the answers? Osteoporos Int 9:220–225
11. Angus RM, Sambrook PN, Pocock NA, Eisman JA (1989) A
simple method for assessing calcium intake in Caucasian women. J Am Diet Assoc 89:209–214
12. Blalock SJ, Currey SS, DeVellis RF, Anderson JJ, Gold DT,
Dooley MA (1998) Using a short food frequency questionnaire
to estimate dietary calcium consumption: a tool for patient
education. Arthr Care Res 11:479–484
13. Brown JL, Griebler R (1993) Reliability of a short and long
version of the Block food frequency form for assessing changes
in calcium intake. J Am Diet Assoc 93:784–789
14. Chee WS, Suriah AR, Zaitun Y, Chan SP, Yap SL, Chan YM
(2002) Dietary calcium intake in postmenopausal Malaysian
women: comparison between the food frequency questionnaire
and three-day food records. Asia Pac J Clin Nutr 11:142–146
15. Green JH, Booth CL, Bunning RL (2002) Assessment of a
rapid method for assessing adequacy of calcium intake. Asia
Pac J Clin Nutr 11:147–150
16. Haines CJ, Chung TK, Leung PC, Leung DH, Wong MY, Lam
LL (1994) Dietary calcium intake in postmenopausal Chinese
women. Eur J Clin Nutr 48:591–594
17. Montomoli M, Gonnelli S, Giacchi M, Mattei R, Cuda C,
Rossi S, Gennari C (2002) Validation of a food frequency
questionnaire for nutritional calcium intake assessment in
Italian women. Eur J Clin Nutr 56:21–30
18. Musgrave KO, Giambalvo L, Leclerc HL, Cook RA, Rosen CJ
(1989) Validation of a quantitative food frequency questionnaire for rapid assessment of dietary calcium intake. J Am Diet
Assoc 89:1484–1488
19. Pasco JA, Sanders KM, Henry MJ, Nicholson GC, Seeman E,
Kotowicz MA (2000) Calcium intakes among Australian women: Geelong Osteoporosis Study. Aust N Z J Med 30:21–27
20. Taitano RT, Novotny R, Davis JW, Ross PD, Wasnich RD
(1995) Validity of a food frequency questionnaire for estimating calcium intake among Japanese and white women. J Am
Diet Assoc 95:804–806
21. Wilson P, Horwath C (1996) Validation of a short food frequency questionnaire for assessment of dietary calcium intake
in women. Eur J Clin Nutr 50:220–228
22. Xu L, Porteous JE, Phillips MR, Zheng S (2000) Development
and validation of a calcium intake questionnaire for postmenopausal women in China. Ann Epidemiol 10:169–175
23. Cummings SR, Block G, McHenry K, Baron RB (1987)
Evaluation of two food frequency methods of measuring dietary calcium intake. Am J Epidemiol 126:796–802
24. Bellu R, Riva E, Ortisi MT, De Notaris R, Santini I, Banderali
G, Giovannini M (1995) Calcium intakes in a sample of 35,000
Italian schoolchildren. J Int Med Res 23:191–199
25. Taylor RW, Goulding A (1998) Validation of a short food
frequency questionnaire to assess calcium intake in children
aged 3 to 6 years. Eur J Clin Nutr 52:464–465
26. Magkos F, Manios Y, Babaroutsi E, Sidossis LS (2004)
Quantitative ultrasound calcaneus measurements: normative
data for the Greek population. Osteoporos Int (in press)
27. Katsouyanni K, Rimm EB, Gnardellis C, Trichopoulos D,
Polychronopoulos E, Trichopoulou A (1997) Reproducibility
and relative validity of an extensive semi-quantitative food
frequency questionnaire using dietary records and biochemical
markers among Greek schoolteachers. Int J Epidemiol
26:S118–127
28. Thompson FE, Byers T (1994) Dietary assessment resource
manual. J Nutr 124:2245S–317S
29. Trichopoulou A (1992) Food composition tables and composition of Greek cooked food and dishes. School of Public
Health, Athens
30. Bland JM, Altman DG (1986) Statistical methods for assessing
agreement between two methods of clinical measurement.
Lancet 1:307–310
31. Institute of Medicine (1997) Dietary reference intakes for calcium, phosphorus, magnesium, vitamin D, and fluoride. National Academy Press, Washington D.C.
32. Nicklas TA (2003) Calcium intake trends and health consequences from childhood through adulthood. J Am Coll Nutr
22:340–356
33. Hankin JH, Wilkens LR (1994) Development and validation of
dietary assessment methods for culturally diverse populations.
Am J Clin Nutr 59:198S–200S
34. Lau EM, Cooper C (2001) Risk factors for osteoporosis in
Europe. J Bone Miner Metab 19:142–145
35. Cade J, Thompson R, Burley V, Warm D (2002) Development,
validation and utilisation of food-frequency questionnaires—a
review. Public Health Nutr 5:567–587
36. Cade JE, Burley VJ, Warm DL, Thompson RL, Margetts BM
(2004) Food-frequency questionnaires: a review of their design,
validation and utilisation. Nutr Res Rev 17:5–23
37. Tsubono Y, Nishino Y, Fukao A, Hisamichi S, Tsugane S
(1995) Temporal change in the reproducibility of a selfadministered food frequency questionnaire. Am J Epidemiol
142:1231–1235
