On "healthy choices"

This week, I received an email describing new nutritional recommendations at one of the campus eateries:

Please come by O's Campus Cafe over the next few weeks to see how they are making healthy eating easier!
O's Campus Cafe has analyzed their menu and labeled their foods in order to make it easier for you to make the healthiest choices.  All foods have been given a red, yellow, or green rating based on how nutrient dense they are. Foods that have received a "Red" rating have very little positive nutrient content, are higher in unhealthy nutrients, and should be consumed minimally.  Foods that have received a "Yellow" rating have some positive nutrient content, some unhealthy nutrients, and should be consumed less often.  Foods that have received a "Green" are high in positive nutrient content, have little to no unhealthy nutrients, and can be consumed frequently.  The healthy nutrients evaluated in these developing these ratings are protein, fiber, unsaturated fat, vitamins A, C, E, B-12, thiamin, riboflavin, and folate, and minerals calcium, magnesium, iron, and potassium.  The nutrients considered unhealthy are saturated fat, cholesterol, and sodium. 

I would hope that guidelines regarding "healthy" choices at a major research university would be backed up with evidence, but it turns out that they were simply based on an off-the-shelf rating system (the Nutrient Rich Food Index). To see whether they are supported by actual evidence, I used PubMed to look for relevant results from randomized controlled trials; I avoided epidemiological studies given all of the difficulties in making solid conclusions from nutritional epidemiology research (e.g. http://www.nytimes.com/2007/09/16/magazine/16epidemiology-t.html). Here is what I found regarding the supposedly "unhealthy" nutrients:

• Reduction of saturated fat intake was associated with reduced cardiovascular events but not with any changes in all-cause or cardiovascular mortality (Hooper et al., 2012). A nice review of lipid metabolism by Lands (2008) highlights how the search for mechanisms by which saturated fats might be harmful has failed: "Fifty years later [after starting his research in the 1960's], I still cannot cite a definite mechanism or mediator by which saturated fat is shown to kill people."

• A review of the effects of dietary sodium reduction in randomized controlled trials showed "no strong evidence of any effect of salt reduction" on all-cause mortality (Taylor et al., 2011).

• I searched deeply on PubMed but was unable to find any systematic reviews of the effects of dietary cholesterol reduction on mortality (please let me know if you know of any); there are lots of studies looking at cholesterol reduction using statins, but that doesn't really inform the question of dietary cholesterol reduction.  However, given that serum cholesterol levels bear very little relation to dietary cholesterol intake (Gertler et al., 1950), the labeling of dietary cholesterol as "unhealthy" seems unsupported by evidence. 

The first two reviews listed above come from the Cochrane Collaboration, which is generally viewed as a highly objective source for systematic reviews in evidence-based medicine.

What about the supposed "healthy" choices?

I also looked for data relevant to the the health benefits of the supposedly healthy choices:

• A recent report from a randomized controlled trial showed that replacement of saturated fats with omega-6 linoleic acid (an unsaturated, and thus "healthy" fat) was associated with increased, rather than decreased all-cause mortality (Ramsden et al., 2013). This makes sense given the association of omega-6 fatty acids with inflammation (e.g. Lands, 2012)  and the powerful role that inflammation plays in many diseases.

• Increased fiber intake in the DART (diet and reinfarction trial) of individuals who had already suffered a heart attack was associated with no effects on all-cause mortality after 10 years.(Ness et al., 2002).  An earlier Cochrane review also showed no effects of fiber intake on incidence or recurrence of colon cancer (Asano & McLeod, 2002).

I don't claim to be an expert in this domain, but the best evidence available seems to point towards the lack of any scientific validity of these "health" labels.  

Isn't it better than nothing?

One might ask whether it's still better to have some guidance that pushes people to try to eat more healthily, even if it's not supported by science.  If the recommendations were made with full disclosure of the lack of scientific evidence (ala the disclaimer on dietary supplements) then I would be ok with that. Instead, the recommendations are made in the context of a "nutritionism" that pretends that we will be healthy if we just get the right mix of healthy nutrients and avoid the bad ones, and also pretends to know which are which.  I personally would rather have no advice than unsupported pseudoadvice.

At the same time, it's clear from the obesity epidemic that people need help in making better food choices, so what should we do instead?  I would propose implementing a simple rating system based the first of Michael Pollan's food rules: "Eat food".  Anything that has been freshly picked, harvested, or slaughtered would get a "healthy rating" and anything that comes from a box, can, or bag would get an "unhealthy" rating. It would be much easier than computing micronutrient values, and I challenge anyone to show that it would be any less objectively healthy than the proposed "healthy choices" which would label a fresh grass-fed steak as less healthy than a pile of french fries fried in corn oil.

How well can we predict future criminal acts from fMRI data?

A paper recently published in PNAS by Aharoni et al. entitled "Neuroprediction of future arrest" has claimed to demonstrate that future criminal acts can be predicted using fMRI data.  In the study, the group performed fMRI on 96 individuals who had previously been incarcerated, using a go/no-go task.  They then followed up the individuals (up to four years after release) and recorded whether they had been rearrested.  A survival model was used to model the likelihood of being re-arrested, which showed that activation in the dorsal anterior cingulate cortex (dACC) during the go/no-go task was associated with rearrest, such that individuals with higher levels of dACC activity during the task were less likely to be rearrested.  This fits with the idea that the dACC is involved in cognitive control, and that cognitive control is important for controlling impulses that might land one back in jail.  For example, using a median split of dACC activity, they found that the upper half had a rearrest rate of 46% while the lower half had a rearrest rate of 60%.  Survival models also showed that dACC was the only variable amongst a number tested that had a significant relation to rearrest.

This is a very impressive study, made even more so by the fact that the authors released the data for the tested variables (in spreadsheet form) with the paper.  However, there is one critical shortcoming to the analyses reported in the paper, which is that they do not examine out-of-sample predictive accuracy.  As I have pointed out recently, statistical relationships within a sample generally provide an overly optimistic estimate of the ability to generalize to new samples.  In order to be able to claim that one can "predict" in a real-world sense, one has to validate the predictive accuracy of the technique on out-of-sample data.

With the help of Jeanette Mumford (my local statistical guru), I took the data from the Aharoni paper and examined the ability to predict rearrest on out-of-sample data using crossvalidation; the code and data for this analysis are available at https://github.com/poldrack/criminalprediction.  The proper way to model the data is using a survival model that can deal with censored observations (since subjects differed in how long they were followed).  We did this in R using the Cox regression model from the R rms library.  We replicated the reported finding of a significant effect of dACC activation on rearrest in the Cox model, with parameter estimates matching those reported in the paper, suggesting to me that we had correctly replicated their analysis.  

We examined predictive accuracy using the pec library for R, which generates out-of-sample prediction error curves for survival models.  We used 10-fold crossvalidation to estimate the prediction error, and ran this 100 times to assess the variability of the prediction error estimates. The figure below shows the prediction error as a function of time for the reference model (which simply estimates a single survival curve for the whole group) in black, and the model including dACC activation as a predictor in green; the thick lines represent the mean prediction error across the 100 crossvalidation runs, and the light lines represent the curve for each individual run.  

This analysis shows that there is a slight benefit to out-of-sample prediction of future rearrest using dACC activation, particularly in the period from 20 to 48 months after release.  However, this added prediction ability is exceedingly small; if we take the integrated Brier score across the period of 0-48 months, which is a metric for assessment of probabilistic predictions (taking the value of 0 for perfect predictions and 1 for completely inaccurate predictions), we see that the score for the reference model is 0.214 and the score for the model with dACC as a predictor is 0.207. We found slightly improved prediction (integrated Brier score of 0.203) if we also added Age alongside dACC as a predictor.  

The take-away message from this analysis is that fMRI can indeed provide information relevant to whether an individual will be rearrested for a crime.  However, this added predictability is exceedingly small, and we don't know whether there are other (unmeasured) demographic or behavioral measures that might provide similar predictive power.  In addition, these analyses highlight the importance of using out-of-sample prediction analyses whenever one makes a claim about the predictive ability of neuroimaging data for any outcome.  We are currently preparing a manuscript that will address the issue of "neuroprediction" in greater detail.