The first thing that should be noted is that this was a "retrospective" study. In other words the authors looked back at data that had already been collected between 2005 and 2010. Now this isn't necessarily a bad thing, but inevitably in these situations there are details that one would want to know (for example in this case details about what the extra office visits and calls observed in the group who accessed the PHR during the study period were for) that just aren't available. So in this case although the researchers demonstrated that the group who had access to their own medical information via the Kaiser PHR appeared to utilize more services we have very little idea what the focus of this additional utilization was. It is completely feasible that this group had a better understanding of their own health by virtue of having access to their records and were therefore more proactive in seeking advice early around preventive health and early diagnosis of symptoms.
The second point is that because of its retrospective design the research did not have a control group of similar individuals who did not have access to the PHR to compare to. Because the Kaiser integrated delivery system in Colorado is large the researchers were able to look at healthcare utilization data from individuals who did not access the PHR system, however this group were significantly different in a number of demographic attributes to the individuals who accessed the PHR during the study period. The non-users were younger, more likely to be male and had fewer chronic diseases that PHR users. The researchers get around this by using a statistical technique called "propensity scoring". In essence this matches individuals in both groups based on particular characteristics. Now as far as I can see from reading the paper, the only characteristics used to do this propensity scoring were the year(s) of data collection and the historical number of baseline office visits individuals made. Although this approach allows for a better comparison between the two groups you cannot by any means say that the two groups were the same. This fact is demonstrated by looking at baseline ER visits between the groups; the group that did not access the PHR had almost 33% higher ER utilization that those that did, and this suggests that their approach to their own health and how they access services was fundamentally different from individuals who engaged with the PHR.
My final point is that even if one accepts that engagement with a PHR somehow drives greater utilization of services how does this translate in terms of patient outcomes? Unfortunately this data was not available to the researchers, but it is one that needs to be understood. In the brave new world of cost containment and outcome demonstration we have to understand whether the extra half an office visit per year per PHR user shown in this research study delivers value (the definition of which is cost divided by outcome) to the healthcare system as a whole.
Don't get me wrong, this article is not an attack on the authors or the research methodologies used in this piece of research. We need to better understand not only how to engage patients in an ongoing dialog about their health, but also what impact that increased engagement has on service delivery. This paper has framed some of those questions, and although not perfect has provided rich food for thought and discussion. It should not, however, tempt us to abandon driving better patient engagement, nor using technology to facilitate this.
