Dr. Steve Rein's Comments on Statistics

regarding Dr. Bucknam's reply to Aney in the AAP News

Having, at last, the opportunity to read Dr. Bucknam's comments in the AAP News on the whole Prep/Babywise controversy, I have a few comments in reply. As much of his reply concerned some statistical comparisons of PDF babies with others, I feel as though my comments will be on solid ground as I have a PhD in Statistics (but, alas, lack formal training in Theology, Lactation and Medicine).

Non-Statistical Comments

• I am sort of shocked that Bucknam claims to be surprised by this controversy. After all, he's been interviewed by reporters on this issue and he most assuredly knows that Growing Families International (GFI), the company that promotes the use of Babywise and Prep has a group of volunteer "contact moms" who reportedly have helped people deal with FTT babies.

• I am pleased that Bucknam points out that they are not opposed to demand feeding but I am a bit surprised as well. After all, his co-author Gary Ezzo once wrote in an early edition of Preparation for Parenting (the decidedly religious precursor to "Babywise"):
  
  ``Working from a biblical mindset and practicing demand-feeding can never be harmonized since the two are incompatible philosophies.''
  
  in his attempt to cast demand feeding in a negative light. Interestingly enough, Ezzo wrote this in the "Prep" materials that Bucknam most likely used himself back around the beginning of the '90s. I guess Bucknam doesn't read very carefully what his co-author writes.
  
  Even in the first edition of Babywise which Bucknam co-authored, demand feeding is roundly castigated. I suspect that Bucknam and Ezzo were simply unaware of what constituted the practice known as demand feeding. Once the AAP released their statement supporting demand feeding and once they re-affirmed that scheduling is suspect, at best, Bucknam and Ezzo realized that what they had been criticizing and calling by the name demand feeding has no relation to demand feeding as defined by the AAP, Lactation Consultants and Le Leche League (LLL - an organization criticized by Ezzo for it's promotion of demand feeding) and practiced by millions throughout the world. (I do need to wonder how a pediatrician, an author of a book on infant care at that, could not know the meaning of demand feeding.)

Statistical Comments

First off, Bucknam's piece is a good example of the sorts of articles I like to bring into my freshman introduction to statistics class. It is brief yet offers even introductory students an opportunity to critique many issues at hand. I thank Bucknam for making his comments accessible in this way. Some issues that I would hope students would identify in Bucknam's article include:

• Peer Review?

  
  None of the data Bucknam cites on PDF infants has been published. We should be reluctant to accept as fact any statistics which haven't been through the peer review process.
  
  

• FTT and dehydration?

  
  The only evidence that Bucknam presents that PDF does not cause FTT and dehydration (something that Aney claims may be the case) is that because of PDF principles he hasn't observed any cases of either in the more than 2000 babies in his practice. I find this incredible.

  Let's assume for the moment that each FTT and dehydration occur in about 1% of the population (Note: I have put out a call to get some authoritative numbers on rates of dehydration and FTT and until I "get the goods", my comments assume for the purpose of some quick calculations a guess at these rates, 1%, by Jan Barger, RN, MA, IBCLC. Barger's guess at the rates appears to be fairly good as another IBCLC and two pediatricians who are internationally recognized breastfeeding experts agreed that 1% is, if anything, a bit low. When, and if, new numbers come in, I will revise my comments accordingly.) Then, Bucknam should be see in his 2000 patients about 20 cases of each and it is impossible that he saw zero either. If the use of PDF on Bucknam's patients caused their chance of each FTT and dehydration to be one half that of the population as a whole, he should still see about 10 cases of each and the chance that he would see zero of either is about 1 in 500 million (this assumes FTT and dehydration occur roughly independently in patients which, although not literally true, due to the way the cases would present themselves is approximately true). If PDF cut the rates of both FTT and dehydration by 80%, the chance that he would see zero cases of either is still large, about 1 in 3000. This means that we can only conclude one of several things about Bucknam's practice based on his claim of zero cases in 2000 patients:

  • Either he isn't correctly diagnosing true cases of FTT and dehydration in his patients. (Which doesn't fill me with confidence in his medical skills.)
    
    
  • Or his PDF method is helpful at reducing the FTT and dehydration rates and he is luckier than the person who wins the Virginia Lotto drawing three consecutive weeks.
    
    
  • Or his PDF method is amazingly helpful and he is still lucky. If his method is good enough to reduce the FTT and dehydration rates by 80% it needs to be seriously investigated as it may be one of those miracle breakthroughs that, at little cost to healthcare providers and insurance companies virtually eliminates some costly medical conditions.
    
    
  • Or, he may have hired a Lactation Consultant who checks in with new mothers in his practice on a daily basis during the early weeks and weekly thereafter to catch FTT and dehydration before they occur. Of course, this is not a cheap solution, but it is a good one. On the other hand, the lack of FTT and dehydration in his practice would, in this case, more likely be due to the intervention of a LC than anything else, including PDF.
    
    
  • Or, what I think is most likely, Bucknam meant to write that he hasn't seen a FTT or dehydration case that he would attribute to the use of his PDF feeding schedule. (To be very honest, this is what I believe he intended to say but it is not what he did write.) No surprise here. Without controls it is remarkably easy to explain away any negative results. Such is the case in medical research. In observational studies without controls, new treatments often look remarkable but upon further investigation appear to be less than impressive. It is very easy (and tempting) for physicians to explain away all the failures as having not been caused by their treatment. This is why we require a more thorough analysis before accepting any particular medical practice as either safe or effective.
    

  In any case, based on the data from Bucknam's practice, PDF appears to be worth the serious consideration of the AAP. An experiment or perhaps a retrospective study may help shed some light on whether PDF is a savior, simply safe or a seducer. Barring a costly study, the issue can be approached on theoretical grounds by the AAP: "Does PDF match up with or go against the current medical knowledge?".

• Healthy Sleep?

  Bucknam mentions a sample of 520 babies, 97 percent of which sleep through the night by 12 weeks.

  • How were these 520 babies sampled? If they were chosen because they slept through the night early it should be no surprise that they slept through the night early.
    
    
  • Even if the sample had been representative of those who use PDF, we cannot truly say that we know that PDF helps sleep at night without a control group.
    
    
  • To suggest that healthy infants need to sleep thru the night (7 to 8 hours) or that ten to eleven hours of sleep per night is a "success" is an assertion with no justification.
    
    Also, I wonder how the length of sleep of the infant was measured. If the information was obtained from a questionnaire, we don't know that, in fact, these infants have simply learned that their nighttime cries will be ignored and so don't bother to indicate their wakefulness (or hunger) by crying. Would that situation be a considered a success?
    
    
  • Even if PDF did help infants sleep through the night, that doesn't mean that PDF is a good thing to use. There may be grave drawbacks to its use, for example if FTT occurs in just 2% of PDF using babies (doubling the risk of a serious medical condition), an 97% success rate at nighttime sleep would be far less tempting to parents.
    
    

• Successful Breastfeeding?

  
  Bucknam mentions a study of 240 moms who use PDF and notes that 70 percent of them breastfeed (not necessarily exclusively) into the sixth month and compares that to data from Pediatrics which shows that across the US only about 20 percent of mothers breastfeed into the sixth month.
  
  
  • The sample of PDF babies was one of convenience. Such a sampling scheme brings with it the potential for bias. As an example, Bucknam points out that in his convenient sample, some 70% of moms breastfeed for at least 6 months. I would suggest that at a typical LLL conference would contain at least 95% mothers who have nursed their infants past the 6 month mark. The conclusion: either LeLecheLeague is better than GFI or convenience samples give data of little value or both. Perhaps if Bucknam had been more clear about how the convenient sample had been obtained we could have a better idea of how representative the sample was of the population of PDF moms.
    
    
  • The sample wasn't just one of convenience from the population of those who had read Babywise or been thru a Prep class. It was a sample of those who had successfully introduced the PDF schedule. We have no information about outcomes in infants who's mothers had attempted to use the PDF schedule but didn't meet Bucknam's definition of "following the PDF method". This may be an unfortunate oversight, but I think that physicians and parents care about all those who attempt to use the method, not just those who use it and succeed. After all, if we only look at success stories and refuse to consider failures, we might think that the bloodletting is the best cure for headaches. Hyperbole, sure, but it certainly does make the point.
    
    
  • Comparing a group of committed PDF followers to the general population in terms of breastfeeding rates is the apples versus oranges problem. Maybe if we could find a group of individuals who felt that feeding on demand was "God's Way" we could fairly assess the impact of PDF versus this other program. (Perhaps religious LLL members who feel that breastfeeding on demand is what God intended?)
    
    
  • Bucknam nowhere defines "successful" breastfeeding but implies that at least some breastfeeding into the sixth month of life is a success. Might I remind him of the recent AAP statement which recommends that infants should be exclusively breastfeed into the 6th month.
    
    Again, without a comparable control group, such figures as the 70% Bucknam cites are essentially meaningless.
    
    
  • Even if a fact, 70% of PDF mother breastfeeding at 6 months of age doesn't speak to the issue Dr. Aney raised. Namely, FTT. Many physicians and parents would also like to know of the risks before simply adopting such a method. I suspect in a case such as this one, if parents knew the true success and failure rates of the PDF method and of alternative methods (such as demand feeding), they would choose thoughtfully. I would again encourage Dr. Bucknam and Mr. Ezzo to allow lactation professionals to do a prospective study comparing the health outcomes of the infants who are subjected to their method and those who are subjected to demand feeding.
    
    

• Excellent Weight Gain?

  Bucknam also cites an internal study which compares the weight gain of 200 PDF and 200 demand fed babies noting no significant differences between the two groups.
  
  
  • Presumably Bucknam finds it significant that there is no significant difference between PDF and demand fed infants. Why, then, write a book critical of demand feeding? If PDF and demand fed kids are essentially the same, why bother with a schedule at all? (Of course, with a larger sample, we may, indeed, see statistically significant differences between the two groups. But no statistically significant difference in this moderately large sample means that even if there was a statistically significant difference between the two groups in a larger sampling, the difference we would not observe would not be clinically important.)
    
    
  • Again, how were the 400 infants selected. The point may be a bit belabored by now, but unless a clear protocol is presented and passed through peer-review, we have absolutely no idea that the two groups are comparable at all.
    
    
  • Even if PDF infants had gained weight better than demand fed (note: I'm still not quite sure which definition of demand feeding Bucknam is using, the one in the first edition of his book which reads something like "feed the baby at every cry but only then" or the one from the AAP that he recently said he agrees with) infants in a well designed study (which we don't know we have), we still wouldn't know that PDF is what helps the infants gain weight. One would need an experiment to determine this. Simply put, PDF followers may be of high socioeconomic status, highly motivated to "follow the rules" and to "do it right" and they may have a solid support network while the typical parent who claims to follow demand feeding in these four practices may have been poor and undereducated, not be as motivated to follow through on all aspects of their chosen method and entirely lacking in a support network. Like in the last two studies, a comparable control group, one with similar motivation as the PDF parents would provide us better information than the one we have here.
    
    
  • One of the reasons that the two groups may have had similar weight gain patterns, even if PDF were inferior to demand feeding is that the PDF infants, as a group, may very well have formula suplimentation in far higher rates than the demand group. This would not be evidence of successful breastfeeding, but it would increase the typical calorie intake of infants who are not getting enough breastmilk on the PDF schedule.

  As a parting note, I would like to revisit Bucknam's comment that "there is nothing more rewarding than helping new and expectant parents." I would like to point out that we still have been offered no information that Bucknam's PDF method is actually helping anyone, parents or children. Bucknam is only of the opinion that his method is helpful. A serious investigation of the matter by the AAP would help sort this out. I would hope that Bucknam and Ezzo would join me and others in asking the AAP to study their method and determine if, indeed, it delivers what it promises.