https://www.chronicle.com/article/As-Cornell-Finds-Him-Guilty-of/244584

When I was doing my Doctoral research, I went to a highly respected quantitative geneticist to talk about my research. I was asking which statistical model I should use for my research. He matter-of-factly said "use the model that shows the results you want". My jaw hit the floor!! I mean, this guy wrote the book on statistical models for multiple gene traits. Yet, now my research and all before looks like a sham.

I would bring my results to my adviser. In order to fit the probability of 0.05 in order to make my research valid, I was made to throw out any outliers or data that skewed the results. I think this was valid in many cases because I know all the methods used and where errors happen. But some of the outliers were actually that and should have been kept in.

I remember a story of one graduate student of a faculty member on my dissertation committee. This student basically didn’t do any work. He used a randomizer and created a fake data set without doing any work. The data was too perfect and when the committee started talking with the student, he couldn’t explain his methods.

To be honest, most senior researchers I interacted with didn’t know a lick of statistics. Their graduate students who were deep in the mud in statistics were the ones who knew what they were talking about. My adviser only stuck to one model that he was familiar with. If we strayed to try a different technique, we were ushered back on to the straight and narrow because he didn’t understand it.

And worse yet, the media who publishes and attempts to interpret scientific results follows the narrative of what they want to believe and present. You should listen to Gary Taubes on the "Pursuing Health Podcast" with Julie Foucher. He talks about his interactions with Nobel Prize winning scientists and debunks their methods and statistics. He says you need to be an extreme skeptic if not cynic when reading most research. This is especially true with nutrition and health. Faulty statistics run rampant in these realms. "Morning is the most important meal", "a bran muffin a day keeps the doctor away", "saturated fat is bad for you", all are myths from faulty research. Every day you hear new research that completely backtracks on things like saturated fat, cholesterol, and salt.

As a scientist, I would always recommend being a skeptic first before changing your life. This is especially true with the Pop Sugar and other magazines that try to give you a fresh Tabata Interval Workout when they don’t know the first thing about Dr. Izumi Tabata and his research. I look at their workouts and scratch my head. They don’t get it at all. Always be critical of what you hear and read. Use common sense. And base what you know on real science.