Personally I'm less worried about outright-fake data than sloppiness. The article mentions Brian Wanskink; AFAIK he never deliberately invented anything but he did a whole bunch of p-hacking and his lab was so sloppy that data got mislabeled (one study allegedly done on 8-11 year olds was actually done on pre-schoolers). He was "caught" when he published a blog post[2] giving advice to young scientists and it went viral. Clearly no misconduct intended.
Most of this garbage research takes place in domains that don't matter, where people are hardly taking the results seriously anyway (see also power posing). Typically when somebody actually cares about the truth of a result, they kick the tires and vet the result pretty thoroughly. The fraudulent LeCour and Green study from a few years ago was exposed by Brookman and Kalla, who were attempting a related study (rather than being science "vigilantes").
But not always. Clearly people acting on Gino's bogus research. The Reinhart-Rogoff paper [0] was discussed globally, and may have actually influenced fiscal policy. They used Excel for analysis, made a click-and-drag mistake, and improperly excluded a couple datapoints. It appears this exclusion was accidental [1]. Nevertheless, including those points changes the conclusion.
Catching this error took 3 years. It probably would've been caught faster if they had published their data alongside the paper, although apparently they actually did provide it upon request, so if people checked these things more frequently it would've been caught earlier.
[1] "A coding error in the RR working spreadsheet entirely excludes five countries, Australia, Austria, Belgium, Canada, and Denmark, from the analysis.5 The omitted countries are selected alphabetically and, hence, likely randomly with respect to economic relationships." http://peri.umass.edu/fileadmin/pdf/working_papers/working_p...
One of the problems is that if you run more studies, you are statistically more likely to eventually yield a statistically significant result. The tests that folks run are not designed for series of statistical tests, but for individual tests. So when you run a ton of experiments and throw away all the statistically insignificant results, you are engaging in the form of p-hacking that the discipline of psychology is currently struggling with. The overarching replication crisis.
And you may be thinking that's all harmless, but it's really not. Folks running experiments are competing for jobs with other folks who might do more worthwhile research. Because psychologists are good at running the pipeline, and they have found a nearly inexhaustible source of publications (experiments run on undergrad students), the up the bar for everyone else.
I remember just a few days ago a discussion on this site about how a hiring committee skipped over the "better" applicant with more publications and citations than the successful candidate. The reality is that there are plenty of faculty on hiring committees out there who are looking for candidates with a good publication record, because that is how they assess if someone is a good researcher. Psychologists have already succeeded in pushing out most of the less experiment-prone folks in their discipline. The pressure extends to adjacent fields.
Most of this garbage research takes place in domains that don't matter, where people are hardly taking the results seriously anyway (see also power posing). Typically when somebody actually cares about the truth of a result, they kick the tires and vet the result pretty thoroughly. The fraudulent LeCour and Green study from a few years ago was exposed by Brookman and Kalla, who were attempting a related study (rather than being science "vigilantes").
But not always. Clearly people acting on Gino's bogus research. The Reinhart-Rogoff paper [0] was discussed globally, and may have actually influenced fiscal policy. They used Excel for analysis, made a click-and-drag mistake, and improperly excluded a couple datapoints. It appears this exclusion was accidental [1]. Nevertheless, including those points changes the conclusion.
Catching this error took 3 years. It probably would've been caught faster if they had published their data alongside the paper, although apparently they actually did provide it upon request, so if people checked these things more frequently it would've been caught earlier.
[0] https://en.wikipedia.org/wiki/Growth_in_a_Time_of_Debt
[1] "A coding error in the RR working spreadsheet entirely excludes five countries, Australia, Austria, Belgium, Canada, and Denmark, from the analysis.5 The omitted countries are selected alphabetically and, hence, likely randomly with respect to economic relationships." http://peri.umass.edu/fileadmin/pdf/working_papers/working_p...
[2] https://web.archive.org/web/20170312041524/http:/www.brianwa...