> Journalists must get quotes from other experts before publishing
No, this isn't enough. This whole way of thinking isn't enough. It's a big part of the reason for the current situation.
Journalists should report what's true, not what Tom, Dick or Harry said. If a journalist isn't qualified to make object-level claims on a given topic, don't write on that topic. For example, if Bob says there's a forest fire, then instead of publishing "Bob says there's a forest fire", you must do enough legwork to tell your readers "There's a forest fire" or "There isn't".
I allow myself to ignore all journalism that don't follow that guideline, and it makes me happier.
- 3 million people lost their jobs <citation link>.
- 3 million people lost their jobs, according to <expert>
- 3 million people lost their jobs, according to <expert>, while <another expert> estimates as many as 5 million during the same time frame.
Which one of these is the best framing of "the truth?" Because rarely is something worth reporting on some axiomatic statement of fact. Not only because boolean states don't normally exist - they're not compelling.
A journalist's job isn't just to tell you something happened. But to give you understanding and context, and make it compelling. What you're asking for is Wikipedia, not the news.
Um, so if you want to publish an article on how many people will die of the virus, how do you ensure it's true? Obviously you either don't write the article, or you quote the most reliable projections you can find. But those are still projections.
You can't honestly write "an article on how many people will die of the virus" which gives a number. No one knows.
What you can do is write an article about projections of how many people will die. In it you talk about the data sources being used in the model(s), the assumptions being made in the model(s), and the result they give including likely major sources of error..
You can put more or less detail depending on your audience. But it should always include enough detail that your audience understands that it is an estimate based on assumptions and likely flawed data, and you should always understand the model you are writing about even if you don't explain it.
Uhmm, no? Reality is way too complex for anyone to make a statement about what is "true" - even for domain experts and especially for journalists, whose, at best, entire specialization is "public health". We may be able to state what is "true" for very obvious matters where there are clear dividing lines for a matter to qualify (e.g. forest fire, election results). But when it comes to the sciences, it is incredibly difficult to make any such statement, especially when it comes to emerging research and predictive modeling.
Instead of journalists demonstrating the effect of Dunning-Kruger in a manner similar to what many computer scientists and engineers love to do about unrelated fields, they should rather listen to the experts and try to gather multiple opinions in order to triangulate what is probably correct.
Given a choice between an engineer who did some calculations on an unfamiliar topic, and a journalist who triangulated expert opinions but didn't do any calculations, I'll listen to the engineer.
1) The comment regarding engineers related to the tendency to assume, because one has specialized knowledge in one field, one also is qualified to comment on other topics, such as the ability to discern the "truth" in a highly volatile and complex social situation (e.g. COVID-19)
2) What you are describing is exactly what I mean: There are dozens of experts ("engineers") who have done their calculations but have come to different conclusions. To presume, as a journalist or expert, one's own calculations will provide the "truth" in such a situation is not only extremely arrogant but also, when it comes to a pandemic, extremely dangerous.
It reminds me of that electrical engineer at Imperial College who thought epidemiology is a cake-walk and wrote a paper predicting 5000 deaths in the UK, which stood in stark contrast to that modeling effort by a large group of actual, renowned experts in the field (epidemiologists, virologists, public health scholars) also at the Imperial College, whose estimates have at least estimated 20,000 deaths. The electrical engineer had to quickly backtrack on his claims after hundreds of scientists wrote in. Now imagine, every journalist would do that and directly publish it. That would be far worse than an article that brings up that 5,000 deaths study but also mentions other estimates.
> Journalists should report what is true, not what Tom, Dick or Harry said.
Especially they should not give 33% plausibility to Tom, 33% to Dick and 33% to Harry. Which is what they typically do, and call that "professional journalism."
If Tom represents 95% of scientists and Dick and Harry the fringe 5% also financed by (let say) tobacco industry or oil corporations, or Boeing, or those paid by the CIA, they should not even mention Dick and Harry in the same article (or TV a show), especially not in anything worth a major headline. They should appear with the smallest possible note in some smallest possible corner and with the title like "oil corporations paid these persons to support them again."
Sadly, but that sounds like a dream. The world would be very different then.
What you describe here is a deeply problematic fallacy and in fact what journalists already do. They are very, very far from your hypothetical "33% to Tom, 33% to Dick, 33% to Harry" scenario and it's one of the factors that undermines trust in journalism.
What actually happens is this:
1. Journalists have a story they want to write. They probably already decided what the message is going to be, but let's be generous and assume they didn't.
2. They consult a rolodex of, almost exclusively, government funded academics. This is true even if the story they're writing is about activity or science that takes place only in the private sector and for which the academics in question have no actual experience. They may consult two or three grant funded academics if they want their story to seem especially robust. If they give the private sector a chance to reply at all (often not) they will cite one or two sentences of a brief phone call in which the person being talked to doesn't know what they're trying to defend themselves against and is probably just confused. It's not a real interview with an actual in-house expert.
3. This story is then published as "Experts say, ..." even if what the chosen experts say flatly contradicts common sense or things that can be checked with 10 minutes and a search engine.
4. Readers comment below the line, pointing out the flaws in the story. If a specific company is involved, they may do a blog post explaining their side of the story which the journalists will either completely ignore, or if they think they can get away with it, selectively quote one or two sentences in a misleading way.
Industrial scientists/engineers are sometimes assumed to be inherently evil and untrustworthy. But the whole idea that journalists are untrusted because they very rarely quote people in industry is a saw of the left; it's not true, which is why the examples given always seem so curiously weird and out of date. Tobacco industry? when was the last time you even read a newspaper article about them? Oil corporations? Those firms that have spent the last 15 years rebranding themselves as energy companies because they now make solar panels too?
There's a lot of really good analysis of the trouble journalism has got itself in for, and a significant part of the blame is laid at the feet of journalists uncritically reporting anything academics say as the Whole Truth and Nothing But The Truth, when in fact they routinely contradict each other, make up statistics, report obvious common sense as "findings", are hugely over-confident in their own predictive abilities, mis-use statistics and so on.
A good book to read on the topic is "Wrong" by David Freedman:
He's a former journalist and so has direct experience of this problem. He cites many examples where expert testimony caused misleading or wrong stories to be published, but IIRC none of them involved corporate scientists. Mostly academics like nutritionists, psychologists and so on.
Claiming that the misinterpretation of science happened only in the distant past is intentional attempt to obscure the real problem.
There is objective truth and it is far from what some people with a lot of money peddle as the truth and what gets replicated across the media. And the media definitely don't cover what effectively advertising campaigns are as such -- paid disinformation for the benefits of some specific corporations or interest groups.
It's funny, we're not even really disagreeing with each other. The difference is I don't see government (academia) as sources of funding any different to a corporation: both sources create an incentive to huge bias, spreading of disinformation and other problems. But too many people and especially journalists like to pretend that getting your money from government grants magically makes those problems disappear. They're willing to criticise work by corporate scientists who may have an incentive to find a certain outcome, but not academics who have just as strong or even stronger incentives to find certain outcomes.
I suspect it comes from academic propaganda about 'free thought', being able to pursue any line of questioning they desire, etc. It's obviously not true. Academics find it impossible to reach a simple conclusion that's reached all the time in the corporate world: "we don't know the answer and cannot know anytime soon".
No, this isn't enough. This whole way of thinking isn't enough. It's a big part of the reason for the current situation.
Journalists should report what's true, not what Tom, Dick or Harry said. If a journalist isn't qualified to make object-level claims on a given topic, don't write on that topic. For example, if Bob says there's a forest fire, then instead of publishing "Bob says there's a forest fire", you must do enough legwork to tell your readers "There's a forest fire" or "There isn't".
I allow myself to ignore all journalism that don't follow that guideline, and it makes me happier.