Perhaps you can limit to training only on peer reviewed sources. The peer review process is imperfect but it is perhaps the closest thing we have to flagging something as the "best" answer for a particular topic.
History (maybe exception for scientific history), politics and current affairs I would say falls outside the scope of "scientific knowledge". I do not think it is possible to avoid bias in those topics.
A significant question is what the cutoff point would be for a model based on "scientific knowledge"? Should subjects like economics, philosophy etc be included as Scientific knowledge or Should it be limited to "hard" sciences only?
Peer review isn't all that and a lot of subjects are censored and even disinformation is accepted if it flatters the ideological inclinations of the publication.
See: Proximal Origins from Nature
History (maybe exception for scientific history), politics and current affairs I would say falls outside the scope of "scientific knowledge". I do not think it is possible to avoid bias in those topics.
A significant question is what the cutoff point would be for a model based on "scientific knowledge"? Should subjects like economics, philosophy etc be included as Scientific knowledge or Should it be limited to "hard" sciences only?