Data is useful! But it seems like (I am not an economist) the incentive structures around these offices are failing. Maybe the benefits need to be more immediate / obvious / visceral.
I haven't seen any papers doing a proper analysis on the topic, just mostly saying this from firsthand experience testing a handful of them and comparing to the model they were based on given same prompt and sampler. It's usually not even close and you can immediately tell that it's notably dumber. Iirc in one case one even forgot how to do basic arithmetic while the original model aced it. Not entirely unexpected results from sticking a digital ice pick into the weights.
Afaik there are only three major sources of quality unaligned model versions, which are Nous's Hermes models, Hartford's Dolphins and Drummer's Tigers. All of them regular fine tunes that are mostly the same or just ever so slightly lower in performance as the original.
Please detail what you mean by "intentionally" here, because obviously, this is the ultimate alignment question...
...so after having a read through your reference, the money-shot:
Intentional hallucinations primarily happen when o1-preview is asked to provide references to articles, websites, books, or similar sources that it cannot easily verify without access to internet search, causing o1-preview to make up plausible examples instead.
First of all - very cool that they were able to image this and work out the mechanical mechanism in such detail.
Second - the rupture event looks so violent but iiuc it’s actually highly controlled:
- the rupture site is thinned early in the ovulation process
- the expansion step pulls in fluid to the ovary (builds internal pressure)
- the contraction phase restricts the cell volume (which also builds internal pressure)
- the oocyte is launched out of the cell at a relatively high speed
"the biggest problems in drug development are not really solved by having faster access to protein structures ... we have far bigger issues with target selection (does this idea of ours really do anything against the disease, in the end?) and with unexpected toxicity (what else does our candidate do in real animals and real humans?) Those last two are where that 90% clinical failure rate comes from."
Who is your ideal user? Do you think this person frequents ProductHunt? That's your answer.
My experience is that the main users of PH are those launching a product.
I'll also add: after my PH launch, I experienced a lot of email spam from "growth hacker" types. I suspect they scrape PH. I tried to take down our post (because the spam was overwhelming), but PH refused.
I like nicer products but last time I found a good product on PH was about 5 years ago. The best of awards every year also hints at how bad it is - most of those products aren't even that good.
I noticed some things did do really well, like a list of decks from successful startups. Again, reinforcing that the main user of PH are people trying to launch a product.
