There are facts about experiences. What is the experience of a fourth primary color? We're in a similar position to Mary before she leaves the room.

There are facts about experiences (such as the fact that some people have perfect pitch), and Mary can learn those facts from a book. It is a premise of this thought experiment, however, that you cannot learn things such as what it is like to see colors (or to have perfect pitch ourselves, for that matter) from books. The notion that this presents some sort of challenge to materialism comes from a) the way we use "to know" polysemously (at least in English), to refer both to having both factual information and memories of sensory input and emotions, and b) the misdirection of limiting what Mary studies to physics, as if this limitation on what you can learn from book studies was particular to the physical sciences. The argument is ultimately trivial, as its conclusion does not go beyond its tacit premise.

There's also the experience of knowing something, and I imagine it's like parentheses, exactly the same as knowing something. Again this whole know/experience distinction is presumptuous given our knowledge of neurobiology.

Case in point, I can guess that the experience of a fourth primary color is going to be close to one of another color. I know it's not going to be warm or anything (I'll guess that it looks kind of shimmery purple, why not, purple's mysterious). People do the same with drugs - it binds to XYZ receptor, releases ABC hormone; and just from some pharmokinetic principles you can guess fairly well how it's going to feel.