They aren’t using the prompt caching on the query side, only on the embedding si... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		spott on Sept 20, 2024 \| parent \| context \| favorite \| on: Contextual Retrieval They aren’t using the prompt caching on the query side, only on the embedding side… so you cache the document in the context window when ingesting it, but not during retrieval.

KTibow on Sept 20, 2024 [–]

It seems a little odd to make multiple requests instead of using one request to create all the context for all the chunks.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact