This reminds me of the single conquistador with a cancer-prone gene, who's responsible for 1 in 300 Brazilians carrying that gene (https://aacrjournals.org/cebp/article-abstract/doi/10.1158/1...)

I'd be curious to read that whole story of Google putting the conpany out of business. Would you consider writing about it in a post?

> We shouldn't have to choose between principles and food on the table.

I am increasingly convinced that these are the only true kind of ethical decision. Painless/straightforward ethical decisions that you make every day - they probably don't even register on your radar. But a tough tradeoff does.

I encourage you to get clear on your goals for a project like this. There is a political and social dimension to this work aside from the technical challenge.

For example, ordinary people already know whether they're being squeezed or not. They don't need a CPI report. If you want them to believe your CPI report over the government's, then you have to do a lot of marketing. On the opposite side, government decision makers are generally moving away from a "technocrat" data-driven approach to decisions, towards a "great man" theory of decision making: giving them better CPI data won't suddenly make them care about using data again. A lot of people are putin in work to dismantle American government capacity, and there are worse problem than inflation tracking.

My personal opinion is that it doesn't make sense to create a community-driven CPI as a direct alternative to the government CPI. But it does make sense to collect a lot of this data for other uses (ex: to highlight illegal collusion among merchants, archival).

From a technical perspective, here are some thoughts from a Canadian lens: * CPI calculations are already fairly transparent. You can read about the work that goes into them (click TOC on upper right: https://www150.statcan.gc.ca/n1/pub/62-553-x/62-553-x2023001...) * Government-collected raw data is not open to the public, because merchants wouldn't have give the govt' access to their POS data if it were otherwise. The Canadian government made a tradeoff: it gets very accurate data, in exchange for it being hidden from the public. * As WheelsAtLarge mentioned, data collection and analysis takes effort and money. Especially when you're doing it for the long term. You'll have to think of funding and keeping people motivated to do the work. * Scraping data might be a lot harder than you expect. I compile the largest open dataset of grocery prices in Canada, and have 20 months' worth of data. Its very difficult to average prices across merchants (because many don't reveal product UPC codes). And prices vary by geography. None of this is insurmountable, but it takes effort and money. Go ahead and play with the data to get a feel for how easy/difficult this is (https://jacobfilipp.com/hammer/)

I hear and accept all your points. I def. recognize the marketing difficulty.

Frankly, I was assuming there were folks doing this and I'd get educated on that.

Also, really good info on Canada and kudos on compiling that data. I followed your article to this one: https://jacobfilipp.com/the-story-of-canadas-bread-prices-in... which was a good read.

Thanks for your work.

"Jedi Blue" - the 2 big menaces, working as friends!

Palantir

I'm a marketing technologist, and it was a 3-year search that you can read about: https://jacobfilipp.com/job-search-with-figures/

In terms of a blind date, an escape room is a large time committment with no built in "out" in case a participant is not feeling it. Try to create an environment with escape hatches.

I'm going to second this: if a book doesn't grip you and get you to be curious, then this indicated it is badly written (applies to both fiction and nonfiction). Give yourself permission to skim books or stop reading halfway - life is too short to read something that's not enjoyable or useful.

We desperately need a high-tech solution to disengage from the torrent of AI slop. I have found a form of photosynthesizing plant that grows in the soil of many places where humans live. Apparently, touching this with the palm of your hand creates a kind of protective endorphin effect that coalesces conflicting versions of reality into the one you're actually experiencing. It'll take a lot of money to advertise this to the general population, but I'm fundraising as hard as I can: this approach will be called "Touch Grass" and I hope to see you all when we do it together.