All the code is at https://github.com/samwho/visualisations. I didn’t use much more than basic JS, CSS, and HTML for this one. The only third party dependency, iirc, was one to parse JavaScript into an AST.
I just finished this book. I thought it was a great read, capturing most of the problems (with product, sales, marketing, competition) Nvidia faced in its life and how exactly the team solved them each and every time. It also goes into how the management, org structure, and culture have empowered its engineers to find creative solutions to staying alive in the brutal graphic chips industry and continuously discover and exploit new market opportunities.
I learned a lot about how important not just superior technology, but better operations, marketing, sales, and culture are all critical to a successful business.
The only con of this book is that it skips over some parts of nvidia’s history like the short-lived crypto boom, failed acquisition of ARM, etc. It’s still just a minor flaw in an otherwise great book though.
You missed Finance. There is lot of financial engineering going on to keep orgs alive and quickly adapt to ever changing needs. Dealing with cross border diff in taxes, subsidies, tariffs, regs, forex, interest rates, real estate/rent, labor costs and their constant changes is no joke.
Each of these specializations (just like tech) are evolving at their own rates and are all equally capable of outpacing each other depending on the environment.
I didn't. In my worldview, I put Finance as the majority stakeholder in "operating" the company even though there are plenty of teams and roles with somethingOps in their titles that might disagree.
The tech part of tech companies is the easy part in my opinion. Step out into these other domains where you have to deal with _people_ and a company's skill or lack thereof gets really obvious really quick.
And, god forbid, meta-issues like long-term cashflow planning when you're making big changes to your internal organizations and processes, which lets you worry less about funding. And/but investing in complex thinking about operations is part of the book's thesis.
Does it mention the investment from sega that saved Nvidia from going out of business. I think the Nvidia chips became part of some sega arcade boards, might be wrong about that.
That's at the beginning of the book, and it was apparently something of a debacle. The NV2 chip that was supposed to go into Sega's next-gen console never made it, but a $5M investment from Sega did keep Nvidia afloat until the Riva128 launched.
Despite buying a Dreamcast on launch day I only heard about this story in the last week or so ago in a YouTube comment with Jensen Huang congratulating sega on the new virtua fighter. Unfortunately he keeps calling it virtual fighter:
The whole ARM acquisition attempt really smelt of subtle stock market games, because the alleged benefits of such a deal made no sense. With the Arm CEO being ex Nvidia I'm sure he was in on it. Looking forward to someone writing up an account of what really happened.
I think that the blocking of the deal by competition authorities shows the potential benefits of the deal (for Nvidia) in having control of Arm, in addition to making Arm-based SoCs, were real.
Also, Rene Haas (ex Nvidia) became CEO of Arm after the deal fell through. Simon Segars was CEO when the deal was announced.
> I think that the blocking of the deal by competition authorities shows the potential benefits of the deal (for Nvidia) in having control of Arm, in addition to making Arm-based SoCs, were real.
China was concerned about more US influence. The other regions were concerned about stifling of innovation. In reality, a successful acquisition would have meant that most customers would have started transitioning away from ARM and sooner to available RISC-V or Cadence/Synopsis solutions.
> Also, Rene Haas (ex Nvidia) became CEO of Arm after the deal fell through. Simon Segars was CEO when the deal was announced.
Rene Haas was EVP of the whole IP business (CPUs, GPUs, NPUs, etc) before becoming CEO. His influence with the CEO can't be understated.
I posit that this was an enormous gamble to boost valuations of both NVIDIA and ARM and that they knew from the beginning that the deal would not go through. Anyone who understands the ARM business model could have told you that.
> I posit that this was an enormous gamble to boost valuations of both NVIDIA and ARM and that they knew from the beginning that the deal would not go through.
You posit it without saying why the valuations would go up given that you think they - and everyone else given what you say - thought the deal wouldn’t complete.
I'm very glad I read the book, but it did lean a little towards the hagiographic side. From the outside it looks like the same high-pressure culture that drives so much innovation at Nvidia also drove their conflict with Apple.
+1 with your assessment of trending towards a hagiography. The author is not Isaacson. I wish he had spent more time with the people that probably don’t have the best view of Huang’s methods, and approach to running the company. Showing the other side in a more vivid way so the reader can make their own conclusion would strengthen the book and provide a holistic, fuller profile of Huang and Nvidia. Probably the price for access, and still better than nothing…
Nvidia appears to be another pressure-cooker, dog-eat-dog SV company, that outlasted competitors thanks to shrewd bets and urgent execution. Clearly, a great accomplishment, but we’ve seen other companies succeed with similar culture… this isn’t a “new way”… a relentless, obsessive, ruthless technical founder with good business sense (Huang like Gates were able to secure partnerships/licenses of needed technology early on.. same mold as Musk, Zuck) who understands the technology and business implications in detail is a movie we’ve seen before..
I sigh a little whenever anyone mentions Isaacson. My book had a different goal. I wanted to write a serious, substantive business history of Nvidia and explore the reasons for its success. Walt has a different style and focus, which wasn't what I aimed to do.
Thanks for the kind words swimwiththebeat. I really appreciate it. I didn't get to the ARM acquisition failure and crypto boom given time constraints. There is so much to cover in Nvidia's three decade history, so I had to pick and choose.
A card released in September 1997 is described as "an aging and slowly dying chipset." I suppose it parallels AI today, where a model released a year ago is already obsolete on the high end...
Nostalgia for the 1990s is real. I loved that era. PC gaming was awesome. I bought tons of 3D graphics cards from Rendition Verite, 3dfx Voodoo and obviously Nvidia. Having John Carmack tweet about my book nearly made me faint (surreal!). I adored his Doom and Quake games so much.
One of the reasons why Nvidia managed to get ahead was that they managed to get a new chip out every 6 months without fail, so there was always a new Nvidia card to upgrade to.
It's not, and the "Talk to Sales" and the ToS (https://www.dalmatian.ai/terms) strongly suggest they're targeting enterprise customers with bespoke enterprise-y pricing.
Right, we are not open source, but the IDE is free to use. The Slack integration and other product offerings that are in the works will be in the 'premium' version of the product that's sold to enterprise
She definitely knows, she’s just trying to avoid any chance of future litigation by feigning ignorance. Makes sense since OpenAI’s been getting a lot of bad press for using copyright data in training their models.
If someone asks an executive whether they do something illegal, and they reply that they don’t know; does it really protect them from anything or does it simply show negligence?
It protects the company, admission is pretty much asking for a law suite.
For these companies it’s especially important since even if copyright claims will come down the line they need to become big enough to be able to push back on them so they could force a favorable settlement.
Calling it "lot of bad press" seems like an overstatement (maybe looking through hn tinted glasses). Overall, I do not get the impression that people care super much relativ to the overall public interest in everything OpenAI.
I would be shocked if there weren't, being a company with over hundred millions users, that is in the process of reshaping the world (for better or worse).
Being sued in general is not that great of an indicator for anything, in a world filled with lawyers and angry people.
As someone who wants to learn and explore more areas of computer science and programming, this outline and centralized list of resources is incredibly helpful! Thanks so much.
I agree, I don't think the purpose of sci-fi is to predict the future. The future is just impossible to predict due to the myriad factors, variables, unknown unknowns, and second-order+ degree effects an action can have.
The purpose of sci-fi IMO is moreso to:
1. Provide an entertaining story/narrative with technology as the main focus of the world and characters' actions
2. Define a set of concepts to help you think about technology and its possible effects on humans and the world
3. Nudge people to think about what kind of future they would want or not want and how they can use or control technology to achieve that
I do think predictions are an unavoidable side-effect of writing science fiction, but those predictions are almost always in service to those purposes you listed. Stephenson must think nanotech book AI is more plausible as a form idealized childhood education than say, magic fairies. But in the end he relies on our suspension of disbelief to get on past that to the big questions of nurture vs nature and the quality of human intelligence.
1. First of all, thanks for outlining how you trained the model here in the repo: https://github.com/NumbersStationAI/DuckDB-NSQL?tab=readme-o...! I did not know about `sqlglot`, that's a pretty cool lib. Which part of the project was the most challenging or time-consuming: generating the training data, the actual training, or testing? How did you iterate, improve, and test the model?
2. How would you suggest using this model effectively if we have custom data in our DBs? For example, we might have a column called `purpose` that's a custom defined enum (i.e. not a very well-known concept outside of our business). Currently, we've fed it in as context by defining all the possible values it can have. Do you have any other recs on how to tune our prompts so that this model is just as effective with our own custom data?
3. Similar to above, do you know you can use the same model to work effectively on tens or even hundreds of tables? I've used multiple question-SQL example pairs as context, but I've found that I need 15-20 for it to be effective for even one table, let alone tens of tables.
Hi, Till here, worked on the DuckDB-NSQL model on MotherDuck side.
1. definitely training data (for me), we explored about 10 different directions before settling on the current approach. It's easy to underestimate the effect of training data on the quality of the model. Starting point was the benchmark dataset though, which we assembled manually (to avoid data pollution and also because there was simply no text2sql benchmark that covers anything else than plain old SQL select statements with a handful of aggregate functions). And training is also not a one-off thing. With large datasets it is hard to evaluate the quality of the dataset without actually training a few epochs on it and run the benchmark.
3. No way - I see a common stack emerging (take a look at companies like https://vanna.ai/, https://www.dataherald.com/, or https://www.waii.ai) that is mainly centered around foundation models like GPT-4 with strong in-context learning capabilities (that's a kind of a must to make these approaches work and comes with long inference times and higher costs). These solutions include things like embedding-based schema filtering, options for users to enrich metadata about tables and columns, including previous related queries into the context etc. around the model. I'd say it's a bit of a different problem from what we aimed at solving.
I didn't see this in the blog post, but did you train this from scratch or finetune an existing base model?
If from scratch, quite impressive that the model is capable of understanding natural language prompts (English presumably) from such a small, targeted training set.
I see so many business leaders touting the promise of LLMs allowing business to "talk" to their data. The promise does sound enticing, but it's actually kind of hard to get working in practice.
A lot of our databases at work have columns with custom types and enums, and getting the LLM (Llama2) to write SQL queries to robustly answer natural language questions about the data is tough. It requires a lot of instruction prompting, context, and question-SQL examples (few-shot learning), and it still fails in unexpected ways. It's a tough ask for people to use a tool like this if they can't trust the results all the time. It's also a bit infeasible to scale this to tens or hundreds of tables across our data warehouse.
It's great that a lot of people are trying to crack this problem, I'm curious to try this model out. I'd also love to see if other people have tried solving this problem and made any headway.
I've been working on the DuckDB-NSQL model on MotherDuck side. I fully agree that general text-2-sql, in the sense of "give me a question, and I'll produce you an arbitrary complex query", is a very tough problem and I actually believe that that's not the right problem to solve. Not necessarily because models are not (going to be) cabable enough, but rather because it's way too hard for humans to express in a single prompt what they actually want. Furthermore, it's simply not the right UX for most SQL users. A much better approach IMO is to keep the SQL analysts in the driver seat, and provide nuanced support wherever/whenever they need it most. The FixIt feature we recently launched goes into the same direction: https://motherduck.com/blog/introducing-fixit-ai-sql-error-f...
In that sense I emphasized in our Blogpost that users should think of it as a documentation oracle that always gives you the exact DuckDB SQL query snippet you are looking for, which is a tremendoues time-saver if you have an abstrat idea of the query you want to write, but you're just not sure about the syntax, expecially with DuckDB having so many functions and SQL extensions.
Here are a few exammples:
- create tmp table from test.csv
- load aws credentials from 'test' profile
- get max of all columns in rideshare table
- show query plan with runtimes for 'SELECT * FROM rideshare'
- cast hvfhs_license_num column to int
- get all columns ending with _amount from taxi table
- show summary statistics of rideshare table
- get a 10% reservoir sample of rideshare table
- get length of drivers array in taxi table
- get violation_type field from other_violations json column
in taxi table
- get passenger count, trip distance and fare amount from taxi
table and oder by all of them
Thanks for replying, that's a perspective I didn't consider. The capability to "talk to your data" just seems so enticing as a solution that I was tunnel-visioned into that UX. If I'm understanding correctly, what you're suggesting is more of a SQL assistant to help people write the correct SQL queries instead of writing the entire SQL query from scratch to answer a generic natural-language question?
I have found LLMs to be extremely helpful in mapping between schemas as well as helping me formulate queries where, because of decay, data in tables and column names, etc don't map to what you think they would.
You need to provide as much context as you can to the LLM. So full schema definitions, and histographic summarization and samples from the tables themselves.
I hear what you're saying, and actually agree that - unfortunately - is the current state of LLMs today. But I think OP still has a point, and I'd argue as well that that is the aim of GAI, to be able to take a simple question, understand and produce the desired results.
As it stands, and as you mentioned, the current generation of LLMS seem a long way from there. Thus the need for prompt engineers. It'll be nice to see how long they take to crack this problem.
I agree. I think full text-to-results (even bypassing text-to-SQL) is akin to L5 self-driving cars. It's easy to get to some reasonable level, say 90%. But to get to the point, where folks can fully trust the system and don't need a steering wheel (or know SQL) may take decades.
We at MotherDuck took an incremental approach. We launched something more akin to lane-assist. We're calling it FixIt - an in-flow visual aid to help you identify and fix errors in your SQL [0].
I think there's gobs of opportunities to improve the analytics experiences without resorting to "L5 self-driving" (e.g. full text-to-results)
I've worked on a similar problem and we have pretty much the same issues as you. An idea that makes things better is having an intermediate representation that exposes the key tables of your dwh and gets compiled to SQL. This allows you to have somewhat better security (even if the model outputs garbage it can't do too much damage because nothing except exactly what you want is even representable), and somewhat better explainability (you can't guarantee the model will get it right, but you can see what SQL gets executed rather than "magic 8-ball says no").
But as you say custom types and encoded domain knowledge is extremely tough and as a result it's very tough to "transfer" the system to different databases.
Lots of folks taking this approach and feels like the wrong entry point, e.g., similar to asking LLMs to generate bytecode when compilers exist or 3d printing a machine vs. building a machine from 3d printed components.
1. Business users aren’t prepared to talk to their data in meaningful ways and this is an opportunity for LLMs to enhance the way users ask questions.
2. SQL modeling languages exist (although I’m not sure there are well maintained open source good ones and this is the biggest obstacle to what I’m working on now) and LLMs can extend these effectively by adding components such as dimensions, metrics, relationships, filters, etc. with less chance of hallucination
3. Deterministic SQL generation from a modeling repository is easier to troubleshoot and provide guarantees than end-to-end generation.
4. Existing SQL can be parsed to generate modeling components that can be committed to the model repository with LLM assistance
5. Much of the richness of going to data to answer questions is context, e.g., how does this dept compare to others, this month to same month last year, etc. Modeling languages are good at expressing these transformations, but business users and often analysts aren’t good at capturing all the permutations of these types of comparisons. Another area where LLMs can help apply tooling.
IMO, LLMs are more effective at using tools than generating complex outputs.
> but it's actually kind of hard to get working in practice
One of the biggest challenges I've personally seen in this space is business "leaders" pushing teams to ship products asap lest they loose face among their fellow CEOs for not pushing out "AI" products before everyone else.
I'm fairly optimistic about LLMs being able to truly be transformative, but it's not going to be through forcing the bread-dead UX of hoisting yet another slightly re-imagined chat interface on users.
The idea of "talking to your data" is a promising one, and anyone who has worked for a large data driven org will quickly agree that organizing and searching in-house data is not a solved problem from the UX end of things. But to truly solving these problems, even/especially with LLMs, is going to require thought and experimentation. Something few "business leaders" have patience for.
My biggest concern is that this will allow people to type a question and get a number back from the database, without being able to tell if the query is right or if the LLM just made up something.
It can work to support business analysts to crank out more reporters, but I wouldn’t roll it out to all my staff.
I've actually done exactly what @qsort suggested and outputted the intermediate SQL query and raw data generated by that query when generating the response back to the user. That definitely helps in establishing more trust with the customer since they can verify the response. My approach right now is to just be honest with our customers in the capabilities of the tool, acknowledge its shortcomings, and keep iterating over time to make it better and better. That's what the team in charge of our company-wide custom LLM has done and it's gained a surprising amount of traction and trust over the last few months.
I assume that "Talk to your data" presupposes the existence of some simple view that has been created and that embeds 99% of the required business logic.
"What was the average order size per customer for product XYZ in the West region?"
Imagine turning that one loose against the typical legacy system.
I wouldn't trust an LLM to figure out the joins or aggregate calculation, LET ALONE the definition of a customer, a product, or a region.
I’m trying to solve for this with my project using RAG and (at least based on what people say in Discord), it’s working really well for them:
https://github.com/vanna-ai/vanna
