To be fair I remember spending almost two weeks implementing OTel at my startup, the infrastructure as code setup of getting collectors running within a kubernetes cluster using terraform was a nightmare two years ago.
I just kept running into issues, the docs were really poor and the configuration had endless options
some of the design interactions are really polished. the section written with the quotes from founders is really cool. the hover effect with the before and after of the YC partners is a great touch too!
to be fair at least half of the software engineers i know are facing some level of existential crisis when seeing how well claude code works, and what it means for their job in the long term
and these are people are not junior developers working on trivial apps
Yeah, I've watched a few peers go down this spiral as well. I'm not sure why, because my experience is that Claude Code and friends are building a lifetime of job security for staff-level folks, unscrewing every org that decided to over-delegate to the machine
Cleanup is less enjoyable than product building. If every future job is cleaning up a massive pile of AI slop, then that is a less fulfilling world than currently.
The primary exfiltration vector for LLMs is making network requests via images with sensitive data as parameters.
As Claude Code increasingly uses browser tools, we may need to move away from .env files to something encrypted, kind of like rails credentials, but without the secret key in the .env
So you are going to take the untrusted tool that kept leaking your secrets, keep the secrets away from it but still use it to code the thing that uses the secrets? Are you actually reviewing the code it produces? In 99% of cases that's a "no" or a soft "sometimes".
> Employees are under contract and are screened for basic competence. LLMs aren't
So perhaps they should be.
> and can't be.
Ah but they must, because there's not much else you can do.
You can't secure LLMs like they were just regular, narrow-purpose software, because they aren't. They're by nature more like little people on a chip (this is an explicit design goal) - and need to be treated accordingly.
Unless both the legalities and technology radically change they will not be. And the companies building them will not take on the burden since the technology has proved to be so unpredictable (partially by design) and unsafe.
> designed to be more like little people on a chip - and need to be treated accordingly
Deeply unpredictable and unsafe people on a chip, so not the sort that I generally want to trust secrets with.
I don't think it's that complex, you can have secure systems or you can have current gen LLMs. You can't have both in the same place.
> Deeply unpredictable and unsafe people on a chip, so not the sort that I generally want to trust secrets with.
Very true when comparing to acquaintances, but at a scale of any company or system except the tiniest ones, you can't blindly trust people in general either. Building systems involving people and LLMs is pretty similar.
> I don't think it's that complex, you can have secure systems or you can have current gen LLMs. You can't have both in the same place.
That is, indeed, the key. My point is that, unlike the popular opinion in threads like this, it does not follow that we need to give up on LLMs, or that we need to fix the security issues. The former is undesirable, the latter is fundamentally impossible.
What we need is what we've been doing ever since civilization took shape, ever since we've started building machines: recognize that automatons and people are different kinds of components, with different reliability and security characteristics. You can't blindly substitute one for the other, but there are ways to make them work together. Most systems we've created are of that nature.
What people still get wrong is treating LLMs as "automatons" components. They're not, they're "people" components.
I think I generally agree, but I also think that treating them like people means that you expect reason, intelligence and a way to interrogate their way of "thinking" (very broad quotes here).
I think LLMs are to be treated as something completely separate from both predictable machines ("automatons") and people. They have separate concerns and fitness for a use-case than both existing categories.
Sooo the primary way we enforce contracts and laws against people are things like fines and jail time.
How would you apply the threat of those to "little people on a chip", exactly?
Imagine if any time you hired someone there was a risk that they'd try to steal everything they could from your company and then disappear forever with you having no way to hold them to account? You'd probably stop hiring people you didn't already deeply trust!
Strict liability for LLM service providers? Well, that's gonna be a non-starter unless there's a lot of MAJOR issues caused by LLMs (look at how little we care about identity theft and financial fraud currently).
i tried this and it's pretty cool, that being said for my use case of spinning up many agents working on my app I'd need a way to specify the docker images that get started with each new VM
i cannot find a way in the docs to start new VMs with a bootstrap script that starts a bunch of services for me and runs a specific docker image
my use-case is that I want a full developer environment for every branch of my project, so i can vibe code on many VMs at a time
EDIT: Just realised there's an image one can pass to the new command. Still it's not clear to me whether private images would be supported and what registry this is using:
exe.dev ▶ help new
Command: new
Create a new VM
Options:
--command container command: auto, none, or a custom command
--env environment variable in KEY=VALUE format (can be specified multiple times)
--image container image
--json output in JSON format
--name VM name (auto-generated if not specified)
--no-email do not send email notification
--prompt initial prompt to send to Shelley after VM creation (requires exeuntu image)
[exe.dev cofounder here] Thanks for the feedback! We do not support private registries yet but it is very much on our mind, it is one of the first things business customers ask for so we know we have to build it.
We are also exploring alternatives for pre-configuring your VM. (Because we make lots of VMs and feel this too, so it is very much on our mind.) One is a sub-second VM "clone" feature, so you can configure a base VM to use as an image.
Our core product is an analytics product that lives on people's website, so we're able to tell what are the main pages that people are reaching with LLMs
For our in-app AI visibility product we use that information for finding prompts at topics that are being used to reach those pages
For this public tool instead we do a best guess of what are reasonable queries companies would want to show up for and run them against Google and ChatGPT
we do a best guess of what are reasonable queries companies would want to show up for
Got it.
I've done this before using an LLM, but I mistakenly thought you had some magic source (magic sauce!) of actual LLM queries. Sometimes it's not the obvious stuff.
Hey there Ferruccio here, I worked on this launch for the past month since joining Amplitude
We built this tool because we’re seeing that LLMs are becoming the main way people compare brands when making buying decisions, even before visiting your site
I’d love to get feedback from the HN community on this.
If you want to skip the video explainer you can generate a report directly by typing in your brand here: https://amplitude.com/try-ai-visibility (no email required, it just takes 5 minutes to generate)
Our chief engineer Wade gave an awesome demo to Claire Vo some months back here: https://www.youtube.com/watch?v=9Q9Yrj2RTkg
I use this basically every day asking all sorts of questions
reply