For any young programmers: live within your means, invest the difference, become independent, and work on what you enjoy. It’s the best (work related) gift you can give yourself. Skip the self promotion politics unless you enjoy it.
Respectfully disagree. If you're maxing out your spend so at the end of the month there is no surplus you aren't living within your means. For a lot of folks this is an ugly necessity, but programmers generally aren't in this group.
I don’t hate work. But at the end of the day, it’s a means to exchange labor for money.
Out of the million of things I enjoy, helping the bottom line of a for profit company isn’t one of them. It’s a necessity.
And I actually like the company I work for. It’s one of the best companies I’ve ever worked for (10 in almost 30 years).
The self promotion politics is the only way you get ahead in large companies with a structured promotion process where you have to show “scope” and “impact”.
I actually enjoy when my work isn't just fun and good, but also contributing to a bigger picture. It's bot just that it gives meaning. It's also that the added design constraints are intriguing, and help me determine when I can stop polishing.
It's not about self promotion, but building with a clear goal set fir me I have found to be much more rewarding than when I have to think of my own goal. The worst is when a fake goal is set, it's the thing about university I liked the least. If I can't interrogate or question the 'why' for the goal, because it is just 'to test me' then it isn't a real goal, just an artificial constraint.
I'm sorry, but this advice can sometimes sound like "sell one of your kidneys so you can eat". What if your means are not sufficient to avoid hunger? Investing negative difference? What if on top of that you're trying to do the work you enjoy and your means - incomes - stop completely? Do you see the problem with advice?
Naturally not everyone is lucky enough to have the income to do this.
The first part though is key - living within your means. It assumes you have means, and that it's possible to live within them.
The advice is good - whether you use it or not is up to you, and of no consequence to the advice giver. Whether you are in a position to take the advice or not is up to you.
For those who can though I can agree with it. Forgoing a new car now might mean retiring a year earlier. Financial freedom (aka retirement) means doing work on your terms, not beholden to your employer. It doesn't mean "not working".
Of course the best way to a better job, more pay, and a sooner retirement is indeed to "sell yourself" making both yourself and your work more valuable.
Do this advice is a corollary to the article, not a repudiation of it.
> What if your means are not sufficient to avoid hunger?
Not all advice is applicable to everyone. It's up to you to decide if you can and want to follow it. The advice was for young programmers and it is solid advice, but again, not applicable to everyone. It is applicable to a majority (probably even large majority). If you are young and earning a sw engineer salary it is very rare to not be able to cover your basic needs and have something left. Most people spends what is left in luxuries, lifestyle creep, etc; which is what the advice is trying to warn people about.
I think you might modify the advice to be something like:
If you're a programmer, and you're paid well, don't assume that will last forever. Don't spend all your money (and beyond) on cars and rent. Invest as much as you can with the goal of being financially independent.
There are. But until the current crop of AI can actually do something useful, it isn't one. Right now it's hype driven development in search of a compelling use case.
In my own field (biomedical research), AI has already been a revolution. Everyone - and I mean everyone - is using AlphaFold, for example. It is a game changer, a true revolution.
And everyday I use AI for mundane things, like summarization, transcribing, and language translation. All supremely useful. And there is a ton more. So I never understand the "hype" thing. It deserves to be hyped imo, as it is already become essential.
I think there is a group of people that don't want AI to be useful and think that by telling other people that its not useful that these other people will believe them. Unfortunately for them, more and more people are finding value with AI. It literally saved my father-in-law's life. It's become my kid's best school tutor. But there will still be people telling me that it has no value.
It’s already ridiculously useful. It does atleast 80% of the work in the teams that report to me and writes/refines almost all the documentation I write. That doesn’t even involve all the hobbies I use it for. Ideas for new furniture to build, pattern generation for wood carving, ideas for my oil paintings etc.
it boggles my mind this is still being written over and over here on HN. like an echochamber everyone like fears AI will replace them or some BS like that. I cannot even begin to tell you how much AI has been useful to me, to my entire team, to my wife, to my daughter and to most of my friends that are in various industries that I personally guided towards using it. on my end, roughly 50% of things that I used to have to do are now fully automated (some agents, some using my help along the way)…
in every thread here on HN there will be X number of people posting exactly what you wrote and Y (where Y is much smaller than X) number of people posting “look, this shit is fucking amazing, I do amazing shit with it.” if I was in group X I would stop and think long and hard what I need to do in order to get myself into group Y…
Similarly, in every thread there’s an AI skeptic who says LLMs are “useless” for coding, and never provides an example query for what they were trying.
If you ask about more niche language features or libraries, chatgpt will make up libraries or functions to fill the gap.
When asking an LLM to write a script for you, I would say 10 to 30 % of the time that it completely fails. Again, making up an API or just getting things straight up wrong.
Its very helpful, especially when starting from 0 with the beginner questions, but it fails in many scenarios.
Thank you for sharing. For any young programmers: live within your means, invest the difference, become independent, and work on what you enjoy. It’s the best (work related) gift you can give yourself.
My investments have always had fantastic results, far above what any index fund could render. I have been forced to involuntary "dollar cost averaging" by having to wait for the next salary in order to invest more. But "dollar cost averaging" is not an investment strategy, it's confusion on the highest level.
Dear Internet Stranger (who is also an amazing investor): Have you considered starting you own hedge fund? If not, are you willing to share some of your best investments in the last few years?
You wouldn't be interested in an answer from an internet stranger. But please tell me how "dollar cost averaging" would help any investor in a real world scenario?
It actually grieves me that your average person is completely unwilling to just take a few days of their life to understand investing and what it is and how they can sensibly do it. Instead they decide to treat it like a casino and chase hocus-pocus like "dollar cost averaging" or "technical analysis" or "day trading".
Every person usually has some area of expertise or interest, or even a hunch. Take that and invest accordingly, long term. There are tools available to make any investment very conservative or very risky, according to taste.
Those are the (subsidized) prices that end clients are paying for the service so that's not something that is representative of what the actual inference costs are. Somebody still needs to pay that (actual) price in the end. For inference, as well as for training, you need actual (NVidia) hardware and that hardware didn't become any cheaper. OTOH models are only becoming increasingly more complex and bigger and with more and more demand I don't see those costs exactly dropping down.
Actual inference costs without considering subsidies and loss leaders are going down, due to algorithmic improvements, hardware improvements, and quantized/smaller models getting the same performance as larger ones. Companies are making huge breakthroughs making chips specifically for LLM inference
In August 2023, llama2 34B was released and at that time, without employing model quantization, in order to fit this model one needed to have a GPU, or set of GPUs, with total of ~34x2.5=85G of VRAM.
That said, can you be more specific what are those "algorithmic" and "hardware" improvements that has driven this cost and hardware requirements down? AFAIK I still need the same hardware to run this very same model.
Take a look at the latest Llama and Phi models. They get comparable MMLU performance for ~10% of the parameters. Not to mention the cost/flop and cost/gb for GPUs has dropped.
You aren’t trying to run an old 2023 model as is, you’re trying to match its capabilities. The old models just show what capabilities are possible.
Sure, let's say that 8B llama3.1 gets comparable performance of it's 70B llama2 predecessor. Not quite true but let's say that hypothetically it is. That still leave us with 70B llama3.1.
How much VRAM and inference compute is required to run 3.1-70B vs 2-70B?
The argument is that the inference cost is dropping down significantly each year but how exactly if those two models require about the ~same, give or take, amount of VRAM and compute?
One way to drive the cost down is to innovate in inference algorithms such that the HW requirements are loosened up.
In the context of inference optimizations one such is flash-decode, similar to its training counter-part flash-attention, from the same authors. However, that particular optimization concerns only by improving the inference runtime by dropping down the number of memory accesses needed to compute the self-attention. Amount of total VRAM you need in order to just load the model still remains the same so although it is true that you might get a tad more from the same HW, the initial requirement of total HW you need remains to be the same. Flash-decode is also nowhere near the impact of flash-attention. Latter enabled much faster training iteration runtimes while the former has had quite limited impact, mostly because scale of inference is so much smaller than the training so the improvements do not always see the large gains.
> Not to mention the cost/flop and cost/gb for GPUs has dropped.
For training. Not for inference. GPU prices remained about the same, give or take.
> How much VRAM and inference compute is required to run 3.1-70B vs 2-70B?
We aren’t trying to mindlessly consume the same VRAM as last year and hope costs magically drop. We are noticing that we can get last year’s mid-level performance on this year’s low-end model, leading to cost savings at that perf level. The same thing happens next year, leading to a drop in cost at any given perf level over time.
> For training. Not for inference. GPU prices remained about the same, give or take.
We absolutely care about absolute costs. 70B model this year will cost as much as it will next year, unless Nvidia decides to lose their profits. The question is whether an inference cost is dropping down. And the answer is obviously no. I see that you're out of your depth so let's just stop here.
