Excuse my harshness but people doing it needlessly is just unprofessional waste and abuse.

Some people seem to have no concern with the needs and timetables of the would be customers but instead burn through cash building fancy nonsense.

It's like going in to a car mechanic for tires and then finding out it took 3 weeks because the guy wanted to put on low rider hydraulics and spinner hubcaps for his personal enrichment.

The worst part is it's inherently ambiguous to the next people. They don't know if the reason something is there is because it's needed or because it's just shiny bling.

I am certainly not saying everything you say is not all true. My comment is dark humour. I really like your last point. Years ago I replaced a huge hadoop cluster data processing job with a single app on one machine with a few CPUs, that reduced a job that took over 8 hours to 20 minutes. What is even dumber is, it was just a python script and gnu parallel, which used to be perl.

I've seen people do hadoop clusters for a few hundred MB.

It's so insane. Like hiring a long haul truck to pick up a sandwich

…but if the bosses at competing mechanic shops hire based on quality of low riders a mechanic can install, of course they'll practice on the paying customers.

I quit working about 1.5 years ago. I think I still love computers while I simultaneously hate "the web". Don't get me wrong, to my amazement people have called me the best web developer they've ever met and I routinely get put on web like things at every company I go to - hardware, logistics, finance, I've been trying to run away from it but it keeps finding me and I think I hate it.

I've got this allergic reaction to bullshit and fetishize successful products and customer satisfaction. I think we've both changed; I'm different than I was 20 years ago and so is web development.

Tight applications with minimal tools that can be pivoted and changed swiftly which require competence and finesse to administer where you don't create developer debt, these are out of fashion.

All profitable hacker spaces professionalize as romantic magic becomes a liability.

I'm a middle aged divorced man, not divorced from a person, but from a profession and I've been trying to date around with new loves.

Just take a look at the level of complexity in home lab subreddits!

I don’t quite get if people do it for interest, for love of the tech, or if they are technocratic and believe in levelling up their skill to get k8s on their CV like you say.

All I think is “this looks painful to manage”!

K8s is painful to get started, and painful to learn. But once you have it up you can just keep adding stuff to it.

I run a k8s cluster at home. Part of it yes, is to apply my existing skills and keep them fresh. But part of it is that kubernetes can be easier long term.

Ive got magical hard drive storage with rook ceph. I can yoink a hard drive out of my servers and nothing happens to my workloads.

I can do maintenance on one of the servers with 0 down time.

All of my config for what I have deployed is in git.

I manage VMS and kubernetes at work, and im not going to pretend that kubernetes isnt complex, but it's complex up front instead of down the road. VMs run into complexity when things change. I'm sure you can make VMS good but then why not use something like kubernetes, you will have to reinvent a lot of the stuff that's already in kubernetes.

It's a hammer for sure and not everything is a nail, but it can be really powerful and useful even for home labs.

I don't run k8s at home, but I have worked in k8s-heavy environments and studied it deeply. This is the accurate, nuanced take.

Few but not no people will ever run into problems at the kind of scale k8s operates at. Plus, learning how it "expects" the programs running inside its Pods to behave is kind of like learning how Django or Rails "expect" a web app to work - it's a more complicated style than just writing your own totally custom, hermetically-sealed Python apps for your personal use, sure, but it also comes with a slew of benefits in case you ever do hit that level of scale and want to move over.

Or, maybe you look over the app you're writing and say "Fat chance." In which case you can justify e.g. not making everything an API endpoint, keeping a ton of state mucking about, etc. But I still feel that's an improvement over not even realizing the questions are being asked.

What you also can do is starting with just a single node, incredibly easy to install with e.g. https://k3s.io/. You still have to invest the upfront effort to understand how it works but you can already reap a lot of benefits with a lot less complexity.

Kubernetes does not force you into the distributed systems hell, you can go that route later, or never.

Kubernetes/k3s on a single node turns what could have been immutable 1-step upgrades into multi-step mutable upgrades, since kubernetes's software itself and all the management components you need are a mutable layer on top of the operating system.

a) It doesn't have to be mutable. You can easily setup k3s on a single node, install the apps and bake an AMI or equivalent. And using something like ArgoCD or GitOps will ensure that your k8s stack is in sync with a tracked and managed Git repository.

b) In what world is upgrading your entire platform ever a single step. Even for a basic Python app you still have Python itself plus dependencies. And then of course whatever front end web server you're using.

You can use Talos linux for an immutable (and tiny) OS.

> K8s is painful to get started

Is that really true anymore? Even self hosting k8s these days (e.g with rke/rke2) is a single yaml file and one command to deploy an entire cluster.. Maybe back when we all used kubespray and networking was more complicated (to the user at least) etc.. But today? I don't think so.

Using a hosted offering is even easier, literally a couple of clicks, a ./gcloud-cli or terraform apply -- again not very hard and all the cloud providers provide you with example code you just need to plug some machine sizes etc into..

Dev setup? Install orbstack and click 'kubernetes' and you're done, your IDE (likely) will automagically pick up your kubeconfig and you can go right ahead creating services, deployments, jobs, whatevers...

I'm not talking about setting up a cluster. I'm talking about all the learning you have to do.

I’m sure there are countless other benefits. But how many layers of abstraction, services and things that need configuring are their compared to basic RAID to get support for magical hard disks that can be yoinked without affecting workloads?

You get an aligned infra layer. You get a great opensource ecosystem (k8s, argocd, git / gitops, helm, helm charts, grafana, prometheus etc.)

You get basic loadbalancing, health checks, centralized and nearly out of the box logging and monitoring and tracing.

You get a streamlined build process (create a container image, have an image build, create your helm chart, done)

Your RAID commment is quite far away of what k8s makes k8s

> Compared to basic RAID to get support for magical hard disks that can be yoinked without affecting workloads?

These things aren't mutually exclusive though. I've spent the last few years working with kubernetes at work and running a 'simple'(but with tons of containers and weird edge cases / uses) unraid server at home for all of my needs. At some point I flipped over from 'jeez kubernetes is just too much, almost nobody should ever use this' to 'wow I have to migrate 99% of my home services to a cluster, this is driving me nuts.' I haven't quite gotten around to that migration, but I do think that k8s cluster for services / temporary storage / parallel jobs and separate unraid box that runs NFS (and doesn't do much else) is going to be a great setup for a home lab.

Aren't disks so large those days that losing a disk almost means you will lose a second disk during resilvering unless that by "basic raid" you're doing not-basic-raid things such as btrfs raid1c3?

> But once you have it up you can just keep adding stuff to it.

I dunno why, but the k8s in my workplace keeps breaking in painful ways. It also has an endless supply of breaking points that makes life painful for anybody that depends on what runs in it, but aren't detected by the people that manage it.

Honestly, that second part is an exclusivity there, but I have never seen people "just keeping adding things to it" on practice.

It depends on how well you know k8s and what your stack is. Rancher is an extra complex version of k8s. Longhorn is pretty fragile in my experience, so is canal. But chillum and eks don't really have the same reliability issues in my experience.

Assembling complex systems is just inherently fun as long as you don't have deadlines or performance metrics to hit.

It's a bit like factorio with the extra dopamine hit of getting to unbox stuff.

K8s is painful to manage. It's a lot less painful than getting paged in the middle of the night because your server is down - And much much less than realizing that you've been down for an entire day and didn't notice. (K8s isn't even a complete solution to these problems! Just one part of a complete ~balanced breakfast~ production stack)

You don't need k8s for all of that, but there's not a simpler solution than k8s that handles as much.

Life is full of pain. Deal with it.

It's because it is complex. And in the long run, things become simpler. The only difficulty is the initial setup and once you are past that, the overall maintenance workload just becomes easier compared to a single VM setup

> And in the long run, things become simpler.

aka, you're front loading the complexity.

You can even think of it as paying insurance premiums upfront. You get to "make a claim" if the requirements do grow into the sort of need that suit such a cluster/complex setup.

But, on the same insurance theme; I am not sure paying 10K a year to insure my 5K car makes a lot of sense, because, in the long run, I might write my car off.

> I think a lot of people do things the hard way to learn large scale infrastructure

Having seen some of these half-rolled, first-time-understood k8s deployments, and the multi-year projects to unravel the mess that was created, overflowing with anti-patterns and other incorrect ways of doing things, I think I would prefer a narrower scope of true experienced professionals (or at least some experienced pros that can help guide the ship for their mentees) working on and designing k8s infra.

And for those that don't need it (the vast majority of startups, small businesses, regular-sized businesses, etc), just stick to the easier-to-use paradigms out there.

Nubank, the Brazilian bank unicorn, described their approach as “if this works, it’s because we reached massive scale quickly” (paraphrased) and started with an architecture that would support that from the beginning. They were very happy with their choices and have blogged about them in detail.

This is a case where “things will be much easier when we scale to a massive number of clients” turned out to be true.

Resume driven development is worth learning to recognize.