My 2 cents on Chef is that's it's long dead. The handful of people I know who used chef professionally all gave strong negative feedback about it and moved away to ansible.
While I can't comment on the exact problems because I didn't use it personally, there was a recurring mentions of losing states, seemingly destroying systems and configurations instead of setting things up. People were adamant that moving away from Chef was the best thing they've done lol, which I don't know about but I take at face value after the third person saying the same thing.
While I'm not a proper sysadmin, Ansible is what I gravitated towards (and use for my own projects) and always seemed like the best choice out of Chef, Puppet, and Saltstack.
Ansible is/was straight forward to use and understand, very F/OSS, relatively easy to orchestrate (even without tower), and had a nice ecosystem. Ansible always won for me because it was simple, and had just enough (though I always felt that the roles/tasks analogy was kind of awkward) and was easy to hack. These things were mostly true about Chef, but outside of being "ansible, but ruby", I don't remember distinctly what made Chef worth doing other than if it was there before you got there.
Someone please correct me if I'm wrong but IIRC Salt was one of the first to start venturing into the infrastructure provisioning realm and ansible went there too relatively easily though it's somewhat obscure[0]. Ansible and Chef (and presumably puppet) were quick to follow/build similar tooling but we all know who won that race -- Terraform.
Funnily enough, Terraform is now making it's way back around to eat the post-deploy logic as well with it's provisioners[1]. Can't wait to see HCL grow into an even more grotesque monster.
Another approach that we have been exploring with Terraform is to treat the machine configuration as resources[1]. It works pretty well when most of the system is vanilla with a few additions to it (like installing Docker).
Thanks for making this available as open source -- I was about to note that this is exactly what providers are for, then read the use-cases and the support/API surface you've crafted, this looks super useful.
I was anti-HCL/Terraform for a long time due to issues with state files, but lately I honestly prefer it over Ansible whenever possible.. although I'm still not happy with the way cloud-init works on various cloud providers.
I'd like to recommend that you try pulumi. Terraform is definitely a fantastic tool, and it really does have just about everything you want (and is extremely google-able with easy to find experts), but I think the bring-your-own-language paradigm is better for this particular use case if a few things are true:
- Your team is comfortable with at least one of the languages Pulumi has native support for, ideally your codebase/infrastructure repo is already written in it.
- Someone on your team isn't going to hang everyone else and themselves with the wide open nature of a fully featured programming language at their fingertips
- You've got some innovation tokens to spend
I think HCL is going to get more and more like a fully featured language as time goes on -- if they ever lean in to it (and introduce support for alternative "language runtimes" or something) then I would go terraform and never look back.
It's hard to recommend pulumi over terraform because of momentum, ecosystem, etc, but it does seem like a better base to start off of (and it's what I use for my own stuff).
Thanks so much for the pointer, didn't know this existed -- it's exactly what I was hoping they might do.
> Today, we are pleased to announce the community preview of the Cloud Development Kit for HashiCorp Terraform which allows users to define infrastructure using TypeScript and Python while leveraging the hundreds of providers and thousands of module definitions provided by Terraform and the Terraform ecosystem.
> CDK for Terraform is in alpha and is in the early stages of development. This project is another way to interface with Terraform using languages like TypeScript and Python. We are working to support additional languages such as JavaScript, Java, and C#. In addition to language support, we plan on expanding the scope of CDK for Terraform project to support other first-class commands to provide a user experience similar to the AWS CDK.
This has to be a shot across the bow at pulumi, glad Pulumi innovated and took this direction (maybe CDK was on the roadmap at Hashicorp, not sure) and seemingly spurred TF taking the turn as well.
HCL, Jinja, Go Templates, etc are much easier for people to read and understand than standard programming languages. If you're going to use a standard programming language to write your IaaS then you might as well just use the native SDKs rather than an overlay.
I brought Chef into the online side of the NFL several years ago. We still use it, though less and less as we get more things into containers. I've got my gripes with it, but I'd never second guess my decision to use Chef, or trade it for anything else.
Those problems sound like people aren't taking the same care in their cookbooks that they are in their regular software. If you write bad cookbooks and don't at least give a cursory review of the community cookbooks you use, you'll end up with a mess. If you treat it like the customer facing software it is, I find it a pleasure to work with.
Our cookbooks have CI pipelines that run some basic tests and we have automation around rolling out changes. We rely on a lot of conventions to avoid storing state, but we still have nodes auto provisioning their own storage and load balancers where necessary. It's let us stand up a ton of infrastructure and make changes safely and quickly. That's not to say we couldn't have made Puppet/Ansible/etc work, because IMHO it's more about your development practices than anything else.
edit: we only use Chef Infra and some Inspec. Not talking about Habitat.
I'm using Chef since 2010 for myself and customers. I never understood the hate but I saw so many developers not getting it and/or overengineering it to a state which was beyond their capability to debug.
For example the different levels of attribute types. I saw "senior" developers writing nasty Ruby code (actually `system("rm -rf directory")` to delete directories (outside the chef DSL) because they didn't understand the appropriate chef resource.
The market decided to take away the power of a real programming language DSL as an interface. People are now forced to deal with terraform (HCL), YAML or JSON. It's a lobotomy.
`The market decided to take away the power of a real programming language DSL as an interface. People are now forced to deal with terraform (HCL), YAML or JSON. It's a lobotomy` I feel this so much. Granted HCL is taking steps to become closer to a real language but its frustrating that some of the features that I used to take for granted are missing (why do we have ternary if statements in HCL but not real if-else blocks. I used to enforce a rule that you should not have nested ternaries, but now that's the only way to do if else if statements)
I can confirm that the NFL Chef cookbooks were great. I worked at the NFL and previously introduced Chef at a startup beforehand. The biggest issue I had at the startup were application developers cutting corners when creating ANY infrastructure code, not just Chef recipes. The devil IS the details when it comes to infra code, and the lack of detail, not the tool itself can really bite you.
I disagree that it's "long dead". Keep in mind that Chef (the company) has several different products too...Chef Infra (traditional configuration management) has fallen out of favor and Ansible seems to be the popular choice these days, but there are still a lot of big Chef Infra shops (e.g. Facebook [0]) that have invested heavily into configuring their environments using Chef Infra, which would require a good bit of investment to switch tools.
Plus, they have been diversifying their products in recent years with things like Habitat (application packaging/deployment) and security/compliance tools (InSpec).
The fact that I've never heard of Habitat or InSpec means either they're extremely niche and not as popular as I think Chef would've hoped, or I'm completely out of touch with both app packaging and sec tooling.
I'm hoping it's the former, but I wouldn't be too surprised if the latter since the 'enterprise' ecosystem of cloud tooling has grown exponentially in the past decade.
You're not out of touch, Terraform + Containers + People realizing that building immutable things is much better than managing state pretty much killed Chef and Puppet.
People love talking about their state being immutable when in 99.9% of the cases it is not.
Here's a simple test: if your runtime container infrastructure is immutable, you should be able to run your container with a file system being read-only. If you cannot do that, then you do not have an immutable infrastructure and hence you will need need something to manage the state. Ignoring it is akin to kicking housekeeping tasks to some point in future and hoping that before you get buried under the multi-year backlog of tiny corner cases before your entire product is replaced with something else.
People still have to manage quite a bit of setup for kubernetes/containers though... and that configuration usually happens with ansible, not chef. I think it's fair to say that Chef has failed to hold their market.
After having spent the past decade across about 12+ companies in various capacities furiously trying to make software immutable and to isolate state to dedicated services like DBs, message brokers, and networked caches, I’m not sure if it’ll go away without many companies rewriting software from the ground up effectively to support cloud native-ish assumptions like 12F but much more like avoiding leaving transactional state on-disk that can’t be recovered. There’s a TON of software out there that is written like it’s the 90s still. The market for managing legacy stateful software is matured to its finality and nobody wants to try to attempt to revolutionize it anymore is what I’m seeing, not that nobody is doing it anymore. Essentially, if one is writing new software that isn’t containerized in 2020 you’re writing the systems equivalent of COBOL.
But anyone could see many years ago that it’s easier to flatten and reinstall stuff than trying to painstakingly manage various moving pieces. We were limited by our technology unless one was using stuff like zones in BSD or Solaris.
The surprise for me has been how rapidly huge behemoth organizations like the DoD has shifted to technologies like K8S when they took so many years just to get onto VMware and some configuration management.
Holy shit; if /u/geerlingguy hasn’t heard of Hab or InSpec, then Chef really could’ve done a better job marketing them. (This user is very well known in the config management space.)
InSpec is really a good tool you should know. I believe you can write molecule tests with it so it’s something you may have already used at this point and not realized it.
Having used both. If I look around I see Chef used less and less and Ansible more and more but that does not mean there is a valid path from Chef to Ansible when you way more than a trivial number of hosts. I'd say go from Chef to Saltstack. If you only have a few hosts, sure Ansible.
I dealt the Progress from about 2003 to 2008 or so. They were an absolute horrible company to work with and essentially held us hostage on an old platform because we refused to pay an exorbitant "platform change fee". And by exorbitant, I mean HUGE. Fortunately, we sponsored an open source OpenEdge ABL and DB replacement project (FWD - https://www.beyondabl.com) and have been running our converted project very successfully since 2009. We went from OpenEdge / Progress DB to Java / PostgreSQL in an automated way. For anyone else held hostage by Progress Software because of your reliance on Progress 4GL (OpenEdge ABL) I encourage you to take a look.
Perhaps the company itself has improved since then. I certainly (for Chef's sake) hope so.
It depends on the price they paid for stock or the exercise price for stock options. If you were an early employee and your received options when the company was only worth $10m or something low, you could make money.
The risk is that one of the later investors had a ratchet or something that would allow them to claim more of the proceeds in a sale. You can't just take $220M cash, subtract $105M in funding and pass that to the founders and employees. The preferred shares were probably "participating" meaning they get a portion of the common.
I don't think the investors (at least the later ones) made meaningful money at that kind of acquisition price.
Founded in 2008, that's 105M for some fraction (maybe a majority) of the equity... minus fees / taxes / legal costs, that's a pretty lousy investment, especially compared to market returns over this timeframe.
What is Progress’s track record with open source? Recently a lot of people have had to switch to Cinc if they wanted to continue using Chef without paying an enterprise license.
As one of those people, I just hope they don’t try to close the doors on the Cinc community.
From my viewpoint, Puppet's portfolio is comprised of mostly half-baked stuff. You have Puppet itself, which is decent, and then an almost Google-level of weird stuff. They keep announcing new products but so far nothing seems to be sticking and they're all ludicrously expensive IMHO.
Meanwhile, their core technology is kind of a dead end. More and more people are looking to manage containers and other serverless type things, rather than managing fleets of VMs. They're not in the same boat as companies like VmWare, but they're in the same flotilla. If somebody does buy them, it'll probably be an act of dinosaurs mating and not much else.
Being a Portland techie, I pay a tiny bit of attention to Puppet and I’d say I agree with your comment re: product portfolio, until about a year ago. They have actually killed off a number of things in recognition of this critique and are trying to bring back focus on a sane portfolio strategy.
Even for those of us still managing VM fleets, containerization means that we can run a single OS image for everything, eliminating most of the need for a dedicated CM tool.
Puppet is a great tool for managing containers, and so is Chef. There is nothing inherently good about YAML.
There is less buzz around those tools now however, and people tend to gravitate to other tools which are easier to get started with.
They may end up "silent heroes" (much like Ruby itself, I would say) or they may end up replaced by other tools. The concept of state as text in version control is a timeless concept and continue to exist in one form or another.
One of the problems with configurations management systems is that they are conceptually easy, everyone has opinions of them, and they really need to control your whole world in order to shine. There was never really room for one clear standard to emerge.
Puppet Inc.'s problem was that they kept chasing shiny stuff (agentless you say? we have that too!) in order to woo enterprise customers. Their customers should have been developers and sysadmins, not their bosses. There is only room for so many VMware in the world.
> Puppet is a great tool for managing containers, and so is Chef. There is nothing inherently good about YAML.
I am not sure what complex config state management solution brings to the table for immutable containers. The state is immutable, so there's nothing to manage after deployment. It's like using an atom bomb to take out a zit.
Puppet/Chef in their glory days, pre-cloud pre-container, excelled at managing desired state across numerous mutable static systems. With immutable solutions, Puppet/Chef are both cumbersome and expensive.
An application running as a container bascially carries the same configuration as one running without the container bits would. That may include things like endpoint addresses, certificates, secrets, but can also be things like feature flags, authorizations, API tokens. Basically anything that's not compiled in is configuration.
The deployment side also carries configuration, including things like desired amount of instances, request routing and filtering, log destinations, log retention, persistent volume sizes and location, backup rules, metrics and monitoring rules etc.
If anything, application deployments today carry more configuration, not less. Fifteen years ago, half of the above didn't even exist. Perhaps you pointed you application to a syslog server and that was it.
All of this configuration exists in disparate tools (JSON files, YAML files, firewalls, metrics dashboards, cloud providers proprietary APIs) and will over time slowly turn into a sprawling mess. Bringing control over this into a central repository is a good thing.
Not sure about expensive, as the aim is maintainability and reduced complexity, but these tools do tend to get cumbersome. After all, they want to do everything. It's a fundamental problem. It's not surprising that has led to a surge in less capable tools.
> That may include things like endpoint addresses, certificates, secrets, but can also be things like feature flags, authorizations, API tokens. Basically anything that's not compiled in is configuration.
But Kubernetes solves most of this problem in an easier way with Pods, ConfigMaps, Secrets, Services and Endpoints.
Yeah, but what's left after you slim down your VMs to a Linux kernel, an SSH daemon and a container runtime? Is a CM system really justified at that point? Why not bake VM images and treat the VMs as immutable?
Testing Puppet manifests locally is (was) THE WORST. Hiera and r10k really shouldn’t exist in 2020. IIRC you essentially need long-lived Git branches for managing different envs for different playbooks.
Ansible is just SSH and a binary. pip install and I’m done. This is great for building VMs, base Docker images, app deployments, whatever you need. Puppet has their own wire protocol and is very dependent on a client-server relationship. You need to edit config files and fake a puppet master. That’s work.
The each their own, these tools are conceptually simple but as you found out, there is a lot to learn.
What kind of branching strategy is most suitable is given from how your testing is organized, not which particular tools are chosen.
Ansible is completely equivalent to Puppet or Chef in this regard. They are all the same family of tools. Many find Ansible easier to get started with due to its agentless defaults, but there are some (perhaps Spotify was most well known at the time) who run Puppet agentless too.
Having a server, something like Tower, is necessary as soon as you want to manage distributed configuration for clusters, and if you manage monitoring and backup systems.
> Puppet is a great tool for managing containers, and so is Chef.
Eh. While I won't say that it can't be used, I will argue that it's not the best tool for the job. I wouldn't call it a "great tool" for that purpose, although I suppose that depends on your workflow. I prefer to take the approach that containers are immutable once built. If you want to change the container, fine- make your changes, build a new instance, hotswap the connections over, and bring down the old one. There's no need for any sort of config management in my eyes, at least not in the traditional sense.
Now, Puppet could be useful for things like managing the VMs that the containers are running on, but I would prefer those stay immutable too, and with containers, they easily can. I'm (sadly) very aware that not all enterprise software can fit in this model though. Enterprise always lags behind, but it will get there. Hell, even SAP is heavily exploring containers. It's only a matter of time.
Not sure what the argument here is. Different builds for prod, qa and test?
Even if that was workable, which is a question in itself, there is still configuration outside the packaged application. AWS keys, metrics and logs, dashboards, that sort of things.
Well, the container manifest (or whatever you call it) is checked into version control. How you choose to deploy that is up to you, but a ton of CICD tools out there are made for just that purpose.
Likewise, how you separate config data and deploy to production is an age old problem with many solutions as well. Sure, Puppet can even be a solution here, but you're just kicking the can down the road. How do you separate your Puppet code from data? Put it in Hiera? OK, how do you promote Hiera? It's the same issue.
Both of these issues were solved long ago before Puppet came along, and what we all learned from back then still applies: separate your data, config, and code. Version control as much as possible. There are many simple solutions that don't require Puppet. Hell, set up a bind mount to an external config store. Or something fancier like your CICD tool bundling the config data and signing it with a cert that only a certain environment can use so you never get dev code running in prod.
The argument is that when your infrastructure is immutable (which is the direction most of us are going) you don't need a tool to manage the configuration of said infrastructure. Because it can't change. When you do want it to change, you just follow the typical code SDLC and use the powers of version control and CICD.
It is absolutely workable to write a script to bundle data with your application. And one that exports configuration data from your data store to configure your cloud provider. And one for your monitoring system. That's where we all started.
Should you at some time replace all this with a common data format, then congratulations you have invented a configuration management system very similar to Chef.
The use of a version control system or a CI/CD system does not somehow negate the need to keep configuration, neither does a configuration management system do the job of the former. Nobody builds their software using Puppet, they use something like Jenkins. In fact, I would consider having VCS and CI/CD in place a prerequisite to any type of configuration management.
In total over 12 years, Chef raised around 105M and puppet raised around 189M in funding.
They're both doing pretty bad in the end and are on a (steep?) downward trend. They'll never be a billion dollar business. They probably won't recoup their valuation. Chef sold out. Puppet tries to survive.
From my perspective Chef users were actively fleeing so I guess it was sold for scrap, puppet might be downward but maybe not as steep so maybe puppet is trying to continue to operate as a small medium company. Also Puppet might be a fairly harder sell because it has both a higher valuation and debt, doesn't look like a great buying opportunity.
They made some serious missteps imho re: strategy. releasing and then not marketing razor server when everyone was shifting to VMs, halting development of the open source dashboard, not really investing in containerized environments, kubernetes, etc.
Simple to deploy, configure, and use. Although I always felt I should perhaps have based my work against the puppetdb to get a more dynamic "dashboard" it certainly replaced the old dashboard, and being only a single binary it is simple to install & manage.
I was surprised when I saw the Chef numbers, ouch. I think they must have been simply running out of money and options. If they had a strong balance sheet, you would have expected them to at least be picked up by a reputable PE firm. I hadn’t even come across Progress As a software co till this announcement.
On Puppet, I think they are a different beast to Chef at this point. They seem hyper focused on the larger enterprises rather than SMB or startup land. I came across them at both of the banks I recently contracted for, for example. Whilst I wasn’t hands on myself, there was a big project to roll out Puppet’s impact analysis and CD product at the last place.
I also ‘think’ Puppet are about double the size of Chef and saw their BlackRock debt round announced. You don’t get money from someone like that unless your at scale and you don’t get debt (especially at a time like this) unless you can afford the payments+interest. So, they must be at least growing if not profitable as well.
Progress has an interesting angle of providing a suite of developer tools with their products crossing the categories of both design-time and run-time. There doesn't seem to be too much focus at this point though, in comparison to JetBrains for example. I'd imagine more acquisitions are coming.
They have the Telerik suite of tools, which is used by .Net developers, but how many .Net developers use Chef?
Anecdotally, I don’t know anyone in the windows / .NET world that is using chef, or have used it and stuck with it. Most people on that platform seem to gravitate to Chocolatey. That doesn’t mean there aren’t any using Chef primarily but it’s pretty rare it seems.
Chocolatey and Chef aren't rivalrous. When I've managed Windows/Linux mixed environments before, I did use Chef, and I used Chef to install Chocolatey packages.
Chef isn't just CM, it's also a lot of policy enforcement tooling these days. I don't have much use for it anymore but it does have its niche.
Chocolatey is a proxy-package manager, automates fetching packages off the Internet installing them (often through the packages own .msi or other installer). So it doesn't compete against Chef any more than yum or apt-get or Homebrew competes with Chef. Chef supports Chocolatey directly.
That being said, the experience on Windows is abysmal. The implementation with ruby on Windows is sloooooooow. You wait seconds just to get things like version.
Agreed. Which is why I find it odd that their products are in a mismatch of verticals. This strategy only makes sense if more acquisitions are on the way.
Fwiw Progress is most notable as a provider of OpenEdge, a database/development hybrid that was interesting/infamous for its contrarian 4GL based approach.
While you could access the database via SQL that was very much the second class citizen; it's native 4GL was fully integrated, and long predated attempts like Linq to hide developers from SQL.
Progress also up until recently stewarded the excellent Nativescript[0][1] project -- though it doesn't always get the most press, I maintain that it is the best non-native development mobile development kit.
Since it seems to be really hard to find this information, want to make a copy of the content of the second link ([1]):
> A New Corporate Sponsor for NativeScript
> Progress is proud to introduce nStudio as the new corporate sponsor of NativeScript. After fostering a robust open source developer community, Progress looks forward to what nStudio will bring to the NativeScript community and customers.
OpenEdge has pretty good SQL support. Progress management I talked to did NOT want to stand behind SQL for CRUD applications. They were OK with SQL's use for ETL and were adamant that OpenEdge should be used only with 4GL for CRUD application. I wrote several blogs to explain the cool CRUD apps one could write. I did observe that 4GL devs were very shy to use anything but 4GL to work with Progress. Yes, SQL is treated as a second/third class citizen even though I felt SQL was an exceptional way to use OpenEdge with other languages. If there were valid technical reasons to not use SQL, I didn't hear it from 4GL devs or Progress management.
To be fair, 4GL and OpenEdge work very well together.
> Progress management I talked to did NOT want to stand behind SQL for CRUD applications.
Of course not - the entire business model behind OpenEdge (which is far from "open") is to have applications built with their laughably irrelevant ABL/4GL language and only working with data in OLTP / one-record-at-a-time because that makes it almost impossible to switch to the conventional (RDBMS + web-service + frontend) architecture that the entire industry shifted to 20 years ago.
OpenEdge (nee "Progress DB") was a cool platform in the late-1980s through to the early 2000s because it supported IBM's AS/400, MS Windows, and Linux - with their UI system that let you design an input form once and have it magically work in text-mode AS/400 terminals and the Windows desktop - but the fundamental design of OpenEdge is still based on its AS/400 roots and it really doesn't work well with modern systems (e.g. it has this design with multiple "broker" processes which is a PITA to configure).
Their anti-open-source article on Progress' website that the GP post linked to was infuriating to read: it felt like the same anti-GPL propaganda that did the rounds around 2003 when SCO was claiming Linux contained their copyrighted code and Microsoft was astroturfing articles about the "risks" of open-source code and seemingly intentionally confusing people about MIT/Apache vs. GPL licenses. Le sigh.
They might claim though - as in the link below - that this is because their products are too niche to support quality input from open source. Make of that what you will.
In my experience, Progress is not very OSS friendly. Prime example: they don’t even have an open CLIENT driver to connect to their database. You need a license just to connect.
"We have a long history of supporting open source communities so I am proud to say that we will continue to nurture the vibrant and thriving Chef open source community."
I fondly remember learning and using Chef back in '10 when it was all the rage. Seemed to work really well given the basic AWS architecture (no containers & microservices) that we had at the time. Also notable was the thriving and friendly community that Chef had going for itself back in the day.
Progress ran on Unix varieties and Windows, had its own 4GL language that you built apps in to talk to the relational side.
Ran some pretty big shops like Ideal Electrical/Burson Automotive etc here in Aus.
This was 20 years ago mind you...
I used to work on govt project that uses a Windows stack and advocated for implementing chef. I feel like Chef totally dropped the ball in terms of marketing / building features out for windows. The documentation, community and reusable recipes was ahead of it's other competitors but needed a lot more. I went to 2 chefcon's and almost all keynotes and workshops were linux focused and every time I brought up a question that is remotely government or windows focused I feel like I'm being treated like a 2nd class user. Government is notoriously slow to adapt to new changes / software, that could've been a space where they could grow with the popularization of containers. There are a few projects I consult on where they still can't / won't use containers.
With that said I'm glad I moved onto a *nix based project and got to learn Ansible and docker
Just to clarify, that would be buying a company with "$70 million in annual recurring revenue" [0], for the price of "$220 million in cash, subject to customary adjustments" [0].
We don't know anything about the profitability or the growth. I can imagine that their growth is flat or even declining.
“100% growth in incremental recurring revenue” doesn’t say much if they are churning a lot of existing customers. (That they chose such a double-speak revenue number perhaps says a lot). My impression was that DevOps software like this is very hard to pull out once you’ve embedded it in the org.
I can give you SF Bay Area perspective, where Ruby and especially Rails is quite popular. With the arrival of Docker and Kubernetes, immutable infrastructure patterns dramatically reduce costs, and the need for a centralized change management solution is nullified. At deploy time, some config and templating is useful, but Chef becomes overkill. People would want to use Chef-Zero, but Ansible in this scope have become wildly popular.
Chef tried to adapt with Chef Workstation to do push-based config, but has inability to extract state from the system configured. The target system would have to update state to a server, which is fetched indirectly. This doesn't scale, so this is another reason why Ansible and Salt are popular.
Puppet also experiences some of the same issues with Ansible eating their lunch and popularity of push-based config for immutable infra. They attempted to respond with Bolt, but Bolt is based on static hostnames or DNS names, which won't scale given dynamic nature of cloud native and transient ephemeral systems.
In the case of managing fleets of systems that are not atomic stateless nodes, where you need to maintain a state across nodes within a set, both Puppet/Chef do not scale, and create outages (though window is small), because they have to synchronously push state to a server, and rely on eventual convergence. This doesn't scale in cloud computing. With push based config, you can set the cluster into the proper state, and then use service discovery (asynchronous updates) to maintain the state of the cluster. In K8S, kubectl/helm would fill the push role, and etcd used to maintain state. Outside of K8S, such as lambda in cloud, pulumi/terraform could push state, and discovery through cloud metadata (labels, tags, etc) or service discovery like consul to maintain the state.
Chef and Puppet could have responded, but couldn't see past their own platforms that are based on managing desired state for groups of individual atomic systems. They also failed to monetize on things like inventory management that enterprises fork over a lot for such things.
A litmus test: who writes Chef recipes in your org? A relatively small group of engineers responsible for infrastructure provisioning, or any engineer who wants to deploy software? The latter is a pathological case and super destructive to engineering teams, but in my experience almost inevitable if you use Chef for the former.
The alternative is ansible. Everybody I know moved away from Chef to Ansible and never looked back.
Historically there were 4 main configuration management tools: puppet/chef/ansible/salt.
The first two faded away. They're unarguably not the most usable and require to program in ruby. The next generation ansible/salt came out and arguably did better on every aspect.
I think it's fair to say that ansible is the safe standard (redhat acquired it few years ago, they maintained and extended it pretty well). Ansible is very easy to use and to integrate in anything as long as you have SSH access.
The only downside of ansible is that it can get slow with thousands of hosts to manage (gotta establish SSH connections). Salt can do better with large amount of hosts.
Ansible doesn't necessarily need ssh or a central control node, just a place to store roles as artifacts, such as Artifactory or a storage bucket.
Pass all of your variables and dependencies (role artifact urls, python requirements.txt, etc) to machines through user data and have the machine download the dependencies and run ansible-playbook to configure itself. Optionally, add a systemd timer to run it on a schedule and you have an "agent".
I've done it this way a large percentage of the time using ansible. The other way is by baking images and using ansible as the provisioner (using ssh but only during the image build).
Some people still like promise theory and the execution model behind chef and puppet.
I don't want to orchestrate against hundreds of systems with SSH and tell them what to do. I'd rather tell them what I want, and let them sort it out locally.
IMHO opinion, another big benefit of ansible is how usable it is from the start. Granted, you can build some bad habits if you're not careful, but being able to step into ansible and use it for stuff without a big building project is a huge advantage.
> The alternative is ansible. Everybody I know moved away from Chef to Ansible and never looked back.
That has been my experience/perspective as well. This was what I found industry...
* ~2012 Puppet golden years
* ~2014 Chef golden years
* ~2016 Surge of popularity for Ansible and Salt.
* ~2018+ Kubernetes ubiquity, Terraform for cloud, Ansible for systems
Related to this, saw popularity in SSR (server-side rendering) with Rails before 2012-2016, and after 2016 rise of popularity in SPA (Angular, React, Vue) on top of micro-frameworks like Flask (Python), Express (Node), GoLang, others. Combined with this are ML and other backend infra that requires managing clusters that scale better on Kubernetes, where Chef/Puppet have little presence on either K8S or backend distributed clusters.
A lot of recommendations for Salt here, but wanted to point out they have had a pretty poor track record of security vulnerabilities. Something to keep in mind when evaluating alternatives.
There's pyinfra (https://pyinfra.com, full disclosure: I built it). Ansible, Salt have been mentioned already. Also CFEngine and Rudder might be worth a look.
Isn't that like killing a fly with a sledgehammer?
You're proposing that he replaces a cross platform tool with an entire OS and on top of that with a tool that has a very peculiar package management system (I'm not saying it's bad, just that it kind of requires full buy-in to make it work).
Tried installing NixOS a few days ago. Live CD created an appealing desktop environment, could not get it to install on a partition though. There is no installer to speak of, I followed the instructions closely but ended up with a starter configuration that lacked network access (which was required to install the network components; ikr). gave up.
In the scope of config state management across mutable systems, SaltStack is the closest.
When doing immutable infra, where managing desired state is only at deploy time, then Ansible is by far more popular. Beyond that you get into container scheduling-orchestration platforms.
Others mentioned Terraform and Pulumi, which manage cloud resources, where Chef manages mostly system resources. Once upon a time, there was an attempt with Chef Metal, and later rebranded as Chef Provisioning, which are drive by chef-solo (or chef-zero) and use fog driver. The end result was something that was so gawd awful terribad, unstable, and unpopular. Instead of Chef investing to improve it, Chef started promoting Hashicorp Terraform over their solution.
Silly question: why do you need an alternative? All of Chef's software is OSS (licensed under the Apache 2 License).
CINC is the recent-ish community build of the code. It's literally a drop-in replacement as the only changes to Chef Client and other bits are branding. It's the CentOS to Chef's RHEL.
Skip AWX entirely and just use Jenkins. As insane as this sounds, I've personally done Jenkins => AWX => Jenkins twice. A good shared library can entirely replace AWX and the level of authn/authz plugins with Jenkins blows away any granularity with AWX.
What’s the track record of Progress buying companies? Is it a case of Oracle or IBM sucking the life out of them due to lack of investment, and living off the installed base? Or something more positive?
I have friends working for Telerik which was bought by Progress. They feel pretty happy so far and Telerik are doing well, though they had to drop native script after react occupied the entire compile js2native space.
> What I dont know is if that is common in that "ui component" sector?
I think this is why almost nobody uses closed-source libraries. Having to pay isn't the biggest problem: it's not being able to fix issues like these yourself.
This 100%. Earlier in my career I managed a bunch of VB developers and we used a number of commercial UI components. Without exception they were a nightmare to deal with.
Telerik seems to be trucking right along and it's been years since they were acquired. I've seen no drop in the quality of their products and support so far.
Huh. My first job as a developer was working with Progress' 4GL language/database/platform thing (now called OpenEdge I think?). I ended up writing an basic Emacs major mode for it.
Disclaimer: I cut my teeth on Progress 4GL back in the 80's, which is now called OpenEdge. It tickles me to see them still around, and to have bought Chef? Well, that's just sugar.
Basically, they're a business apps technology provider, and have been around since 1981, providing app development tools for enterprise scale applications. Progress 4GL was a database-backed application environment, quite widely used (albeit quietly) by big businesses all over the globe to build apps, and has transformed into OpenEdge.
As the Web evolved, Progress seem to have embraced all of it, and - as you say - somehow seem to be using smoke and mirrors to describe what they do - but actually their tooling is pretty awesome. Write an app in OpenEdge, deploy it on any web server. They were doing big relational database stuff on cheaper hardware (PC's, minicomputers) in an era when "big database" meant investing heavily in "IBM" or "Relational Software" (now: Oracle) resources.
Its pretty neat to see them still around. I had a lot of fun with Progress 4GL back in the day, but haven't touched it since mysql/postgresdb was released ..
Ditto this. I was a ASP/VB-then-.Net developer 20 years ago and happened upon Progress 4GL when I had to create apps either communicating with or extracting data from Progress sources. My only beef was the horrifically quirky ODBC driver (and the convoluted SQL Server DTS packages & openqueries that resulted) -- 4GL was rock solid on AIX, and also on CentOS/RHEL once we moved away from AS400... and this was for high volume factory execution systems at a global EMS firm. Around 2010, we started a migration from Progress --> PostgreSQL+Java, approximately when the majority of the original devs began retiring. It was painful and the resulting "new" versions of our apps & DBs were universally crappier, even if "easier" to develop for and manage using "agile" methods. I frequently miss the old days, when programming was WYSIWYG and everyone explicitly understood how computers worked.
My job for years was developing and supporting Progress applications mainly on AIX. So much less crap to deal with than is common today. Things actually worked.
My experience with AIX as a sysadmin ca 2005 was dramatically different.
Things "worked", but everything was OLD. CLI tools had bugs and warts that GNU figured out decades previously.
I remember having to use obnoxious workarounds when tooling broke because 'find' wouldn't even work properly once you exceeded a certain number of files. (Several thousand, IIRC)
And that curses-based wrapper for doing administration tasks... glad I forgot the name of it, to be honest.
Getting Bash and the other GNU tools to run on AIX 3.2.5 was definitely a chore but I still miss those days.
The culture was a lot more active in the 90's. I mean if you were trying to port something to AIX and having trouble with gcc, you could go on comp.unix.aix and IBM employees would actually respond and be helpful.
huh. That's not been my experience. To me, it looks like they stopped improving things circa 2005. Things that are new tend to be half-baked (.NET integration, OOP language features). No command line compiler (although you can write one yourself and a java-based community project exists), no linting or other static analysis tools. Their IDE is eclipse-based and frequently crashes when working with GUI for .NET. The autocomplete and in-IDE documentation that it provides is terrible- missing methods, not context aware, etc. They have a docker image, but only for their app-server product. I'm currently building a REST API using their app server product and I keep running into stumbling blocks. I've built similar APIs in PHP and Python and those were (largely) easy- I could focus on the problem and not the tooling. Here, I keep stumbling over things that work differently than anywhere else or that aren't well documented at all.
Their earlier stuff is quite impressive when you judge it by the time. They just seem to have fossilized.
Progess is better known for their Telerik controls suite - many .NET developers use it in their stack as it greatly improves upon the builtin controls available in both winforms and asp.net
Progress offers several products I've used. You can build apps on their framework (OpenEdge) that manages all of the services and data storage. They also sell component libraries for .NET development. WhatsUpGold is a network monitoring tool. Telerik Fiddler is an awesome thing they own that lets you inspect network traffic on a local machine by proxying everything.
Oh, they make WhatsUpGold? That actually gives me more confidence in them - it's been a simple but reliable monitoring tool that I've seen in enterprise use for many years.
Back in the day, Progress offered hands down the best platform for developing multi-user database-centric applications. They still sell it, but the product is now called OpenEdge and the company is called Progress (it used to be Data Language Corporation).
They have a lot of other products now as well.
Their website is marketing puff but the products are great.
OpenEdge is their DB. It has 1 storage engine but 2 ways to get to it: SQL and 4GL. If you write SQL, OpenEdge is relational DB. If you write 4GL, OpenEdge isn't relational unless you write relationship on application layer. Yeah, it's interesting.
4GL (aka ABL or Advanced Business Language) is the language that works best with OpenEdge. You write 4GL code, compile and deploy it to a specialized Tomcat environment called PASOE (Progress Application Server for Open Edge). It's their container to run 4GL apps.
WebSpeed is their middleware to serve APIs using 4GL code. Companies stuck in the desktop world are using WebSpeed to refactor their code and expose business logic to other languages through WebSpeed API middleware.
They acquired Telerik some time back. Those controls are used by .NET web devs and desktop devs. They are cool controls. I like them. Telerik controls are also integrated with their 4GL desktop environment. Kendo UI are tools/controls for the web and are pretty neat as well. Fiddler is also their product now.
DataDirect is a suite of connectors to make it easy for devs to connect between different DBs from 4GL, and for other languages to use DataDirect to connect to OpenEdge or call 4GL methods.
Corticon is their version of Dell Boomi or Alooma or Prismatic.io. It's a tool to run business processes with no code or low code. It's neat if you have tech savvy business analysts who like to use tools like that.
Sitefinity is their version of WordPress/Drupal/content frameworks. Sitefinity has its place.
NativeClient is their product to build mobile-first application across various platforms. They whipped up a neat app at their conference and it really worked well. Yes, it may have been cowboy-coded a bit, but it worked as intended.
Now they acquired Chef. I am sure there was a demand for orchestration within their client-base. They are making improvements steadily. They have a strategy. They have a niche target market, primarily in LATAM and EU and less so in North America. They do not have a great version control, static analysis, dynamic analysis, orchestration tools. Companies like Intuit and Fiserv seems to have built their own ecosystem around 4GL tools, but many companies just have primitive means of development with 4GL, still. Progress is trying to stay current. Their customers are much slower at adopting to changes in engineering; perhaps what they have just works for them. Not sure.
Acquiring chef is a good move. I've worked with their products for 2-3 years. It wasn't fun. I don't like 4GL. I'd like if Progress made SQL a first class citizen and invite PHP/Python/Ruby/etc. communities to write code against OpenEdge. From a business standpoint, I feel they are trying their best to stay modern and relevant. They are acquiring products that are indeed making their ecosystem cohesive.
If you work in more traditional Windows-based IT, you know Progress. But yeah, they're not startup-y, it's unsurprising if the Silicon Valley crowd isn't familiar.
They are very well and truly entrenched in the enterprise application industry. I know a few medium-sized businesses that have maintained their Progress 4GL -> OpenEdge code for decades now.
That's because they are one of the companies that prints money by charging enterprises to do non-sexy things that startups don't find to be very interesting.
While I can't comment on the exact problems because I didn't use it personally, there was a recurring mentions of losing states, seemingly destroying systems and configurations instead of setting things up. People were adamant that moving away from Chef was the best thing they've done lol, which I don't know about but I take at face value after the third person saying the same thing.