Recipe reuse in chef is very much at the source code level
- checkout a recipe and edit it for your specifics. Pallet
places much more emphasis on using recipes (or crates as
we call them) as library functions, in cleanly
versioned packages
This is huge; when I was using chef re-use was an absolute mess. Ruby has all this great existing gem infrastructure for first-class dependencies and distribution, but chef ignores all that and just sends you back to cobbling together git repositories.
Opinions of Rubygems differ greatly and I would consider cargo culting gems as a distribution and dependency management infrastructure for Chef cookbooks to only be great for those who are strong with Ruby.
Each community has its own requirements and those of the Chef cookbook authors in general are different than those of Ruby library authors.
Lets say that I right a cookbook for apache2 on Ubuntu. What should I name the gem? Many other people will write this cookbook, particularly since mine only works for Ubuntu. Perhaps 'chef-apache2-ubuntu-btm'? Where do you store such metadata so a user can find, use, and contribute to a cookbook?
This isn't a simple solution and is a bit unique, which is why Opscode has been crafting a solution at http://community.opscode.com. It is easy to get a copy of a cookbook from the repository using 'knife cookbook site install COOKBOOK_NAME' and you can search the site through the web interface and using 'knife cookbook site search' to find the cookbook you're looking for.
The community site development has been slow, but continuous. In December we added source browsing so you can review cookbooks without downloading them.
I've never used Chef either, but those don't strike me as unique challenges. Building a Chef-only distribution mechanism to support discovery, versioning, and source viewing is fine and all, but such things do already exist in any sane dependency management infrastructure (whether we're talking about gems or maven jars or cpan or whatever). In particular, the dev side of me is baffled by the notion of having a separate toolchain for downloading source dependencies. git submodules and such are bad enough, but mucking around with a special workflow to download sources seems like it's a long path with few benefits and perhaps some big downsides (i.e. build/process repeatability?).
That said, perhaps none of that really matters; I wonder if all this comes down to hidden assumptions re: constituencies and heterogeneous skillsets, which I think is what you were pointing at in the beginning of your comment. i.e. "you need to learn the Chef toolchain" may be a more sale-able/scalable message than "you need to learn the Ruby/gem toolchain".
If package management was as a happily solved of a problem as you imply, I think we would see fewer solutions out there. Each tends to solve for the needs of its own environment, users, and sometimes the whims and bias of its developers. When you ask, why not use "gems or maven jars or cpan or whatever" I think you have to stop and ask, "why are there so many options already?"
There are many views out there for what the answer should be for Chef cookbook workflow. What you are "baffled" by, another person uses daily and argues for.
When using knife to download a cookbook into your git managed chef repository, it creates a 'vendor' branch to store the upstream version. You can utilize this to create diffs and patches to contribute back to that cookbook project. It is up to the author exactly how to do that.
Some really like git submodules, others prefer one cookbook per git repository, some just have one company-wide repository and ask you to fork and apply branches against.
I think you're right about the 'Chef toolchain.' There are [large enterprise] users of Chef who don't care that Chef is written in Ruby nor particularly want to learn Ruby. They want to use Chef. Opscode continues to strive to provide tools to make the workflow easier for Chef users as a whole over preference to a particular toolchain, especially one tied to and solving problems for a single language.
