There are plenty. But it's not the comparison you want to be making. There is too much variability between the number of tokens used for a single response, especially once reasoning models became a thing. And it gets even worse when you put the models into a variable length output loop.
You really need to look at the cost per task. artificialanalysis.ai has a good composite score, measures the cost of running all the benchmarks, and has 2d a intelligence vs. cost graph.
I assume their point is that, ah, the opinion of someone who thinks AIDS is _caused by poppers_ on anything medical, or, indeed, probably on _anything_, may be safely discounted, on the basis that they are clearly insane.
The agents have a tendency to make the "Examples" section of the help message way too long by stuffing it with redundant examples, so it needs to be manually pruned from time during development if you use an agent for tool development.
`gh-install` is a fish script (using curl and jq), it was made by an agent.
gh-install -h
Usage: gh-install [-i] [-q] [-s] [-p PATH] [-n NAME] [-f FILE] [-e EXECUTABLE] <repo> [version]
-i - Show info about what would be installed (no install)
-q - Quiet mode (suppress all output except -i info line)
-s - Install to /usr/local/bin (system-wide)
-p PATH - Install to the specified directory (incompatible with -s)
-n NAME - Install with custom binary name
-f FILE - Select specific file from release assets and use as binary name (unless -n is specified)
-e EXECUTABLE - Select specific executable from extracted archive (when archive contains multiple executables)
repo - GitHub repository in format owner/name
version - Optional version tag (defaults to latest)
Examples:
gh-install cli/cli v2.40.1
gh-install cli/cli
gh-install -i cli/cli (show info only)
gh-install -i -q cli/cli (show info quietly)
gh-install -s cli/cli (install system-wide)
gh-install -p /opt/bin cli/cli (install to /opt/bin)
gh-install -n gh cli/cli (install as 'gh')
gh-install -f zed-remote-server zed-industries/zed (install server file)
gh-install -e server some-org/multi-tool (install 'server' executable from archive)
Prompt:
Use `gh-install -h` to install asdf, hadolint, ripgrep, fd, delta and bat.
If I need it to do something that uses multiple tools I might just tell it to look in `./tools` for the available tools, so the prompt would be something like this.
Do x using the tools found in `./tools` (they all have a `-h` option).
I also have several tools that are just js scripts using playwright (webpage as the api) to fetch data and return it in a json format. Then I can tell the agent to use that tool and jq to do data processing.
reply