What a great idea. One of those clever ideas that are immediately obvious once you see them (but not before).
I find man pages a chore to read, and that has definitely held back my skills, so if this actually works—i.e., you can type in any command and get a readable layout of what it does—I can definitely see using it.
Edit: It might be tricky to make this robust enough across all valid commands. I tried a few variations and they didn't seem to work as well. Beware the cherry-picked example.
Also, everyone asking for pipe support is dead on. It's how most commands actually get used, so it's really needed. Plus it would take this tool to a whole new level of coolness.
Edit 2: Who made this? I think if you wanted, you could turn it into a new kind of learning environment for the shell. It already is that, in embryo, but there are a thousand directions it could go. The trouble with learning the command line is the overwhelming mass of information with no way of navigating it or distinguishing what's important from arcane detail. What's badly needed is a simple organizing principle that feels good to work with and lets you learn what you care about right now (and hides the rest). You've got that here.
An it-just-works interactive tool is a really promising way to go. There are countless tutorial articles and books, but they're one-size-fits-all, and once they get too complicated they suffer from the same problems that man pages do. A tool that can actually be used to play with and learn interactively would be a real contribution. But it might be a lot of work to turn this into that. I doubt that the corpus of "all standard Unix commands plus all their options" can be sucked up and re-presented by any generic script; the variations are too unpredictable. There would probably have to be a lot of attention put into important special cases, and a lot of careful design to get it to it-just-worksness. And that is a must-have for beginners.
I definitely went into this with a 'let's start with something good and useful for as many commands as possible', so I haven't focused on accuracy for those edge cases. But from what I played with it so far, results have been pretty good.
Agree again on the pipe et al. support. It's not easy, hence why I haven't added this right out of the gate. I basically have to build a parser for bash, although I can probably get away with handling only a small subset of it that is interesting in this context, such as pipes, redirections, etc. I still have to figure out how to display multiple commands, because chunking them all in one page isn't going work visually.
I'll be happy to hear more about turning this into a learning environment. It might not work for this particular tool, but maybe it can use the foundations.
It's possible, but I'm not sure how robust it will be. There are a lot of heuristics being used to guess what the options are, if they expect an argument, etc. So when feeding it with a command that was typed by someone, which is most likely correct, it's fairly easy to look up each option in all the possibilities. Not to mention some commands have a very complicated set of rules (e.g. find), and the position of some options is context sensitive, which explainshell isn't (it could be, but that would require a lot of manual work).
It's interesting because I had an idea once to do the opposite: see if explainshell could use completion files from bash/zsh/fish as another source of options besides the man page.
Completion scripts from bash are far too much of a kludge to extract information from.
Most tools are written in C, and the standard way to parse options in C is with getopt(3) or getopt_long(3). I actually looked into searching the compiled binary for the getopt_long structure, but that wasn't so easy. Since these functions have information on all the possible options, it would be nice if there was a standard way to get them to spit out the option list.*
See my reply to your comment in another thread. It would be a killer app to have better autocomplete support, but I can't offer assurance that it's feasible.
* I had another idea: you could use ELF tricks (LD_PRELOAD) to load a different version of getopt/getopt_long that would spit out the specifications.
Fish shell does something like this. It's nice to start from, but there are definitely not enough on their own. Git for instance needs extra rules to complete branches and so forth.
- adding support for shell syntax is definitely the next thing on my todo list. It's a bit tricky since Python's shlex (which I'm using to parse the command line) is fairly limited in functionality and pretty much only handles quotes. So I'll have to write a mini bash parser which isn't trivial.
- the automatic options extractor isn't perfect and requires manual corrections here and there
- add an API for command line tools
I'll be happy to answer any questions you might have.
I wrote a very sophisticated iptables command line argument parser a while back and found shlex to be too limited. Instead I used Pythons OptionParser module--in reverse! It worked out great:
The mechanism used by that code may or may not be helpful for you but feel free to email me (daniel.mcdougall at liftoff software dot com) if you want to collaborate. I'd love to put together a module that anyone can import to interpret/explain commands that can perform man page lookups and whatnot.
At some point I wanted to convert it to a DFA because I thought it will give me more control and flexibility and be more maintainable over time. The thing to look out for is not make it too permissive so it starts catching things that aren't really options.
I see. I started doing almost the exact same thing at one time. My goal was to create autocomplete rules. I think I used the output from the tool's --help or -? rather than manpages, and I would have been using awk and sed, since I never learned Python.
Bash autocomplete specifications are messy, but you can sidestep a lot of the mess. You're allowed to run an arbitrary command to generate the completion list. I was envisioning some kind of standard option grammar that the autocomplete program would interpret for a given tool. The grammar would be partly generated automatically, but would likely need hand-tuning.
I can't edit my previous post anymore, if someone can maybe add something like "EDIT: copy/paste friendly version https://gist.github.com/ekyo/6383547"
or better, fix the formatting so it works...
Once the source is released, it should be possible to make such a client without needing to connect to a web service. These breakdowns are figured out by the Python code written by the author.
Soon (I hope) you'll be able to "sudo pip install gateone" on your Ubuntu machine and be able to access it in your browser on localhost (or remotely if you so desire).
I had plans to add a feature like this to the help plugin but it didn't work out and I forgot all about it. This article has suddenly renewed my interest :)
Looks awesome. Is it possible to list explanations in the same order as they are in the command line? For example, options for "tar xzv" are listed in the "v, z, x" reverse order.
should get you started. But if you rely on looking only at usage(), you'll miss "undocumented" features. Try reading the OpenSSH source for example. There is at least one undocumented option.
If the source code is so voluminous and the organization of the project so convoluted that it is prohibitive to scanning through the source to figure out what the program does, then that in itself tells you something.
The smaller the program (and its source code), the easier it is to master it, in my humblest opinion.
Generally, the larger the program, the higher the probability of bugs and vulnerabilities. This, you might say, is the cost of adding "features".
I prefer small programs where I can read the source code quickly and easily. They need not have many features. The less output the better. As long as they do one thing well enough, and reliably.
Not to mention what it does to ease of use every time you add more "features" to a program. Look at the OpenSSL binary. It is like they are going for the World Record on the number of options you can cram into a single command line program. Insane.
Any reason why this is stuck in browser? I've got most oif the raw data (man pages) already on my machine and the closer this is tied to command line the more likely I am to use it. For instance I'd love to be able to do;
~$ tar xzvf archive.tar.gz
~$ explain !!
explain tar xzvf archive.tar.gz
The GNU version of the tar archiving utility
-x, --extract, --get
-z, --gzip, --gunzip --ungzip
-v, --verbose
-f, --file ARCHIVE
Absolutely fantastic. That's going to be SO helpful.
I really like what you've done with the colour coding and the lines to link up the boxes of explanation with each argument, but it is still a little difficult to see what's going on when you've got lots of args. With an example long command line [1] (the sort this is really useful for) I kept having to scroll up and down from the whole command line to the explanation.
Perhaps when you do the hover over, rather than just greying out the other options, you could hide all of them and only show the relevant one. It would be right at the top, on the same screen as the argument (assuming you had the res). You'd get explanation of the particular arg, along with it's position and context.
Just a tip: when the command got a long man page you can't really see the line matching the option in the top of the page and the man section down on screen.
If you guys could keep the command on the top of the screen even when scrolled down that would be superb.
I almost never use my browser full screen, often it's snapped with 50% width. In opera chrome and firefox, the width is a tad too big for that. It'd be nice if it was more fluid.
Well that improvement wouldn't be that useful for me tough, I'm going to integrate your website in zsh and emacs if someone doesn't do it before me.
Also, if you can make it so that the input area is still there when showing results, it saves a frequent 'back' in the case someone wants to check multiple commands.
This is really awesome, but what would be awesome as well would be the inverse, which is the ability to find potentially relevant options from man pages based on keywords.
This site is a great idea for when you want to find out what some command line black magic is, but the inverse would be great for "I know this command line tool can do this thing, but I don't know the options".
Not progressing as fast as I'd like. Stumbled over a few roadblocks regarding parameter binding. The rules are complex (and underdocumented, I might add. E.g. why man gci displays three parameter sets, or why -fi always binds to -filter and not to -file is unknown to me. CmdletInfo.ResolveParameter sounds like being great in theory but is unusable in practice. At least when I'm done I might be able to revisit parameter binding in Pash. Preliminary repo is at https://github.com/ygra/ExplainPS – once parameter binding is done the rest is just fetching information from Get-Help which is the actual trivial part.
It appears to be incorrect with oddities like ps aux, which isn't the same as ps -a -u -x since -u expects a user list (and then reads in -x as a user). But this is a weird case because of backward compatibility.
A nice-to-have would be to have explanations for non-option arguments: for example, for "sed -e s/foo/bar/", give "replace foo with bar", and even feed foo into a regex explainer.
echo -<tab>
--help (Display help and exit) -E (Do not use backslash escaped characters) -n (No newline)
--version (Display version and exit) -e (Use backslash escaped characters)
Nice. It's a shame it can't break down the more complex ssh/scp parameters such as -L [bind_address:]port:host:hostport
Those are the commands I'm always Googling...
Oh wow, this is great. Many thanks! I intend to use it :) I see you already have most of my requests listed, so I'll just leave it at that and say "awesome work".
It allows you to share your code with other people who share their code, but not people who don't. Afterall, why would you want to share your work with someone who doesn't share theirs?
curious where to submit bug reports? for the heck of it I put in a long ffmpeg command; it detected the "-i" argument twice and the lines look a bit crazy.
here's the line to reproduce the error:
ffmpeg -i infile -s 640x480 -vcodec h264 -r 30 -b 450k -profile:v baseline -tune film -bufsize 2000k -maxrate 550k -an -t 16.0 -passlogfile infile.mp4.passlog -pass 1 -y outfile.mp4
I find man pages a chore to read, and that has definitely held back my skills, so if this actually works—i.e., you can type in any command and get a readable layout of what it does—I can definitely see using it.
Edit: It might be tricky to make this robust enough across all valid commands. I tried a few variations and they didn't seem to work as well. Beware the cherry-picked example.
Also, everyone asking for pipe support is dead on. It's how most commands actually get used, so it's really needed. Plus it would take this tool to a whole new level of coolness.
Edit 2: Who made this? I think if you wanted, you could turn it into a new kind of learning environment for the shell. It already is that, in embryo, but there are a thousand directions it could go. The trouble with learning the command line is the overwhelming mass of information with no way of navigating it or distinguishing what's important from arcane detail. What's badly needed is a simple organizing principle that feels good to work with and lets you learn what you care about right now (and hides the rest). You've got that here.
An it-just-works interactive tool is a really promising way to go. There are countless tutorial articles and books, but they're one-size-fits-all, and once they get too complicated they suffer from the same problems that man pages do. A tool that can actually be used to play with and learn interactively would be a real contribution. But it might be a lot of work to turn this into that. I doubt that the corpus of "all standard Unix commands plus all their options" can be sucked up and re-presented by any generic script; the variations are too unpredictable. There would probably have to be a lot of attention put into important special cases, and a lot of careful design to get it to it-just-worksness. And that is a must-have for beginners.