In karate, this has caused the dilution of most forms of karate (with the exception of Kyokushin) into little more than ritualized movements, where the vast majority of even high-level teachers don't know how it relates to a real fight. Pure theory has taken over - 'when your attacker does a simple forward kick ('teep', don't know what it's called in English), move the leg with your hand!'. Except that that never happens in the real world, and yet it's a core movement in many kata (and kihon etc, not just kata).

Now, how does that relate to software? Just knowing how to write a binary tree, or linked list, or singleton, or whatever, doesn't make one capable of writing a program that users want to use. Yes, maybe it makes one capable of writing a perfect program even, in the sense that it may even be mathematically proven correct; yet that's absolutely irrelevant outside of the realm of the theoretical. The basics of this nature are well-documented everywhere already, and it's not necessary to know all of them before doing more integrated exercises; full applications that people actually want to use.

Or to make the connection back to martial 'arts' (I hate that word, what's 'arty' about it? We need to get rid of the mystical aura around it - it's fighting, a methodology to inflict maximum, concentrated or controlled damage onto another person, sorry for tangent - it's a pet peeve ;) ); we need to get rid of the karate's, tae kwon do's and capoeira's of the software world (disclaimer: I have brown belts in judo and karate, I get to have an opinion on it ;) ) and we need to go back to the essence - the mixed martial arts of software development: the misv or startup or whatever you want to call it. Actual products for actual users, not fetishising technology for technology's sake.

(second disclaimer: I know I'm sort of arguing a straw man here, nobody said anything about not making end-user products, but it's a lot easier to make a point when you don't have to put in all the nuances ;) )

I would consider it helpful if there were more people in the industry who would know what a binary tree (or whatever comes to mind) is and when to use it. all means to get there are welcome as far as I'm concerned.

This l'art pour l'art accusation comes back once in a while, often in language bashing ("elitist lispers" I read some days ago). As a former musician I recognize it in the shape of "technique ruins your soul" or how one would like to phrase it.

I think that the dichotomy is just wrong. It's not detailed knowledge versus pragmatic solution. I spent a great amount of my time in environments where the "shipping it" dominated "understand what you are doing", and I also spent time with drummers who could not keep the time because practicing would ruin their soul. It's knowledge vs. ignorance.

What concerns software development: the essence you want to get back to smells to me like the vbscript world I was happy to turn my back to. I would like to see more capoeira, less hash-tables because they are built in.

To follow up on your analogy with martial arts: I practice fencing once in a while, a rather ritualized sport, (for some 100 years already): I would not like to inflict maximum controlled damage to my opponent, or the other way round: I'm not training to kill. You might call it a kata I'm working on.

Your point "it's not necessary to know all of them before..." I cannot argue with. I would think the same. But the inequality "written by a programmer who cares about knowing his craft" != "people don't want to use" follows not from that statement nor from any other point you are giving.

A typical quote from these people might be 'the user doesn't care about the implementation details' - that's true, in the same way a car owner doesn't care if the car's engine is made out of cheese.

Somewhat understandably people start to form an ego around this idea, making it harder and harder for them to assess whether there might be better means of engineering their software.

As with much of software engineering, the problem is with the wetware rather than the (soft|hard)ware :)

When I studied Mathematics in elementary school I didn't memorize all of the multiplication tables because I thought I could always use a calculator if I wanted to multiply. Years later, when learning Algebra I had to go back and memorize those tables because having to calculate simple products and quotients was slowing me down. Knowing multiplication tables isn't necessary or sufficient for being able to do Algebra well, but it sure speeds you up.

I often practice my coding skill by doing problems like the algorithm problems on TopCoder. I find focusing on coding and solving problems quickly improves my work productivity in a way that doesn't come just from doing that work alone. I think these Kata will be even more useful for building the low level skills I use often in my work, so thanks to the original poster!

Sorry, as a black belt in karate, I am not going to let this stand.

Would I be able to handle myself in a fight? Maybe not. I really liked the stylistic, artistic side. But my teachers, who mastered both the stylistic and the practical, sure as hell knew what they were doing. In the first place, they had honed their punching and kicking skills to the point where they would be able to execute them perfectly in a fight.

But we practiced all sorts of moves, many of them grappling on the mat, that derived from the kata. For example, the Naihanchi ("iron horse") kata is done standing up, but really relates to ground fighting. Most of my teachers were extremely competent either standing up or on the floor, and we did lots of partner drills which got as close to a fight as could be done without injury, and which were heavily informed by the kata. A lot of times the connection was subtle.

I'm sorry that you got brown belts and were disappointed in your training. There are teachers that suck and teach total BS. Also, karate may not be the best thing to learn if you simply want to handle yourself in a streetfight.

But "little more than ritualized movements"? Have some respect.

That said, I disagree with the rest of your point. Naihanchi does not in any shape or form prepare one for any sort of fighting, be it on the ground or standing up. Of course when one does kumite 3 times a week, that does train one as a fighter, even if not maximally efficient; but it's far-fetched to say that because the movements of limbs in kumite sort of resemble the movements of limbs in a kata, that the kata adds significantly to the training for kumite-style competition (or 'actual' fighting, for that matter).

I also did not say that I was disappointed in my training. I got a lot out of it actually, and if I had more time I would still be training; but I no longer pretend that I learned effective fighting from it. I see it more as a fast paced yoga, and fun mix of medium-intensity cardio training and flexibility exercises. Which is fine. Also without actually doing it, I wouldn't have been able to draw the conclusions about its place in martial arts that I do now.

Finally, I'm not saying that the theoretical parts can't be fun, just like in the software cases that we started with. I have spend numerous hours toying with non-practical examples or test programs, for aesthetics or a minute performance enhancement or whatever. Similar in karate - I spend many hours in front of a mirror getting my side kicks perfectly level, for merely aesthetic reasons. Many people do - which is why I said the things I said, that the original goal has largely been forgotten. I mean seriously, kata competitions are seen as the 'noble' form of competitive karate, and kumite is just for youngsters with too much testosterone and roughnecks; does that make sense?

> Naihanchi does not in any shape or form prepare one for any sort of fighting

What makes you so sure? I don't think karate is like the Karate Kid movies, where you spend all morning painting the fence and all afternoon doing some ridiculous-looking kata, and then end up applying those moves exactly in a fight.

But I do submit that these kata are designed to teach precisely the kind of movements one would apply in a fight.

I have not ever actually been in a streetfight, and I confess my argument smells a little bit like "I've been doing this ten years, I refuse to believe that I've been wasting my time". Still -- are you prepared to argue that kata training definitely does not teach one fighting skills? Seems like that's an extremely difficult point to argue.

Kata, and code kata, is about practicing. So you're ready for battle. You shouldn't be practicing when you're fighting, and you shouldn't be practicing when you are getting paid to deliver a product.

This is true in anything. There are tons of "guitar" players who would rather play by ear, learn a few chords, and start a band. Then there are those musicians that spend the time to learn the boring fundamentals. And they practice those fundamentals over. And when you hear them play, you'll know they are masters. I think that holds true for coding too.

Kata teaches discipline and patience. It's like learning to breath. Practicing the basics in excruciating detail over and over again ensures you don't freeze up under pressure.

http://zedshaw.com/essays/master_and_expert.html

I mean, all of you must have had karate teachers that would let you walk from one side of the gym to the other, doing straight punches, and taking take or proper foot placement, hip movement, fist height and position, etc? While the teacher walks around and corrects posture and style? All those hours you've done that (and I have, too) are completely useless from a 'learning to fight' point of view. They are needed to become a good karateka, yes, but none of it will help you throw a useful, as opposed to beautiful, punch.

Lego bricks are hard and consistent and unchanging, and you need them to build incredible novelties.

BTW, my personal belief is that they are called "martial arts" because "art" is simply creating novelty within a system of constraints. Also, you are incorrect that all martial arts are concerned with inflicting damage--there are many arts and corresponding practitioners who encourage and value the mystical aura, because martial arts can be used as a vehicle for personal growth, discipline and spiritual nourishment. Please be careful about generalizing your experiences!

What I'm saying is - either some style is a dance or stylized movement form, or it's a system to fight effectively. The first I don't mind calling 'art', frankly I don't care much about it. The second is not 'art', it's science - it's about finding (through hypotheses, falsifying them, empirical testing, the whole yadda yadda) the most effective way to inflict various forms of damage to a human body (one can debate how far the use of tools can go in this - if swords are OK, why not landmines; but that's a different discussion).

"skill acquired by experience, study, or observation" - does that mean that when I learn to butcher a pig by watching youtube videos of it, it becomes an art? Of course not. "art" implies some form of subjective addition to it, usually for a "higher purpose", not just functional (I'm not trying to come up with a definition of "art", as whole libraries have been written about it, I'm just stating that you implying that anything that is learned in some form is an "art" does not correspond to the common usage of the word).

You state that theory is diverged from practice, which in general is one of the most rotten notions people commonly hold, but the very same sentence seems to indicate that you don't really understand the theory you're talking about - you first equate Computer Science to mathematical proofs of correctness, then you imply that proofs of correctness are useless in practice, both statements being equally unwise. Your users still want to use fast software, solve hard problems, do not like to find bugs in your programs and your colleagues still would prefer that the code you are writing is easy to understand and easy to maintain. It is not possible to go long-term about solving hard problems (writing fast programs being one of them) without knowing or inventing the necessary theory, and without lots of experimenting as well, for which there is ample historical evidence in the history of science and engineering.

I don't care if you do book exercises, katas from the web, or invent your own toy project, but you have to practice writing programs, including small programs, involving some actual domain problems (theory!) to be a really good programmer. A few times when doing recruitment for my company I asked the candidates to prepare a small program for me, first time a program extracting links from a given url, the other time a small TCP chat server and client. Such a program can be elegantly written in a page or two of code, yet 90% of people can not write a bug-free program of this kind at all, not to mention achieving elegance (ease of understanding the code + good maintainability + good performance). They of course won't cope well with a larger program and/or larger problem, they are just used to patch things together in a chaotic way until they find a combination that works. This tactic doesn't scale well to hard problems unfortunately.

The only reasonable function of the analogy to Karate is that calling an exercise a "Kata" might be motivating for some people and sounds nice. Carrying it any further doesn't have any meaning. For one, 90% of the time you spend doing Karate is doing various kind of exercises and you get in a "real fight" a few times in a lifetime perhaps. No wonder martial arts that spend those 90% of the time on something more related to the situation you want to be prepared for actually get you better prepared for the situation. I also doubt the goal of Karate is beating people up in the street or in some fancy cage. In programming the exercises are just an addition to performing your real work, which is probably done almost every day anyway.

Anyway, what your users want is a program that they can figure out how to use, and one that they actually can get on their computer, and that does the things they need them for - all of which have absolutely nothing to do with whether you used a quick sort or a bubble sort. Being fast, solving "hard problems" (methinks you're a bit full of yourself - when was the last time you heard someone say 'man, if only I had software that would solve a hard problem'?), and being maintainable are all secondary to the actual use - i.e. having a user interface they can understand, provide output in terms they understand, etc.

In your second paragraph you are actually making my point for me, but you don't seem to realize. What users want is a program that does something for them, like extracting links - and they don't care if those links are stored in a linked list or array. Maybe in some cases they will care at some point in the future that, when they download 10's of thousands of links, that happens a few seconds faster in some cases than in others, but that's not their main concern.

Even I, having had only rudimentary training in more 'goal-oriented' fighting styles, fairly easily dominate practitioners of more 'classic' styles (not because I'm good, but because of the ritualisation and artificial limiting to make things 'more pure'). I catch every judo player (once it goes to the ground, which is not hard to do with them with a single leg takedown since they don't know it...) in either a guillotine or straight leg lock. I mean, come on - that's groundfighting 101 right there.

(it's a bit unfortunate that I use judo here to illustrate, since they much less pretend to be a 'martial art' and refer to themselves much more as a sport than others do, it's just the example that was the first that came up to me).

And it's true that opinions don't make anyone right, but I consider it generally accepted and broadly empirically proven that practitioners of 'traditional' martial arts stand no chance against more goal-oriented and broadly skilled fighters. I'm not going to go into a 'my teacher could kill anyone with his death pinch if he wanted' argument here (bullshido.com forums style 'discussions' get old very soon).

(yes I realize you mean Sambo ;) )

For 'systems' that don't have 'minimize the time to learn it' as a goal, there isn't one style as such any more, imo. This is the beauty - no holds barred MMA competitions let the superior techniques float to the top, and everybody else learns from it and includes it into their training regimens. There isn't one 'style' that is taught, it's only people who are tested on their skills as a whole. And if one guy thinks he holds the truth in what style or techniques are superior, great - step into the ring, and put up or shut up. No more guild-like secretiveness, no more mystical uberman techniques or meditating for 15 years on a mountain top, no more 'I can't show you because I'd kill you'. In MMA, rationality prevails, once again.

Either something works or it doesn't. Tradition doesn't, precisely because it's tradition - a rapidly evolving style will easily beat one stuck in the fourteenth century. The Gracies dominated early MMA because they had decades of experience in Vale Tudo matches, fighting all comers in bouts with few or no rules. They knew what worked and what didn't through bloody, painful experience.

Martial arts in the traditional sense are simply woo, no better than homeopathy or divining. They have been tested and they don't work. There might be some value to what is essentially a form of dancing dressed up as combat, but take it into a ring, a cage or a pub brawl and the best you can hope for is that it is useless - most traditional arts are actively harmful.

And yes, some of the rules of MMA make it so that some techniques (groin shots, hair pulling, nipple twisters) cannot be used. But if the underlying suggestion is that there are somewhere out there in the jungle people who do use these techniques but they just have to suffer their superiority in solitude because their lethal pinky fingers prevent them from having interactions with mere mortals, I strongly disagree. My point throughout this discussion is (and I guess it's way off topic for this site...) that 'martial arts' have had their time in the context of systems who claim to contain or teach effective fighting. They're OK as systems of ritualized techniques to be practiced recreationally, but have been proven as in the best cases inefficient and in the worst cases wrong when it comes to teaching practical skills.

The early UFC fights (for all their faults) did show conclusively (at least, I don't really see how to explain it differently, nor have I ever heard anyone doing so) that one-sided training and a reliance on stylized movement are inferior, even when deployed by top practitioners, against techniques that were trained and developed in circumstances that mimic 'real' fights more closely. And I do understand the apprehension the more traditionally schooled feel against the new breed of fighting styles; but that's just another manifestation of man's natural tendency towards conservatism, I guess.

The weather.dat file in particular is like a vexing issue I've come across when converting PDF tables to text and then writing a script to delimit them. You can't just use a regex because some rows are entirely blank for some fields.

I wrote a solution here for PDFs I had to munge for work: http://www.propublica.org/nerds/item/turning-pdfs-to-text-do...

Basically, you start with some regex pattern as a delimiter and loop through every line and record the left-most position of each value. Then you compare it to a global array that stores the left-most position so far found in the document for a particular column. If the current row has a column x that starts farther to the left than the global array's column x, then that must mean there is a column between x and x-1.

There's some tinkering that has to be done with that, especially if the PDF to text conversion was ugly. But it would work pretty well for this particular set of data.