Sunday, May 05, 2013

Regarding the smug opposition to programming contests

It is an inevitable fact of life. A news bit will appear. Maybe it will be news about how the Chinese or Russian team won the ICPC, which forces Americans to come up with excuses. Or maybe it is news about someone winning the Hacker Cup, which forces programmers to come up with explanations as to why they didn't win it. The inevitable thing is that someone will start the discussion about how programming contests are not worth it. They don't teach you "real world" programming skills and they are bad metric for hiring.

There are times at which it just gets ridiculous. It could be an article about how Petr Mitrichev is such a high profile competitor and champion and ... Google hired him to improve their search engine and you will still find comments of the sort.

You know, I have spent my past life (A 7 years period is a life) in programming contests. I have plenty of things to say.

Real world programming

What I do at work consists primarily of X. Therefore, real world programming consists of X. Programming contests do not consist of X. Therefore, they are useless for the real world.

This is the part about these arguments that can really annoy me. It is an arbitrary separation between different parts of programming. A dismissive one. Because of course is some programming is not part of the real world, then it is most likely a game, or an illusion, or something like that.

In my case, I make my money from these contests so it is very difficult not to feel that they are very real things. But I know that I am an exception so let us ignore that little detail. It is true that most jobs in computer programming are different than the things you'd do in most programming contests. However, unless you are doing an internship, it is unlikely that a single activity will prepare you with the exact complete set of skills that will required for your job. You are not going to learn people skills from programming contests. But that doesn't mean that participating in these contests will make you unable to learn people skills. You can probably learn somehow. I am socially inept myself, but I know that plenty people that are into programming contests (and are better than I) are perfectly socially adapted people. I am pretty sure I am the odd case. Specially out of the samples I took from actual tournament finals I attended.

Programming contests won't teach you all the skills. So what? Spending your time in programming contests is not so different to spending it in the demo scene. Or in game modding. Personal projects. Hacking open source projects. Etc. You will not create a complete career of programming from this. But why is it that you think that you HAVE to? Some activities are just fun to do and that is good enough. Some activities are just entry points for something else. Some activities might allow you to learn a couple of things, but you will most likely need to do more than just spend time in those activities to have a complete learning process.

Anecdotally. I can tell you that you WILL learn some very real things. You will learn a lot about the programming language you pick. I learned 99% of my C++ knowledge from contests alone. You will learn about how to get an effective programming environment that suits for your needs. You will get used to coding and to enter into Code mode. You will keep your mathy brain hemisphere active and healthy. You will be able to quickly estimate if a piece of code will run in the time that you want given technical specs of a computer and other factors. You will be able to calculate big O easily. You will learn of your weaknesses and how to find and debug the most stupid bugs you could make. You will learn to make sure that something is actually correct before jumping to implement it. These are all very practical technical skills.

No one needs speed

There is a time component in contests. Therefore, only the fastest programmers win. Being a fast programmer is not good. We need code that is maintainable , not quick to write

I think this argument originates from having little/superfluous exposure to the programming contests world. It is true that most contests use speed as some sort of tie breaker when people solve a similar amount/quality of problems during a contest. There is a speed factor in the scoring. But I think that the outsider's eye might be getting the impression that this speed factor is far more important than it actually is.

It is very easy to fall into this trap when you are just getting started with programming contests. You assume that you are doing badly because you are not taking enough shortcuts when writing your code. Maybe because some of the top solutions are very bad role models, you get into doing awful things like abusing #defines, you mess up your code style and you begin to constantly rush when coding a solution. The thing is that after this transition to a speed freak, you begin to notice something strange: You are not improving that much. Maybe your score has improved, but you are still pretty much in the same tier as you were before. Worse, lately you are spending more time debugging your rushed solutions and taking a bit more time to solve the problems.

After all speed is merely a tie breaker. If you learn to solve the easy problems faster, you will still find yourself unable to solve problems that the better coders can solve. The amount of code needed during a contest tends not to be humongous, so the time it takes you to type it is not that important. If you jump quickly into typing a solution and the solution turns out to be wrong, you are going to take MORE time and you will like still get 0 points anyway. If you type something too fast or if you are careless with your code, you will also score a flat zero. You will find yourself reading your code more often than you write it.

Debugging is a vital part of programming contests. One that tends to be implicit and has a role that is hidden from the outsider eye. While you rush to type your code, you have to be careful to make it also easy to modify. You get used to things that are deemed good practices. Not because you like to follow the rules, but because you learned the hard way that you can get screwed by not following them. Programmers that finish a solution faster do it mainly because they have less bugs and they know how to debug themselves out. This is something that is evident from watching Petr's screencasts.

I used to be a programmer that rushed. With time I learned to still calm down. I discovered the hard way that coding solutions right away without making sure that they will work and rushing to type a solution very fast are ways to fail more often than not. I ended up learning a set of rules that I shouldn't break while coding. For example, I learned to avoid having multiple return values in a function because of the many times I had hard-to-catch bugs with memoization after breaking this rule. I learned that speed was not as important as theory and having a process to be able to convert new problems into problems I already solved. Contests do not only have a focus on speed, they have a far more extreme focus on correctness. In most of these contests, just a single wrong case will make your whole solution wrong. You learn to be careful of things that cause bugs. I developed an irrational phobia against floating point arithmetic.

I guess that there is a chance that those are just anecdotes and most people just code badly when participating in programming contests. There is still something important to say. If I started to type code twice as fast as I do. If I started to think of solutions twice as fast as I do now. I would still not be able to get better positions than the very top coders. I am talking about people that can solve in 20 minutes what I cannot solve in a week even with help. We are talking about a set of skills that is not as trivial as being "fast". We are talking about being fast when coming up to solutions to very tough problems. There is more than raw speed involved in this.

Concurrency is like super important, man!

The format is very limited so you never get to see parallel algorithms. (Or randomized algorithms, or approximate algorithms, etc.). There are technical parts of programming that are not dealt with, like caching. There are no interactive algorithms. Plenty of algorithms do not show up. Etc.

My problem with this is that it really overstates the importance of some subset of algorithms. I think that the algorithms you find in programming contests are varied enough. I also think that it should not be a matter of quantity. A good programming contest shouldn't be about how many algorithms you have learned. There is always time outside of algorithm contests to find/learn Voronoi diagrams in a library and they will be implemented much better than yours. They should be avoid solving problems. There is a standard set of tools that you will most likely use during a contest: Dynamic programming, Greedy, Binary search, BFS, DFS, Dijkstra, Max Flow should cover 99% of what you need. This is a good thing because it evens the grounds - Time you spent sitting in a classroom is not so important - . Theory is nice, but contests shouldn't be exams about how many algorithms you know. They should be more about the process of solving a problem. Reducing a problem to a combination of those 5 only tools is more interesting than having 1000 tools and knowing by memory what to use every time.

There is more to say about that argument:

I have never seen X in a programming contest. Therefore, they don't have it

Some of the criticism I read seem to deal with only a very partial view of programming contests and ignores that there are programming contests of all flavors and sizes. Most of the criticism seems to be really directed at the ICPC. The ICPC is, frankly, a terrible tournament*. Only universities are allowed. Only groups of three are allowed. Very ridiculous scoring rules (Probably what contributes to the perception that speed is all that matters). Weak example cases. Arbitrary rules for advancing teams that differ between regions. Even more arbitrariness in what allows a team to reach the regionals. Boring problems. Run all the test cases of a problem in a single program instance stuck to a gigantic queue. A lot of your usual criticisms of programming contests tend to apply mostly to the ICPC and similar. For example:

Language and technical limitations, Lack of multi threading, no concern about caching or external memory/ etc

Contests like the google Codejam or the IPSC don't have these limitations. Because your objective is to produce an output file rather than send source code to run. The IPSC is even better, because your algorithms are not limited to run only a few seconds per case. Although most people go standard in the Codejam, they don't have to. I don't have to. I have taken (limited) advantage of multi threading a couple of times. I think it should be possible for someone more experienced in concurrency to make paralel algorithms to solve certain problems. Since you run your code in your machine, you can go crazy and bother with far more technical aspects than usual. If you want to run each test case in one of the 120 computers you own, you can do it.

And those are only the contests that I have first-hand experience in. There were AMD sponsored contests in TopCoder that focused on multi threading and taking advantage of it. I never participated in them, but they existed.

Approximations, randomized algorithms, heuristics, interactive algorithms

Even online judges have this sort of problem every once in a while. But most importantly, the TopCoder Marathons and similar contests exist. A TopCoder Marathon is a contest that is very different to ICPC contests. There is not a correct answer. The author most likely will not have as strong of an answer as the best competitor. You need to keep improving your algorithms. The best algorithm wins.

(Because the best algorithm wins, there are many situations in which worrying about very low level speed matters. I remember having to deal with memory caching at least once. I know that the top level competitors in this setting go really low levels).

In addition, you do not only need to solve the problem. You need to know how to test your solution. I had to learn a lot about benchmarks and threading to be able to test solutions. I needed to be able to compare algorithms. This is usually how participating in a Marathon looks like.

Topcoder Marathons are certainly not the only contest that allows this. The IPSC has very interesting and nice problems that do not necessarily stick to the ICPC format. The codejam seems to be starting to try some problems like this. There are also plenty of similar contests that I do not have experience with.

Being able to read somebody else's code and find bugs.

Like really. Both TopCoder and Codeforces have this as part of their format. You can win extra points by quickly finding mistakes and the reason someone else's piece of code fails.

More professionally, though. TopCoder has invented a category of contests called Mod Dash. In this, programmers are paid to identify the source of an anomaly in a piece of software.

Actual software design and development

There are competitions in which you deal with having to design long lasting software. Development competitions in which making the objective is to develop the most maintainable code. Architecture contests... I feel like they are most likely very boring, but they exist.

To sum up

Programming contests are a fun activity. There is no shortage of opportunities to learn important lessons about programming. There are various kinds of programming contests for different tastes and interests. A programming contest may not open you the doors to the professional world, but they don't have to.

I am not saying that you should/must participate. If you don't like them. If you would rather do something else, go and do whatever you like. I think that in the programming world, the best you can do is something that you like. If you do it well, the opportunities will come by themselves.


* I mean, we all owe the ICPC a lot for being a precursor to the programming contest paradise we are living in, and a lot of our competitors wouldn't have been part of programming contests if it wasn't for the ICPC. But it has really aged terribly.


RA said...

Awesome post... agree with (almost) all of it.
I felt like reading an answer to "Do you agree with ?"

Karol Danutama said...

Being fast is good. But in real world, competitive programmers really need to learn how to make your code maintainable and have good quality. Trust me, learning that will be VERY easy for competitive programmers. While normal programmers will learn it the hard way. Thus I insist every CS students must learn competitive programming to improve basic Software Engineering skills. In testing/QA competitive programmers can usually be more thorough and better in cases coverage.

anon said...

"A programming contest may not open you the doors to the professional world, but they don't have to." it did for me!

vexorian said...

I guess I meant to write "may not necessarily open you the doors ..."

Xun Li said...

Cannot agree more. Definitely should put this on Quora... I felt sad when I saw those negative answers on it.