Programmers need to know math! June 2, 2012 | 08:12 pm

OK, I’m wigging out again. The debate has come up *again* as to whether or not programmers need to know math. The answer, in my mind, is an unarguable “yes”. If you don’t, then many things will be extremely hard, and some things impossible, which people who do know the math will find quite easy. But rather than take the high road, and make vague and unconvincing general arguments, I thought I’d dive down into the details and list the higher level mathematics I’ve found useful in my programming career. Note that the high school level mathematics- algebra, geometry, trigonometry, I’m going to ignore. This is the higher level stuff- stuff mathematicians think is mathematics. This is a personal list, and off the top of my head, so it is no doubt incomplete.

Let’s start with graph theory. If there is on branch of mathematics a programmer can’t live without, this would be it. It’s about things (nodes) that have relationships (edges) with other things. Like cities connected by roads. Or- in the more programaticaly useful, objects with pointers to each other. Pretty much all data structures are graphs. Run the standard graph theory algorithms to find out which objects can be reached from some set of root objects, and you have a garbage collector. Or how about this: functions that call other functions- the functions are the nodes, edges represent that one function calls another. Reachable algorithms tell you what functions can be dispensed with. Do a clique finding algorithm, and this determines which sets of functions for recursive loops. And so on. ‘Cmon, people- damned near everything we do as programmers is dealing with things that have relationships to other things.

Next up is linear algebra. Want to do 3D programming? Welcome to linear algebra. Lots and lots of linear algebra. Learning quaternions and geometric algebra is probably a good idea too- but linear algebra is a requirement. But linear algebra is useful for way more than that. Lots of other forms of mathematics (for example, graph theory) tend to reduce to linear algebra. Machine learning also uses linear algebra heavily- if you want to write an algorithmic trading platform or a search engine, better brush up on your linear algebra. Add in a dash of calculus, and you’re solving systems of non-linear equations (via Newton-Kantorovich).

Numerical analysis is how to implement linear algebra is the real world (i.e. on computers). Also, it covers the care and feeding of floating point numbers. If I were king, programmers would be required to pass a class on numerical analysis before being allowed to use floats.

Abstract algebra and number theory are useful for cryptography, random number generation, hashing, error detection, and a couple of other things. For example, 2′s complement arithmetic makes perfect sense once you realize that it’s just arithmetic modulo 2n.

Statistics. How could I forget statistics? There is more to statistics than Bayes theorem (and Bayesian reasoning), but for that alone it is indispensable.

I’m not sure what branch of mathematics Fourier transform falls over, but it’s another one that shows up all over. Including weird places, like multiplying large numbers.

There are a lot of “little branches” of mathematics, which aren’t full disciplines themselves, but are still worth learning. Knowing the relational calculus makes SQL make a more sense, knowing the Pi Calculus makes Erlang make a lot more sense, knowing the Lambda Calculus makes Lisp and Haskell make a lot more sense, and so on. Being able to at least read and follow the mathematicians gives insights into the fundamentals of what is going on.

For the record, I don’t know category theory, and don’t feel any deep seated need to learn it either. I haven’t got my head around geometric algebra yet, and I want to take a swing at algebraic topology at some point (to take a deep approach to relativity and quantum mechanics). Speaking from experience, it’s certainly not necessary to learn Haskell or figure out monads.

But, at the end of the day, there’s this. Programming is about solving problems- that why anyone cares about it. What’s important isn’t the code produced, it’s the problems solved. And mathematics is about how to solve problems. How do these two things not go together like peas in a pod? Richard Feynman once compared knowing different mathematics to having extra tools in your tool chest. Not all mathematics is applicable to all problems, granted- but learning mathematics is like learning new APIs, or new languages. It allows us to solve problems we otherwise couldn’t.

  • http://arw.livejournal.com/ Tony

    I would love to agree with you on math, but I don’t. There’s an excellent NY Times article recently about Japanese living abroad, http://www.nytimes.com/2012/05/30/business/global/as-global-rivals-gain-ground-corporate-japan-clings-to-cautious-ways.html . It raises a very big issue of the danger of being “over spec” as a candidate. Knowing more can make you less a team player, more likely to disagree with what’s taught inside a firm, and more likely to bolt and quit (perhaps because one gets a better offer), which brings the value of what a firm invests in you to zero. This isn’t only an issue in Japan, it’s an issue in many firms. One of the worst things one can do on a team is to claim you’re right based on information no one else possesses. In practice, one needs to either work for a firm which is tolerant of some knowing a lot more than others, no matter what their hierarchical position, or work for a firm committed to train everyone in a common language and/or recruit everyone with above a certain fundamental level of knowledge. Maybe those conditions are met. Often they are not. People dream they program alone. But in firms, they’re paid to program on a team, and that means submitting to shared solutions, not best solutions. Teams don’t have gladiatorial competitions where the best math person wins. If your best conflicts with the team, you’ll lose. If your math “skills” increase your conflicts, you will lose more.

  • http://enfranchisedmind.com/blog/posts/author/bhurt-aw/ Brian Hurt

    There may be no “i” in “team”, but there is “eat me”. :-)

    Two serious points. First, technology is inherently disruptive, and software doubly so. When you solve a problem, you change how things are done. Presumptively the new way is better than the old way, but it’s different- and thus disruptive. If technology isn’t being disruptive, it isn’t creating the value it could- at which point, your real job description is to fulfill some sort of socio-political role (i.e. be a pawn in some one’s empire building). Which might not be a bad career, except for the fact that simply because you don’t cause the disruption doesn’t mean someone else (probably at some other company) won’t come along and cause the disruption anyways. Corporations may think what they want is techno-serfs, whose main job qualifications are not rocking the boat and not standing out- and, by implication, being unable or unwilling to solve the hard, transformative, disruptive, *valuable* problems, but this is always a long-run bad idea.

    Second, let’s take a look at this from the point of view from the programmer. What sort of career is techno-serf, anyways? A lousy one. It’s a way to end up fifty, unemployed, and with a skill set so out of date you’re unemployable. Yeah, the company doesn’t want you to have a better job out there- but that means when they let you go, you’re screwed. And let you go they will, sooner or later. A disruptive technology will come in, which you won’t have adopted because it’s disruptive- but when the company decides to adopt it, they will simply hire a younger, cheaper, version of you with an up to date skill set, and let you go. Which would you rather be- the guy with the option to go some where else, or the guy without an option? Of course the company doesn’t want you to have an option- they don’t care about how screwed over you get, and you not having an option gives them more leverage.

    “So why not go into management” you ask? Two things. First, there are damned few management slots to go around. You only need one manager for every three or four programmers. Back in the 60′s, 70′s, and even into the 80′s, when the software field was growing exponentially, getting into management was easy. These days, not so much. Second- fine, but then management, not programming, is your career. And other advice is probably applicable. By the way, if you’re 25 and haven’t gotten at least a toe hold in your chosen career, you have a problem. Which means if your plan is to become a manager, and you’re still coding at age 25, maybe it’s time to rethink some things.

  • http://elehack.net/michael/ Michael Ekstrand

    I agree. Knowing math allows the programmer to not only write a correct program, but understand and argue about why it is correct. And design it to be correct in the first place.

    Just in the last few months, an colleague and I used graph theory with a bit of set theory, order theory, and dynamic programming to build a dependency injector. Result: 6K lines of Java (including tests), configuration capabilities impossible to express in any other injector we know of, and we believe it to be correct with respect to a robust definition of the problem. Math (incdentally, little of which involved any actual numbers) allowed us to identify what the core problems really were (and where several things were actually the same problem!) and come up with simple yet robust solutions to them. After it was built, it then proceeded to demonstrate all sorts of interesting emergent behavior that resulted in code and usage simplifications we didn’t anticipate when we originally set out to build it.

    One major benefit of math is that it allows you to abstract the concrete problem (satisfying object dependencies) into mathematical entities (graphs) that don’t care what they’re representing. Turns out that, while there may be little prior work on your specific problem, it probably reduces to a mathematical consruct that has seen a lot of prior work that you can leverage.

    It’s also a lot easier, in my experience, to build a correct algorithm if you can think in higher level terms (e.g. build this set, take the complement, do something with that…) rather than lower-level constructs (iterate all these things, check if they should be used, do something with them).

    Long live mathematical programming!

  • Marc

    I’ve been a professional programmer for over 20 years, working at numerous companies as a contractor, and I have never once needed anything beyond simple algebra. So I call BS on your whole argument. You could argue that’s anecdotal, since I’m just one guy but this seems to be the prevailing opinion of almost every programmer I talk to. I’ve never had to do any of the things you used in your argument and I know a dozen programmers off the top of my head would say the exact same thing. Now, there are many types of developers out there but only a very small portion do anything that requires any higher math. The rest of us are building business apps — not garbage collectors. Seriously, what century are you from? The vast majority of the CS people I know forgot almost all the higher math they learned in school. In short, I have about 50 things I would look for in a programmer above higher mathematics (and assuming they at least know algebra), just to give you an idea where I put it on the priority list.

    While you ponder that, take a wild guess how many times I’ve been asked math questions in an interview.

  • Matt

    As a counter-anecdote, I’ll point out that at my work we have a mathematician (specializing in statistics) on staff. Several of the programmers are conversant in advanced mathematical topics. They all have deep knowledge of the kernel, REST architecture, or some other abstruse and arcane area of computing. This is a fantastic job – literally the best place I have ever worked, or even expect to work, largely because of this level of knowledge in my colleagues.

    I think it’s also important to distinguish a couple of different categories here: there’s knowing the actual mathematics, and there’s knowing how it applies to your field (programming). Then there’s being actively hostile to this type of knowledge – the “you’ll never need it” argument.

    Here’s the hard truth: every good job I’ve had has actively screened out candidates with a math-hostile attitude. It’s pernicious and ultimately self-destructive even in an otherwise superlative programmer. It’s a time bomb for teams: a huge red flag that this person is not willing to adapt to things they do not know, and is unable to apply critical, rational thought to things outside their experience.

    An example from our interviews: you get asked about the big-O complexity of a piece of code. The acceptable answers fall in a spectrum roughly from ‘rule-of-thumb’ answers like “Well, it has a sort, so it must be at least O(nlogn)” to actually deriving an answer. Not knowing big-O is OK too, as long as you have a decent sense of efficiency and you’re strong in other areas. The only sure-fire black mark is to say something like “no one ever needs that stuff.”

  • http://twitter.com/hugo_dc hugo_dc

    Thanks for this very interesting post.

    I’ve been interested in learning Haskell.
    In the company where I work, no-one uses Haskell, they have never heard about Haskell, and probably I’ll never use this programming language in a productive project for this company, but I really enjoy learning this language. I would like to learn Mathematics the same way I’m learning Haskell, just for fun. I think that is the spirit of a real programmer, to enjoy solving problems, no matter what tools you are using.

  • http://impandins.blogspot.com/ BMeph

    I agree completely with your argument, although I’d emphasize the “tools in the tool box” part of it more. Basically, the more ways you have to look at a problem, the greater your chances are of coming up with a working solution.

    And working solutions to problems get businesses, and business payrolls, paid.

    On another point, though, I’m amused that near the end, you said, “For the record, I don’t know category theory, and don’t feel any deep seated need to learn it either. I haven’t got my head around geometric algebra yet, and I want to take a swing at algebraic topology at some point (to take a deep approach to relativity and quantum mechanics).” I’m amused, specifically because category theory is more-or-less abstract algebraic topology, and CT still uses much of the vocabulary of AT, even though they are (now) separate fields. Also, because CT is “simpler” in a similar way to how Lisp is a “simpler” programming language to others.

    In a nutshell, category theory “is about things (objects) that have relationships (morphisms) with other things.”

    Oh, and to “Marc,” I call BS on your BS call. I’ve been an apprentice electrician, and seen journeyman electricians at work that read building plans better than a road map. However, just because they don’t use “anything beyond simple algebra,” doesn’t mean that knowing it would be useless. At the very least, they could have made outstanding engineers, or being able to talk to engineers in their own language, they could help make the engineers better at their jobs, thus making the electricians’ jobs easier. I’d say that if you don’t know how some extra insight could help you solve your problem, you can’t say that it could never have helped you. That, is “BS” of the lowest order.

  • elias

    “I’m not sure what branch of mathematics Fourier transform falls over,”

    Funny thing. Here, computer science students see it (en passant) on Calculus III, and they generally hate it. Computer engineering students see it on “signals and systems” and “signal processing” (and actually use on topics like “circuit theory”, “dynamical systems”, “control theory”, “data transmission systems”).

    The most prominent applications on computing seems to be multimedia (including lossy compression). Interface with the analog world. Filters, signal processing (including image processing).

    Seriously, I know people that majored on computer science but have no idea on how JPEG, MPEG or MP3 actually work. The concept of attenuating or discarding less important frequencies is totally alien. They studied a transform as a weird integral, not as a practical tool to solve real world problems.

    Polynomial multiplication is isomorphic to convolution (an operation commonly studied on a signals and systems course). You can use a FFT to move this problem to frequency domain, simplifying it (and then use it again to go back to your usual domain). That’s how you can do a fast multiplication with FFTs. But they are much more useful than that. This online book show a bit more for the interested: dspguide.com

    To your list, I would add optimization. Knowing convex optimization is key to understanding tons of programming. For example, lots of AI algorithms (such as backpropagation on neural networks) is really just optimization algorithms.

  • Andrew Cole

    I agree with you 110%. I think the real problem is that too much of the hard work is done for you most of the time. These people use API’s they could never understand, much less reproduce. Beyond that they just rely on lady luck and Google to save the day. You can write a program that does very sophisticated things without knowing your ass from your elbow. However, it’s probably full of bugs and logical errors. What a shame.

  • Andrew Cole

    I agree with you 110%. I think the real problem is that too much of the hard work is done for you most of the time. These people use API’s they could never understand, much less reproduce. Beyond that they just rely on lady luck and Google to save the day. You can write a program that does very sophisticated things without knowing your ass from your elbow. However, it’s probably full of bugs and logical errors. What a shame.

  • frank kamaa

    I agree. Mathematics is a pivot to programming.