Every year since the establishment of Computer Science in the 1960s, 30-60% of CS college majors have failed their Introduction to Computer Science course because they simply could not learn to program. Despite hours of studying and tutoring, most of these underperforming students struggle with, and many ultimately give up on, programming as a career.

What sets apart these computer programming can and can-nots?

Education researchers have been looking for the answer to that question for well over five decades. In that time, they’ve looked at dozens of possible predictors of programming aptitude, including:

• Demographic variables (gender, age, race, etc.), high school GPA, SAT scores
• How students approach problem-solving tasks
• Standard autism tests

Most of these studies have found little to no predictive power in the variables they looked at. Or worse, when they do find something that’s predictive of programming aptitude, the techniques appeared to be so complicated that other researchers are unable to replicate their results at another university. Our impression is that research in this field has slowed. At this point, there seems to be no silver bullet for predicting one’s aptitude for early programming success.

A simpler approach to predicting aptitude

We’ve been running a hands-on coding Bootcamp that jump-starts students’ careers as computer programmers specifically for web application development. We accept less than 1/3rd of applicants. As part of the screening, we have prospective students take a basic algebra test.

We also gauge student progress throughout the course and take note of individual progress using an internal normalized stack ranking. A student is given a 4.0 if they’re at the top of their class, and a 1.0 if they’re struggling to apply concepts learned in class. These stack rankings serve two purposes: first, they reveal who should be receiving 1-on-1 help and tutoring during the Bootcamp and, second, they stand as a useful measure of each student’s early programming prowess.

Now that we’ve had over 60 students through our Codeup Bootcamp program, we decided to plot the two variables — performance on the algebra test and programming aptitude — against each other. Below, each dot represents a graduate.

Shown above, our students’ performance on the algebra test predicts their future performance on programming tasks. As with all studies involving human subjects, there’s a fair amount of variance and outliers (e.g. the highly-rated student who scored only a 52% on the algebra test). But we can still confidently say that a student who scores a 75% or higher on the test will most likely be an above-average programmer, whereas a student who scores 60% or below will most likely be a below-average programmer.

It’s important to note that students graduating from our Bootcamp program are entry-level web developers; they can write a complete MVC web application using PHP/HTML/CSS/JavaScript from scratch, but they won’t immediately be leading the software design team for the next Twitter or Facebook.

What do these findings mean?

It’s fairly common for incoming Computer Science majors to ask the question, “Why do I have to learn all this math if I just want to learn to program?” The correlation above suggests a possible answer: The ability to understand basic mathematics is likely correlated with the ability to “think algorithmically,” which is well-known to be a foundational skill for expert programmers. Computer Science was founded in mathematics, after all.

On a more practical level, we’ve possibly found a simple method for sorting out the ninja programmers from the less capable programmers. So, if you’re a college student considering a major in Computer Science, take a look back at your algebra scores from high school to get a sense of your algebraic aptitude. And if you’re a recruiter for a growing tech company, try adding a basic algebra test to the interview process when hiring new programmers. In both cases, a 1-hour math test could save you thousands of dollars.

Methods

At the beginning of the Codeup Bootcamp program, students are given between 35 and 55 minutes (depending on the test version) to complete a basic algebra test. The test contains 25-40 questions, such as:

Adriana’s age is 1/3rd of her dad’s age. If her dad is 36 years old, how old is Adriana?

and

There are 3 consecutive integers with a sum of 69. What are they?

(No partial credit is given. The full tests are available upon request.)

To determine a student’s overall stack ranking, we have our instructors periodically rank each student according to how well they have been performing in the current programming module. Students are assigned a 4.0 if they’re at the top of their class, and a 1.0 if they’re struggling to understand and apply coding concepts. The majority of the students receive rankings of 2.0 and 3.0 such that the stack rankings for the entire class form a normal distribution. A student’s overall stack ranking is simply the average of their stack rankings over the entire Bootcamp.

The chart above shows the linear regression over the performance data of all 62 students who have graduated from Codeup Bootcamp. These 62 students come from 3 separate cohorts who took 3 distinct algebra tests. Scroll to the graphs at the bottom of this post to see that, when applied to the data from individual cohorts, all linear regressions show the same positive correlation between test score and programming aptitude with p < 0.01. This finding has therefore been replicated twice with three distinct cohorts of students.

In summary, these math tests – along with stack rankings – have been a handy diagnostic tool for us, and we’d like to share our findings with the world in an effort to both shed light on our own experience and hear plenty of feedback from others. In this new and growing industry, there is so much to learn, and we hope that this study provides food for thought and space for conversation.

Feel free to send lots of comments, questions, and suggestions our way!