What I Didn't Understand as a Junior Programmer

I remember the first time I saw a codebase over a million lines.  It was at my internship, a large 10+ year old system, in multiple languages, thousands of unit tests, organized into several projects and dll's that would take the whole night to recompile.  Some of the projects had complex build processes, requiring extensive scripts, and even our source control had custom hooks preventing us from committing code that didn't follow our style guides.  It looked like it would take me a week just to read through all the documentation.  My lead programmer told me it usually took people a year to understand the project in depth, but my internship was only for 3 months.

Wow, I thought.  Is this what it takes to be a real programmer?

At that time, I thought I knew my stuff.  Yeah, I understood the algorithmic complexity of all those common sorting algorithms.  My classes were a breeze.  I had published a couple games I made with friends, and I "knew" all about object oriented programming.  I was made humble.

It feels terrible to write bad code, especially on projects involving real money and people instead of some silly class project.  Worse yet, you feel your senior peers looking right through you.  They know.  They know you just fiddle around aimlessly, deleting 90% of what you write over and over, until you give up, look over your shoulder, and throw some crazy hack in.  If you have a code review, they see, they frown and try to speak, but your code is so mangled that you have bamboozled your lead programmer, and he can't even formulate a suitable design after seeing your nonsensical partial interfaces and spaghetti methods.  He looks at the screen with a frown.  "It's not on the critical code path, so we'll run QA on it, and we can come back and refactor it if we need to."  Shit.

Nightmares.  Sure, I'm exaggerating a little, but the feeling is there.  Programming is tough.  You get sucked into your workspace and spend the day juggling code style, programming patterns, runtimes, debug tools, interfaces, searching, planning, interruptions to your train of thought.  Oh, and you have to make sure you solve the problem in the meantime. 

But you overcome.

I ask myself the same question often: can some classes of knowledge, which are inevitably accessible with experience, be taught or only truly comprehended after numerous mistakes?  That's a question I can't answer yet, but I have a particular battle I'd like to tell you about.

The number one thing I could not get into my skull as a junior developer was, you cannot solve a bug without understanding an execution state that manifests it. The execution state, as I define it, is all the data, in any relevant scope, that is read or written by the instructions in the vicinity of the bug.

I remember those times clearly.  I spent so much effort, wasted so much time, changing variables around aimlessly, hoping to fix bugs that I didn't understand.  Even worse, I prided myself on being skilled at it.  I knew that if I poked at enough lines of code, the bug would change, maybe be less noticeable, or appear to be fixed entirely.  The worst transgressions I would make would be on math, often related to user interface or rendering work.  If alignment is too high, change "y - box.height / 2" to "y - box.height / 4".  It could fix the math error?  Maybe?  If I put a breakpoint, I'd look at a couple variables, but intermediate calculations often look like garbage numbers anyway, so the math that I had written myself would turn into some magic black box as soon as I moved to the next line of code.

This type of bugfixing is obviously barbaric, anti-intellectual, primitive idiocy.  But it's surprisingly common.  It works excellently for class assignments and weekend projects, because they are low complexity, TINY codebases with no maintenance and limited user coverage.  If it passes your teacher's unit tests, it's an A+ assignment.

The amount of time developers spend spinning in circles on problems they don't understand is staggering.

Now, I believe that even if you fix a bug through fiddling around, it is more than likely you haven't solved it.  The code that is modified the most often has the most bugs, and you just touched that function to fix it.  Not only is there usually another bug close by the first, but you also have very little ability to maintain both the line(s) you fixed, and the rest of the original code, since you do not understand the execution state that the function is operating on.  Even worse, your version control logs are now in danger of misleading programmers who look at the code in the future to make assumptions based on how you implemented your fix.  These issues just stack on top of each other, and I have been convinced of the potential danger from first-hand experience.

Luckily, I learned some things along the way and come to you with some tips on how I was able to improve on this category of debugging.  If the code is very convoluted, you may have to start refactoring it into more human readable form (making sure to check if the bug still manifests after a refactor).  Compose several test cases with a theory of how they will transform the data, and verify if your assumptions were correct.  If the data can be visualized, like if it's geometric or uses a complex container structure, consider outputting the representation in a visual way.  If there is a lot of intermediate data, get out a pen and paper, or a digital notepad, and track the execution state yourself.  Ask the author for assistance.  If you lose your bearings, take a walk outside and come back fresh.  Breathe.

I'm convinced that for any code, no matter how complex, you can follow the data within the execution.  Code can be written by someone who doesn't work at your company anymore and implemented in a spaghetti mess, and all variables named after his or her favorite letters of the alphabet, but with enough simplification, visualization, stepping through, verification, and time, you will have the insight you need.

Of course, the execution state is exponentially easier to understand from consistent use of good programming principles and design patterns.  If you drink my kool-aid and start evaluating what's really going on, you'll quickly be convinced to write strong, maintainable code.  More posts on this will come!

There is no excuse for not understanding the lines that you have written, or you are debugging.  There is no shortcut, even if it feels like there is.  Now I even spend time stepping through code AFTER I've fixed a bug, to make sure I fully understand it.  If necessary, I refactor it into better code for the future, or comment what I've learned from my analysis.  Only then do I consider it resolved.

You might feel like a genius by bypassing the step of understanding code, but that shortcut will bite you and your company dozens of times in real projects, with real users, and indefinite maintenance.  Avoid my poor habits.  Spend time learning to parse complexity, master debugging tools, and become comfortable writing tooling code to help you see clearly in the soup of 1's and 0's.  It will pay off.  No more nightmares.

"Most bugs are a result of the execution state not being exactly what you think it is."

I have to give credit to John Carmack for a blog that really made my thoughts more concrete, his post is here.