How should I assess answers to the programming assignment I use for hiring

I wouldn't evaluate the solution without the candidate being involved. I think it's unfair to make assumptions based on the code you see without having a way to check those assumptions.

Instead of just looking at it for bugs, design flaws, etc. why not bring them in for a couple of hours to pair program on the solution they provided?

This will give you an indication of how well they know the code the wrote, if they understood your requirements, how they work with somebody else, and if you can work with them. There are tons of other things you can learn about a candidate as well during the pairing session.

Ultimately, the goal of the interview is to find people you can work with right?

If you're asking for some data processing (that's what your description looks like), you should be able to have some test cases that allows you to place it in either "seems to be correct" and "definitely not correct" with some simple automated testing. This is good, because it allows you to automate part of the grading.

For the "reasonably efficient, concise and easy to understand" bit, you're starting to verge into subjective territories. The efficiency bit may be objectively gauged by having problem sets of varying sizes and see how the execution time depends on the problem set.

Conciseness may lend itself to a simple code size check ("is the source code less than X KB when fed through a pretty-printer"?).

Easy to understand is definitely subjective.

In a previous company that I worked for, we had a written C++ or C# programming test that would be given to candidates to do during the interview. Interestingly, one day, some of the programmers in the team started discussing some of the exercises and each had different ideas on how some of the questions could be interpreted, showing that what one person thought of a wrong answer could actually be argued to being right.

Perhaps, if possible, it would be an idea to get other programmers in your company who haven't seen the programming assignment to take it and see what they come up with. That would then put the candidate answers into perspective and be able to give you an expectation of what answers you should be getting.

Now that you have already received solutions, the answer should be: don't.

It's simply unfair/unsuitable to setup this scenario, then evaluate it using criteria, that the authors did not know prior to the evaluation. Like this you get a complete reality shift.

Whatever your criteria are, you cannot come to the conclusion, that a candidate is unable to provide them. If you tell the candidate in a transparent/fair way how your evaluation works, then they can optimize for these criteria. If you don't tell them, you will never know if they would have been able to do that.

Of course, one might reason the other way round, as in business, we often get assignments, which are later evaluated on a different basis. While this is fact, one must however also see that whenever this happens, you have a valid point of objection toward management. For an employee it is totally fine to tell the manager something like "of course it's slow in these extreme cases. We haven't received any requirements that state it should be otherwise."

You should allow at least the same level of objectionism to your applicants. Hence, I second the opinion of others here in that you should talk to the applicants about whatever bothers you about their code.

In addition, the errors you found in that code will similarly occur in the work environment later on (unless you truly want to tell me you have developers that write error-free code all the time). Hence, it gives you even more information: you can find out how the applicant deals with errors when you confront him with those.

We require both a pre-screen design exercise as well as several white-board exercises during the interviews. We'll also request a code sample if I can't find anything of theirs online.

I have found a strong correlation between the design exercise and how well they'll do within the white-board exercises. It also appears to correlate with their on-the-job performance as well.

Generally, I'll forgive a mistake or two within any of the exercises. We don't require compiled code for the design, but bonus points are definitely awarded if it's a complete, working program. Likewise, the white-boarding exercises give me insight into the candidate's mindset since I'm watching them while they sketch out their answer.

In your particular case, I would say listen to what the code is telling you about the candidate. By the time you see their solution, you've had a number of touch points with them: their resume; notes from HR on the phone screen; your notes from the phone interview; etc... If you think they're sloppy and the code they submitted is equally sloppy, consider that a red flag. Ask if your team can handle that level of quality coming on-board. Determine if that's something you can train out of the candidate.

Likewise, if the candidate has been otherwise stellar but you notice a "rookie" mistake or a corner-case not being covered, then consider proceeding with the interview process. If everything the candidate did was already perfect then they would be demanding a much higher salary. There needs to be opportunity within your team for the candidate to grow and learn to become a better programmer.

These types of exercises are just a data point in the overall process. As they require more effort, they can be more telling as a data point. But they aren't always a make or break type decision point. It's kind of like requirements gathering. The initial rounds are usually a bit murky and you have to keep imploring to get to the actual needs. Ultimately, the interview process is about identifying a match between the candidate's skills and your projects needs. Likewise there needs to be a match between the growth and learning opportunities your team presents and the candidate's interests.

In short, use their code project as part of your overall assessment of their skill level. Interviewing is like fuzzy logic - it's generally not a binary up | down vote at this point, but the code exercise will sway the analog signal into the well defined regions of 0 or +1.