Should CS students be doing their tests on paper?

54

9

In some CS courses (especially undergraduate and high school courses) the tests and the final exam are written on paper without the use of a computer. But if the students are in a CS course, shouldn't they be allowed to use the computer? If some mistakes are difficult enough to not be noticeable while writing code on a computer, they are nearly impossible to be found while writing code on paper.

So, my question is, are there any advantages in doing written tests on paper, and is this the best approach?

Safirah

Posted 2017-06-13T19:51:53.660

Reputation: 571

Depends what the test is on. – Kevin – 2017-06-15T18:30:05.093

3Why do most answers assume that the only alternative to paper is a fully featured IDE and not a minimalistic plaintext editor like notepad? The biggest advantage of using a text editor over paper is that it supports insertions. – CodesInChaos – 2017-06-16T08:41:13.927

@chrylis Dijkstra programmed on a computer. He just never wrote articles with a computer, he preferred a typewriter or his Montblanc pen – Viliami – 2017-06-16T10:02:03.277

FWIW I barely used a computer at all in my first 2 years of my CS degree and think I got a good education. In my later two years, I did a lot more programming for classes, but it was never really the focus. – ell – 2017-06-16T14:35:53.880

2am I the only one who read this question as specific to exam questions where the candidate is expected to answer in code? All the answers and comments here about how "computer science != coding" and whatnot seem to be missing the point entirely. Of course it's fine to have to explain in words the intuition behind bubblesort etc. But it's also irrelevant to the question being asked. For me the question is, "if there is a question asking you to write a simple bubblesort implementation in code, shouldn't that part of the question be handled / compiled on a computer as opposed to on paper? – Tasos Papastylianou – 2017-06-16T19:13:03.423

1You are looking at it the wrong way. Usually, a test written on paper allows you to not get hung up by a missing semi-colen and get graded how well you expressed the algorithm/idea/ ectera. If the problem is difficult to the point you'd have trouble doing it correctly on a computer, you will probably score MUCH better on paper. – 8bitwide – 2017-06-17T02:32:37.053

@apnorton It seems then that Dijkstra's point was that for studying theory of CS/astronomy, no additional tools are necessary; however, for studying practicals of CS/astronomy, yes, you're absolutely going to need a computer/telescope.  Extrapolating that into a brief answer to this question— for CS fundamentals paper is fine; for code you should teach on / test on a computer. – Slipp D. Thompson – 2017-06-17T04:31:23.977

Wow things have changed. Back in 1996 I took the freshman intro CS class at an accredited university in the United States as a requirement for a BS in mathematics. The class was literally all coding in C++ on Solaris SPARC workstations using cxx as the compiler. We took our tests on paper and we had to write at least pseudo code for many answers (there were many purely theoretical questions as well), but we had the option to write answers with actual code in the language of our choice. Obviously we weren't graded harshly on typos or even omitted semicolons or everyone would have failed. – Todd Wilcox – 2017-06-18T07:03:04.753

I'm very grateful for all the answers this question has received so far. It really gave me a new perspective in the matter. Since there is really no correct answer in this case, I'll choose as the "correct answer" the most popular one. But, if you have an insight on this problem that was not mentioned here, please post it, it may be very helpfull! – Safirah – 2017-06-18T14:35:48.333

2The problem with computer science (in English) is that the name is misleading. It's not about computers, it's a science related to automatic processing of information. For instance, in Italian (and Polish I've been told) it's called Informatica from informazione + automatica. Tests should be done on paper of course. The only time in 3+2 years I've done an exam with a computer it was a probability and statistics exam where we were allowed to use R as a calculator (the test was still on paper). – Andrea Lazzarotto – 2017-06-18T21:23:04.220

1Also, if "coding" on paper you are not distracted by typos and costantly trying to compile. There is a reason why we always tell beginners to not start coding until they have a clear sketch of their solution on paper (or by heart, if it's a trivial problem). Coding distracts algorithmic thinking unless you've built some experience. – Andrea Lazzarotto – 2017-06-18T21:23:58.923

Written on paper they can't blame typos or auto-complete for making the errors. – Gypsy Spellweaver – 2017-06-13T20:23:14.503

2@GypsySpellweaver, typos occur all the time if you write your code on paper – Safirah – 2017-06-13T20:44:58.723

10Dijkstra never used a computer. – chrylis – 2017-06-14T00:52:46.990

31"Computer Science is no more about computers than astronomy is about telescopes." [Dijkstra]. Paper tests are still relevant for CS courses. – apnorton – 2017-06-14T02:43:58.877

9IMO it depends on what you're teaching. Traditional CS is as much about theory as anything, so the knowledge and understanding is what's important. If you're teaching coding and/or for real world usage. (e.g. a 3 month intensive iOS program) then much of what's important is one's ability to use the tools available to them, so a paper test would likely be kind of terrible. – PEEJWEEJ – 2017-06-14T17:02:27.867

Answers

50

Edit: This relatively high voted answer seems to be confusing to some, as it does not seem to give a clear answer. TL;DR: yes, doing tests on paper is usual at least in my university, at least many years ago. My anecdotal experience means to give a background where this might be coming from, together with the title "CS!=coding". The last section at the end gives my opinion on why it is a good thing.

CS != coding

You also have to acknowledge that CS is not coding; coding is not CS. You can have great coders who have not the faintest idea about CS (and vice versa: rubdos mentions the example of Alan Turing himself, who died around the time the first programming languages were invented). I can't remember whether in my CS studies I ever wrote actual real world code in a test. Lambda calculus, sure, and other formal stuff (more like mathematic formulae/proofs than programming).

I do clearly remember that coding/programming was not part of the curriculum. It was assumed that people who went to study CS already were able to program. At that time, Java 1 was around the corner, and my university offered a voluntary 2-week Java crash course before the actual semester started. I did not take part (having a few other real world programming languages under my belt at that time), and never missed it. There was a bit of programming during some practical courses, but here also the profs assumed that everything we needed to know we'd pick up ourselves. And right they were.

Funnily enough, one of my profs was very active in the Java community, and used it to motivate proofs in temporal logic / correctness. But that was very much theoretical, about concepts and principles - "coding" had nothing to do with it whatsoever. It was more about "how do you create a compiler" instead of "how to you write a program". That's what university level CS is about, after all.

Hence, as it was not part of the curriculum (being assumed to be known as a matter of course), no need to ask about it in a test; hence no reason to use a computer in tests, not any more than in any other field.

EDIT: this answer is my experience from university level CS education; it somewhat applies to earlier levels as well, in my opinion - it is more important to teach the principles than the actual coding; and a test would be better served with checking that they understand roughly what it's all about instead of writing perfect syntactic code with a pencil. I'd rather have a pupil explain what a basic design pattern is, instead of showing the actual code for a simple for loop. Programming languages come and go, principles stay (adjust to your own curriculum of course).

AnoE

Posted 2017-06-13T19:51:53.660

Reputation: 1 109

This is the answer I most agree with, so I will add my own anecdote. I am a fairly recent graduate of both a BS/MS in CS from a mid-level state university and I took three years of CS in high school. In all of these programs, tests were handwritten and focused mostly on answering theoretical questions, drawing diagrams, and only rarely did we debug code or write our own (and handwriting LISP is a massive pain). Quizzes were also handwritten but focused on pseudocode to describe a concept. Assignments were the true coding projects. CS is about theory, not "how to write language X". – Brian R – 2017-06-15T14:55:21.187

2so.... should they do their tests on paper or not? – Viliami – 2017-06-16T10:06:29.417

2@BrianR "handwriting LISP is a massive pain" -- random fact: LISP is the only language currently in widespread use that was originally designed to be handwritten. – Jules – 2017-06-16T10:11:52.843

@BrianR, Viliami: yes, they should do tests on paper. I thought it were pretty clear with the last paragraph, and have added a small TL;DR at the beginning. – AnoE – 2017-06-16T11:30:56.177

Your title makes as much sense as “English Writing ≠ writing”.  Suggestion: To better fit your points, change the title to “CS > coding” or “CS ⊃ coding” (and reword a few things within). – Slipp D. Thompson – 2017-06-17T22:45:20.070

2Most of my CompSci education was theoretical. There were assignments; write code, get it working, submit it, etc. That's not CompSci; that's the APPLICATION OF CompSci. How does a red / black tree work? How is searching it O(log (n))? If you can't explain it, how can you apply it? My Database class dealt, extensively, with relational model. Most of it was done using algebraic notation. Once I had that, SQL was a breeze (2 class periods). Today, I write pretty complex SQL. "An exceptionally good mastery of one's native tongue is the most vital asset of a competent programmer" -- Dijkstra – Meower68 – 2017-06-18T15:48:59.067

I agree with this answer, although my experience is a bit different. Of course CS is not coding, but in my BSc (University of Venice) functional programming, imperative programming and OOP were all mandatory courses. You expect that a computer scientist can turn an algorithm into code. I finished my MSc in 2016, FWIW. But still, programming is just a tool and CS is much more... today many improvised "coders" don't seem to understand this. "Serious" tests shall always be on paper while assignments can be practical. – Andrea Lazzarotto – 2017-06-18T21:30:36.430

1@Skipp Computer Science is about computers in the same way Astronomy is about Telescopes (misquote of "Edsger W. Dijkstra"). Would you say "astronomy is a superset of telescope operation"? – Yakk – 2017-06-19T06:04:29.407

"You can have great coders who have not the faintest idea about CS (and vice versa, I assume)" Vice versa: Alan Turing is considered a great computer scientist, but he died around the time the first programming languages were invented (the 50's).

So you assumption seems correct. – rubdos – 2017-06-19T08:43:49.183

@rubdos: Nice! I've integrated it into the answer. – AnoE – 2017-06-19T09:12:14.430

Hey AnoE, @username won't work in answers; you can however put a link to the comment if you want to (right click the comment's time, copy that link) – rubdos – 2017-06-19T09:13:34.037

@all: while I see what you mean with the "!=" discussion, it's just a qip to show the gist of the answer on first glance. I believe the stylized "!=" is ubiquitous enough to carry the intended meaning. – AnoE – 2017-06-19T09:14:25.633

2I had a similar experience. We had some Pascal in the first year and after that professors assumed we are able to code: You have to do some exercises on operating systems? Go get a C book and learn it yourself. – Giorgio – 2017-06-14T17:22:19.793

1On my CS course, programming was a core module (Java) and other modules often included some coding elements (either real code or pseudo-code). In all cases code had to be written out on paper as the OP describes. IMO this answer spends way too long describing your particular CS course and making assumptions that don't necessarily generalize well, and not long enough addressing the actual question (advantages of paper vs computer). – JBentley – 2017-06-14T18:16:57.173

50

100% yes. In beginning courses, it's practically a cognitive requirement. Let me see if I can break down for you why it is so important. It comes down to what we can know that the student knows.

Let's say we have a student who is a rather mediocre coder and who, to be honest, really has no idea what is going on in class. We're going to give him a problem in Java, which is to go through an array of ints, add all of the numbers in said array, and return the result. We're also going to let him use an IDE to build it, so that he can find syntax errors and spot-check himself for silly errors by running the code. The first thing he writes is this:

for(z=array[0], z < array, z+1){
    z=array[0];
}

So, how much can we say that he actually knows? I can forgive that he used commas in his for loop. I might even, depending on his background, forgive that he forgot to declare z. But what can be said about the fact that he set z to array[0] instead of to 0? Or that he is comparing z to array instead of array.length(), or that he is never actually changing z in his z+1 statement? These are mistakes of comprehension, not syntax. What, also, can I say about the fact that he keeps setting z to array[0] over and over again?

The fact is that, if he writes this into an IDE, a few things will happen. A bunch of code will light up red. A red line will remind him that he forgot to make a return. Another red line will tell him that z < array is not possible, because they are of different types. So, he will start typing things until the red lines go away. Does he necessarily understand the things he is changing, or has he been trained, Pavlovian-like, to change code that has been underlined in red?

As he makes some red lines disappear, new red lines pop up. He then turns his attention to those until he makes all the red disappear, and now he has this:

int g = 0;
for(int z=array[0]; z<array.length; z=z+1){
    g=array[0];
}
return g;

This code is much, much better. It's still not perfect. But, given the flaws in his first draft, do we have any reason to believe that he really understands this new version and why it is actually better? Or is he just changing code until the red goes away?

Now, he runs it, and it immediately ends by returning the value 0. So, since he still has about 5 minutes left to his exam time, he starts just changing things at random. He changes his int z to be = 0. Now he starts getting a number returned! He tries putting g into g=array[0], then he puts z into it. Now he has g=array[z].

He dithers about a bit, and finally adds a + in there. He now has the correct code:

int g = 0;
for(int z=0; z<array.length; z=z+1){
    g+=array[z];
}
return g;

He turns it in to the teacher, who says, "Boy, he sure understood this! That's one student that I don't need to give any extra attention to, because he is in great shape in my class!"

The takeaway questions then are:

  1. How well does the student understand the material?
  2. How well does the student's grade reflect that?
  3. How much worse are the odds that the student will now receive the extra help that he so desperately needs?

We need tests that actually show what you really know, which is not always the same thing as what you can produce with prompting help from an IDE.

Ben I.

Posted 2017-06-13T19:51:53.660

Reputation: 17 660

5"Does he necessarily understand the things he is changing, or has he been trained, Pavlovian-like, to change code that has been underlined in red" -- this is an eye-opener. +1 – Firelord – 2017-06-15T08:10:50.473

3I went to school with a guy (who graduated) whose idea of bug-fixing indexing errors was to add +1 or -1 to various terms randomly and hope the issue was just an off-by-one error => x+1 fail. x-1 fail. z+1 fail. z-1 fail. y+1 fail. y-1 pass. Problem solved, and he didn't even have to understand it. He also just googled most of his homework... so I agree with this answer 100% – TemporalWolf – 2017-06-15T18:53:05.870

@GypsySpellweaver I do not know where you are from, but from my experience it's quite different. Yes, there are not very good CS students, but the software development field is often filled with improvised "coders" who just learnt a language following some tutorial without knowing a bit about algorithms, data structures, complexity, formal languages, ... you name it. Or designers who suddendly are "web developers" because they tried Dreamweaver. – Andrea Lazzarotto – 2017-06-18T21:35:14.053

@AaronD please note that science and engineering are different fields. They do overlap on some stuff but differ on other. Even though many CS and CE people become software developers this doesn't mean that they have the same background. An engineer can beat a scientist on some stuff, and viceversa. No-one is better than the other but they differ. – Andrea Lazzarotto – 2017-06-18T21:37:01.273

@BenI.: Is this Java? If so, sorry for being pedantic: length is actually a property of the array object, instead of a method. For the code to work, you'd have to have z &lt; array.length :) – homersimpson – 2017-06-21T03:50:34.250

LOL exactly the kind of mistake that would never be penalized on an exam :) (Quite literally - among many, many other things, the AP rubric explicitly makes length/size() confusion, dropping extra parens on paramaterless methods, or adding parameterless parens on fields nonpenalized errors) – Ben I. – 2017-06-21T10:32:56.880

7+1 Great motivational answer. It would almost be funny if that were not what is actually happening in the real world (in businesses) a lot, in practice - programming by trial&error. – AnoE – 2017-06-14T13:01:16.310

2I'm a career teacher, and your comment just made me sad. Is that really what goes on? – Ben I. – 2017-06-14T13:12:27.583

Not always, but often... – AnoE – 2017-06-14T13:16:04.520

13@BenI.: As a maintenance coder of 17 years' experience digging through other people's code, I can say, this is exactly what is going on far too often. – DevSolar – 2017-06-14T13:16:13.220

6As sad as it is, that's very common. Even worse, the offenders are usually "schooled" while the ones not doing this are typically self-educated. The first group learn to pass classes, the latter learned the language and how to use it. – Gypsy Spellweaver – 2017-06-14T16:35:40.737

I don't agree that CS students should take tests on paper, but I really like this answer. – ShadSterling – 2017-06-14T17:53:38.727

I know what you mean. I similarly upvoted an answer I disagreed with (@Winterborne) because I thought it really contributed to the discussion. – Ben I. – 2017-06-14T17:55:20.600

9I learned to program long ago without an IDE. (Shoot, I did some of my learning on punched cards and teletype.) When my co-instructor pointed out that our beginning students seem to just chase the wiggly red lines around the screen, I was horrified. Visual Studio should have a mode that lets you just get something typed in before it starts trying to 'help'. I have seen students take nearly perfect code and turn it in to garbage because they were trying to solve the wiggly lines instead of coding their solution! This is completely aggravating. How did we end up in this place? – None – 2017-06-14T20:17:10.043

1@nocomprende I think we got here because the IDE helps experienced programmers spot typos. But I do see a cost in comprehension among my students, and I have to fight it vigilantly. – Ben I. – 2017-06-14T20:21:27.093

1@nocomprende This also ends up making it even more difficult to figure out what was wrong in the first place, since by the time you are looking at the code they have no idea what it does and no idea where they veered off the correct path. – IllusiveBrian – 2017-06-14T22:31:33.547

@IllusiveBrian Right. If you don't know where you are going, any code will get you there. – None – 2017-06-14T23:35:27.073

1@GypsySpellweaver ...the offenders are usually "schooled" while the ones not doing this are typically self-educated. I completely agree, having a Computer Engineering degree myself (like Computer Science, but with the fancier parts of software replaced with circuit design). I learned to take tests as a separate skill in its own right, and graduated with a 3.9 GPA, so I definitely know how this works. However, I also have a (perhaps rare) personal desire to actually know what's going on, so you might actually call me self-educated with some formal help. – AaronD – 2017-06-15T01:00:19.660

22

Although most of the answers here seem to be in agreement that coding tests should be done on paper, I would like to offer a different opinion. Any test that includes coding would benefit the student, and most likely the teacher, if it was done on a computer with an IDE for the appropriate language rather than on paper.

Since the question contains the tags for High School and Undergrad, I will focus on tests which are common in lower level programming classes which are often focused on a single language and contain significant amounts of coding based questions.

Here are a few of reasons:

  • Writing is messy - Not everyone has the best penmanship skills. This is not just an issue of readability, on a paper test space is often a premium and so creates an issue for anyone who is not able to write small enough to fit the entire solution in the space provided. I have had at least a few instances I can recall where I miscalculated the number of rows my solution would need, prompting me to either erase parts of my code to make the writing more condensed, or else doing an awkward scrawl in the corner to fit the last line. This could be solved by providing sufficient space for the question, but since this is completely reliant on the instructor's assumptions about the intended solution and the size of the writing this assumption rarely hits the mark.
  • Writing is slow - It's been a while since I sat down and wrote anything of any significant length, but it's always painfully apparent whenever I do that my typing speed has far outpaced my ability to write. Not to mention that after a few pages, my hand starts to cramp from the unusual strain. In the age of smart phones and PCs, I imagine this is representative of the majority of the population, and should be moreso for CS focused individuals. Tests are timed. Forcing a student to use a slower method of recording information adds additional strain which can resolve itself in many different ways. Typos, bad naming practices, less readable code, take your pick. Worse, it creates a sense of pressure, where even if you know exactly what you need to write for a perfect answer you are limited by how fast you can get it down on the paper.
  • Writing is not representative of their past experience in the course - I will lean on my undergrad teaching assistant experience here to say that students rarely have to write any code on paper throughout the course, except for tests. All homework assignments are submitted online to ensure the submission timestamp matches the due date. Instructor notes that include code are available online to streamline the lecture process. All coding is done on the computers in the lab portion of the course. Major coding assignments are also done on the computer, as the first check(and a decent chunk of the grade) is "Does it run?". And then they get to the test and suddenly all those skills and muscle memory they've built up over the course of the semester is not directly applicable to the problem at hand. There are too many things that IDEs simplify that simply go unnoticed until you have to do without. And while some may say that that is exactly how it should be during the learning process...
  • Writing is not representative of real world experience - I can happily say that besides a couple of interviews, I have not had to write out my code out on a sheet of paper. I have used a white board on occasion to brainstorm ideas or write out chunks of code for others, but I was not required to present a solution that was polished enough to work "as is" if typed into a computer. Because it's not the syntax that's at the crux of the job. It's the ability to reason through and come up with a solution to a particular problem within the bounds of the programming language(s) you are using. The syntax is secondary. It's why APIs, and other manuals exist out there. It is not your job to memorize the names of the methods of all the libraries that are available to you. All you need is to know which does what, and how to best leverage them to achieve the solution you are trying to implement. Unfortunately, written tests treat syntax as part of the grade. And if you can't recall the name of that one conversion method part of the Integer class? That will usually cost you quite the chunk of points.

So, in conclusion, I believe there are significant downsides to forcing students to take tests that require coding solutions in the written format. Paper tests enforce memorization and a focus on the syntax of the language rather than concentrating on the problem solving and abstract thinking that is at the heart of programming.

This answer is based on my experience with the US educational system, and so may not be directly applicable to other countries.

Winterborne

Posted 2017-06-13T19:51:53.660

Reputation: 321

2Nice contribution, but I don't agree with your last point. "Writing is not representative of real world experience". In our daily work, we do many things that does not require pen & paper, like fixing compile error, writing simple script, querying database. We did it hundred of times, so we don't have to resort to paper. But when a new problem comes, for me it's mostly the case of finding an ad-hoc algorithm, I found pen & paper really useful. – hucancode – 2017-06-15T09:13:44.570

@hucancode This could be a difference in approach. I tend to resort to pen and paper when I need to write out a logic chain, or a particularly tangled set of business requirements. This rarely takes the form of code, and is a far cry from a fleshed out solution. It provides a structure for the code that I will write later. This is very different than what is required on pen and paper tests, where you are expected to present a code that will compile with no issues if plugged into a computer in order receive top marks. – Winterborne – 2017-06-15T17:19:14.143

Thankyou for this answer. We also don't write code in an editor the same way that we do on paper. On paper it needs to be right, or we have to scrap it and start again. On a computer... well, lets take TDD as an example shall we? in TDD you begin by taking the simplest requirement, (writing a test) writing the code to match that, and then repeating until the algorithm is correct. This involves a lot of editing, but none of it is "trial and error". The editing is simple on a computer, but on paper? – Baldrickk – 2017-06-21T11:11:30.683

I have to conditionally agree with this based on the professor. One exam that I wrote on paper (about 60% code) was judged by whether the code ran when put it into an editor verbatim. Things like a simple variable name typo would cause students to get quite literally a 0 on the question. As expected, that class had approx. 60% failure rate. If a professor is going to be this strict about marking, I definitely believe not even necessarily an IDE but at least a text editor should be considered. The exam when I retook the course was on a text editor and it was significantly less stressful. – RageCage – 2017-08-28T14:24:20.487

4Upvoted because I believe this contributes to the conversation, though I obviously disagree with your conclusion. How do you get around the problems in my post and in @AnoE? – Ben I. – 2017-06-14T16:49:54.343

@ben-i I think AnoE's answer covers more advanced courses where coding is not the primary focus. If so, they may be entirely correct, though I would still argue that typing responses rather than writing could be of some benefit. Your answer has a good point of being able to fake understanding with the help of prompts, but time is often a limiting factor in a test. There is only so many random changes you can try before running out of time. This should not be a common occurrence unless the time given is much greater than the difficulty of the task. – Winterborne – 2017-06-14T17:20:57.863

4I think some of the criticisms you have (it's easy to get syntax wrong/not remembering certain methods can be fatal) can be solved by simply being more lenient with minor syntax issues while grading and by providing students with a cheatsheet listing all the built-in methods they might need to use (and perhaps even providing examples of syntax). – Michael0x2a – 2017-06-14T17:53:21.673

3Another potential problem with doing tests on computers is it makes it easier to cheat, unless the instructor has full control over the computers being used, which is often infeasible due to budget constraints or large class sizes. If the school has a competitive majors system and the intro course is an important part of admissions, that only compounds the problem. (And of course, this is always a potential problem in high school, since students want high grades for college admissions). – Michael0x2a – 2017-06-14T17:55:30.787

5I'd also like to point out that especially in US colleges/high schools, students are expected to take a variety of other classes in other disciplines unrelated to their primary interest as a part of a 'general education' requirement (literature, history, other liberal arts classes). It's common in such classes to require students to write essays and short answers on papers during exams. As a result, I think it's reasonable that STEM classes can expect students to do a comparable amount of writing on their exams. – Michael0x2a – 2017-06-14T18:01:54.840

1And finally, about your comment up above -- I don't think all instructors try and add time pressure to their exams. Some instructors deliberately remove time pressure by giving more time then they think is necessary for the benefit of students who don't work as well under pressure/to give students the opportunity to answer all the questions to the best of their ability. (After all, if a student submits an incomplete answer, how do we know whether that's because they didn't understand the material or because they're slow at writing/typing and ran out of time?) – Michael0x2a – 2017-06-14T18:07:14.190

1@Michael0x2a My own experience would disagree with most of your points. However, let me address it point by point. Cheatsheets with syntax provided by instructors could solve a part of the problem. I have never seen it done, and I doubt a useful one could be put together easily though. And again, this could as easily be solved by the functionality of an IDE that will tell you that a List has the add method, rather than include, or something like that. Cheating IS a huge issue that would need to be addressed. Disabling/Removing the WiFi adapters is a possibility if computers can be provided. – Winterborne – 2017-06-14T19:12:38.263

@Michael0x2a I do not think that applying the standard of Gen Ed. classes to CS is either fair or relevant. By that logic all classes should require students to write at least 3 research essays per semester/quarter because English does. And finally, regarding the time issue. Yes, most institutions allow you to request extended time for tests. However, I have never went through the process myself, so I am not sure how well "I'm not used to coding on paper" will fly as reasoning for that. Besides that, I have often seen about 50% of class take all of the allotted time. So it's not that generous. – Winterborne – 2017-06-14T19:18:09.490

4I write on the whiteboard at the front of class, and my writing is poor. Real programmers write on whiteboards in meetings. If they cannot sketch out a solution, they cannot be part of a team. – None – 2017-06-14T20:20:48.743

20

Speaking as a former student and as a coder: yes.

Perhaps there's an element of in-my-day geezerism here, but under the assumption that by "on a computer" you mean "in an IDE" I'll point out that when you have to write it out, you don't get:

  • syntax highlighting
  • autocomplete
  • as-you-type compiler checking
  • new project wizards ("What was the signature for main again?")
  • ...

Remembering to end every line with a semicolon without the benefit of syntax highlighting may seem like tedious detail, but if by the final you haven't picked up that habit, doesn't that indicate something about you as a coder?

If you say that as a teacher you don't care about the semicolons, you just want to see the algorithm, then fine. Don't count off when they're missing. But don't let it be a "gimme".

G. Ann - SonarSource Team

Posted 2017-06-13T19:51:53.660

Reputation: 1 029

1But do you think that this also applies to more advanced programming courses? – Safirah – 2017-06-13T20:53:52.063

1@Safirah What's your definition of 'advanced programming course'? – Pikalek – 2017-06-13T21:05:55.977

I'm talking about a test that tackles concepts like: OOP, visual programming, etc... – Safirah – 2017-06-13T21:09:37.660

I guess your "yes" is meant to be a "no"? – Paŭlo Ebermann – 2017-06-13T22:54:19.950

2Will you say the same if the students are given some normal text editors with ability to compile rather than IDEs ? – A---B – 2017-06-14T00:30:58.807

1As far as I'm concerned programmers shouldn't worry about things the system is going to deal with. I don't memorize anything like all the signatures--I know basically what's going to be wanted but let the IDE show me the details for anything I don't use all the time. – Loren Pechtel – 2017-06-14T04:24:54.627

I'd say this is a little too purist for my experience in the software industry; I think having students being able to reason out a problem in broad strokes and pseudocode on paper is a good thing, but the IDEs and dev-tools are there to help. This would be like saying that a worker in a car factory should be able to hand assemble every part of the car instead of relying on the robots. Technology and tools move on, even with the benefit of an IDE, a bad coder is a bad coder, doesn't matter if it's on a computer or paper. – Clint – 2017-06-14T07:32:33.743

5As a uni student and programmer, in my experience the courses have two parts. The first is assignments and/or computerised tests. These are throughout the course, heavy in programming and done on a computer, usually at home or in labs. The second is exams. These are the end of semester ones. They almost never expect you to write code and rarely expect you to read code. This is because the course tests your coding ability through other parts of the course. The exam is about verifying that you understand the concepts that they have been teaching in the course, so are typically wordy answers. – Programmdude – 2017-06-14T07:51:01.873

14

As a student who went through 4 years of undergraduate in computer science, writing all my exams on paper, I do think there are advantages to a paper exam.

  • If you need an IDE to tell you the things that are missing, then do you really know your material and syntax?.

  • Having an environment to test your code over and over isn't really practical for examination purposes or the real world. You should understand the obvious cases and edge cases while writing your solution.

  • As a lot has already mentioned, CS is not purely coding. From personal experience, I would say my undergrad was 75/25 split between theory/coding. Anyone can code, but learning the theory behind algorithms and best practices is why we are computer scientists.

  • Continuing on the point above, courses with heavy programming involved usually had a 60/40 split on grading where 60% weight is from code and only 40% from paper exam.

  • It is a lot easier to ask high level questions such as draw a 2-3-4 Tree, how a certain step in heapify or graph theory looks on paper than on a computer.

To address your point of mistakes being too difficult to detect even with a computer, then it is even harder on paper. No, as a graduate and TA, I can tell you when I'm grading first/second year exams, I can clearly see if they do not understand a certain subject or leaving out edge cases. We do not deduct marks for brace brackets, semi-colons, etc.

In conclusion, I think you are mixing the concept of coding challenges such as ones from Leet Code where it is purely coding based, with a discipline that incorporate a lot of theory and algorithms

Kaneki

Posted 2017-06-13T19:51:53.660

Reputation: 760

1I agree, there is a definite difference between concepts and code. Concepts should definitely be done on paper. It is far easier to sketch a diagram etc than it is to answer on a computer. For programming tests though, doing it on paper is awful. – Baldrickk – 2017-06-21T11:13:03.473

1I appreciate getting a student perspective. Thank you for posting. – Ellen Spertus – 2017-06-14T21:18:14.487

12

Speaking as a former student, a coder & a teacher: yes.

In addition to the points raised by G. Ann, I would add the following:

  • Problems on tests are typically far simpler than full real world problems. It's reasonable to expect increased attention to detail (missing semicolons, etc) when there are fewer details to attend to.
  • In a writing class with long answer question, I would expect proper use of grammar & punctuation. Common programming syntax is not different enough to warrant special treatment.
  • The instructor is free to adjust the distribution of points as they see fit. I typically gave most of the points if the general shape of the solution was correct & held some points back for students who could provide compile ready answers. Different course / class /teacher combinations may have different needs. Good teachers adjust things as needed to provide the best possible learning experience they can provide.

Regarding OP's comment about advanced course - I found advanced courses don't lend themselves well to asking programming questions on exams. Students should have covered programming concepts in intro courses. I prioritize concepts over implementation details in upper course exams & leave the implementation details for implementation assignments / projects.

Pikalek

Posted 2017-06-13T19:51:53.660

Reputation: 414

12

Speaking as a former student, a former competitive programmer, a real world programmer, yes.

Forcing student to write code on paper is not pointless. It has following benefits:

  • Giving you syntax awareness, avoid computer generated code. Therefore giving you deeper understanding of the concept.
  • You will find a way to write shorter code, since you can't copy-paste.
  • You will be more careful, since you can't undo/redo. You will develop the habbit of drawing the concept before going into the code. My personal experience is, don't open IDE before you have a solid solution in paper.
  • Pen & paper is always nice for sketching up idea. You should get used to them.

hucancode

Posted 2017-06-13T19:51:53.660

Reputation: 121

@SeanHoulihane, I agree. – hucancode – 2017-06-15T08:38:29.570

@SeanHoulihane The number of folks who occupy that category are greatly outnumbered by those who think they occupy that category. – TemporalWolf – 2017-06-15T19:54:06.120

4You have to acknowledge that for some (frequently extremely talented) individuals, this forcing will be stressful and counterproductive - but as a generalisation you make very valid points. – Sean Houlihane – 2017-06-14T09:18:23.770

The pen is mightier than the MSWord. (the pencil is mightier than the pen: it has an eraser.) – None – 2017-06-14T20:18:46.943

11

Maybe some complementary points to what as already been said (as a CS assistant professor, giving both theoretical and practical courses).

  • Even for practical courses (those which involve coding), writing code on paper is good. The point is not to fill your page with boilerplate code (and I cannot stress that enough). Writing sequence of Java imports or mystical incantations to setup say a Map/Reduce transformation in Hadoop on paper makes absolutely zero sense. However, being able to write correct application logic and being aware of implementation aspects (some object can actually be null, problems with side-effects...) is a must. The hard part (which is actually an art I have not fully mastered yet), is to make exams that tests theses aspects without the noise and being of course lenient about small syntactic mistakes (missing ; capitalization and so on). At some level, what I want to test is this: "Here is a problem (from which I have abstracted the noise for you). Can you solve it with programming, without having to compile it 10 times until there are no more syntax/type errors and without running it a 100 times with printfs all around to see why this does not work ?". Because if the student can do that, then I know (s)he will be able to take it to the next step and generalize it to harder problems or use it in other contexts. If a student takes whatever is in the lecture notes, paste it in an IDE and fiddle with it until it somehow works, I cannot make the same guarantee. Also, you cannot copy/paste on paper, which forces you to think about your code and write it in a terser manner. I have to initialize an array. Ok I'll write a loop rather than cut/and paste 25 lines and change each one.

  • Conversely, even for theoretical classes (complexity/calculability, type-systems, logic , ...), I think it is good to have a bit of coding. Write a lambda calculus interpreter. A small (even naive) SMT solver. Code several sorting algorithms and try to see what constants are hidden inside your big O. Is the impact of sorting in place noticeable without resorting to having to copy your data? In highly advanced courses (in particular type systems and logic) you only really see what's going on when you can test large examples. Pen and paper only take you so far.

Kim Nguyễn

Posted 2017-06-13T19:51:53.660

Reputation: 441

9

Your question has an implicit assumption: that CS is programming and what you're learning (and will always learn) in CS is programming. That's not necessarily true. Even though an introductory course "teaches" a language, probably something like Python, C/C++, or Java, in actuality you're learning a type of programming language, a procedural language (control flow) or OO. Ask someone who's learned Scheme as their first programming language how different it is from C.

Following on from intro, a data/discrete structures class (the second class in CS) could be taught language agnostically, but really you're learning things like sets, relations, and combinatorics which can all be tested about conceptually. Anything beyond discrete depends on the class. An OS or networking class could have C as the required language since it's "systems" class, along with an upper-level undergraduate/first-year graduate database class. And exams in those classes aren't about programming, rather more along the lines of "When is a semaphore better than a mutex?" or "What's CIDR and why is it important?" If your DB professor is unkind, they'll ask you to give pseudo-code for a B+ tree and ask you a relational algebra question. Graphics sometimes introduces OpenGL, but concepts are taught and tested mathematically, something along the line of given a 8x8 grid of values, what's the interpolated value at some point in the grid. Volume rendering is expressed as solving an integral. Image processing can be taught as a signal class, i.e. use PDEs. Scientific computing could ask why is Householder more stable than Gram-Schmidt.

Turning towards other CS things, algorithms (runtime complexity) and theory (abstract machines, automata) are all done with pencil and paper. Ultimately, the higher you go, the less programming is taught and more concepts are taught and then expressed through programming.

TL;DR Yes.

merelyMark

Posted 2017-06-13T19:51:53.660

Reputation: 91

2+1 (+2 if possible) for the first paragraph. I suspect this is whymy first language at Uni was gcl, for which there was no compiler. – Hennes – 2017-06-16T16:48:26.337

2this answer does answer the question, but it's not entirely to the question. The question asks about written tests about code, and not about cs in general. You are right that CS is more than just code, but that is not relevant to the question. – ItamarG3 – 2017-06-14T06:20:51.033

@ItamarGreen maybe the implication is: "if it is not a coding question, it should be answered on paper"-? – None – 2017-06-14T20:25:06.857

9

Having taken and TA'd for classes which use both approaches, I would have to say that it all depends on what you want to see from the students.

On the side of having paper exams, it has already been mentioned that CS!=Programming. If you're trying to test concepts of CS, there doesn't have to be a single line of code written on the exam, and a paper exam will probably give you a lot more insight into how a student is doing on comprehension than allowing them to use an IDE during an exam.

In that vein, you can ask:

  • multiple-choice questions (hard to write well, but can be extremely useful in large classroom situations)
  • open-ended "explain to me how you would think about this" questions (some people mentioned handwriting being a problem, but in grading 100's of students' papers, I never found one I really had a hard time reading)
  • human-computer "what would the state of the program be after X iterations" questions (tests recall knowledge, which is sometimes all you need to tell how well they're picking up a specific language)
  • fill-in-the-blanks in code, e.g. "I'm trying to write a summation program; what needs to go in the blanks for the program to compile and run correctly?" (allows you to test specifics of syntax knowledge without worrying whether they're just re-enacting Ben's scenario)

On the side of computer exams, I've had a few of those too, and I think they can be useful in making sure your programming assignments actually taught what you meant to teach. I had a Data Structures professor who gave us so much sample code "to get us started" on our homework that most of the class failed and had to retake the final exam which was done on a computer, in an IDE. The class had gotten used to waiting for the "starter code" before starting their homework, so when the final exam asked us to write a stack instead of the linked-list we had written for the homework, most of them were stumped.

If you really want to have "programming" exams (possibly to test the sort of behavior I just mentioned), then I would suggest that comments should count for a large portion of the grade. This does two things: it allows students who are not quite as fast at churning out code to explain their solution, and it makes it so that that inevitable debugging nightmare that you know to look for but they don't doesn't have to completely tank their grade. It also gets them in the habit of commenting in code, so please, for the sake of every professional developer out there, grade their comments!

In my experience, it's a lot harder to write, and to grade, a "program this now" type of test that actually tests the concepts of CS than a paper one. As I said, I have one very salient exception to that rule in my experience, and if you're worried about academic dishonesty, programming tests on a computer that can't access the Internet are sadly about the only thing you can do. But if you have the option of a paper exam, it's usually a better fit for the subject.

Kristen Hammack

Posted 2017-06-13T19:51:53.660

Reputation: 221

1Instead of "comments", I would consider outright asking for a (non necessarily super formal) proof of correctness. I value those immensely because I remember how I used to program before those were hammered in my head in Programming 101. It was not pretty. – Tobia Tesan – 2017-06-16T19:51:49.650

Very good points made here. I have seen that method of providing "starter code" but I never took the bait. It just felt weird to me. – None – 2017-06-14T23:47:46.400

8

I think this question is perhaps mis-framed or at least misleadingly framed. Fundamentally, the question an educator has to answer is: what knowledge do you want your students to demonstrate? Once you know that, the next question is how do you strip away all other distractions so that whatever product you evaluate gives you a good proxy for what knowledge your students actually acquired? The real issue here is how to tailor the evaluation medium to the desired learning outcome.

I think written-response tests (regardless of whether they are "written" with Word or a pencil) are tremendously valuable because they let you probe the degree to which students understand the theory. For the same reason, I think significant programming assignments are also tremendously valuable because they offer different information: is the student able to translate theory into working code? This approach gives you a much broader view of the student and gives you the opportunity to understand any inconsistencies. E.g: a student does awesome on coding assignments but sucks at written tests, maybe there's a learning disability that could be better accommodated?

notquitezeus

Posted 2017-06-13T19:51:53.660

Reputation: 81

8

As much as I hate the fact that we need to consider this, paper and pencil makes it harder to cheat. Even if you restrict them to using lab computers and not their own laptops, and cut off networking to the room, it'd be easy to smuggle in a USB key with entire books worth of notes and the source code from every one of their labs and projects. (Yes, with enough effort, you could lock down the computers to the point where they can't even mount a USB key, but the logistics are a lot trickier to manage than just scanning the room periodically to see if anyone's got a small library of cheat sheets on their desk.)

(I don't think that this should be the only factor considered, or even a deciding one, but other answers have covered various pros and cons well enough already, and this is a factor that hasn't been mentioned yet.)

Ray

Posted 2017-06-13T19:51:53.660

Reputation: 459

1Good point. It is a factor, so far overlooked. – Gypsy Spellweaver – 2017-06-16T02:19:53.450

I don't know - having the K&R on hand and the previous homework seems borderline acceptable to me, under the assumption that the test is not essentially a rehash of homework but offers instead an entirely new challenge. And, are you telling me disabling USB ports on lab machines isn't the standard already? :) – Tobia Tesan – 2017-06-17T17:34:13.547

@TobiaTesan If you decide it's acceptable, then you're naturally free to allow it. But so long as there's any amount of reference material that you don't want to allow, there's a benefit to making it harder to smuggle in. As for USB ports being disabled by default, that probably varies from place to place. We certainly don't do it here; we want students to be able to backup their work. – Ray – 2017-06-19T22:27:14.430

7

When I went to college, we had 2 exams. A practical and a theory exam. The practical was on the computer and we could use all resources including the internet. The theory exam was basically what you describe a paper and pen closed book exam where we would either write code or find the error in some pre-written. There were also standard question as well.

I see value in both methods. If I were teaching a course, I would probably do the same thing. Give a relatively hard practical exam (because students have all resources at their disposal) and also give them a written test. This will not only test their memory of syntax, but also give them a chance to really build something.

SaggingRufus

Posted 2017-06-13T19:51:53.660

Reputation: 171

1My students have a 'theory' exam on paper in my course tomorrow. Cross all 1010 of your fingers... – None – 2017-06-14T20:22:07.023

6

If some mistakes are difficult enough to not be noticeable while writing
code on a computer, they are nearly impossible to be found while writing code on paper

I don't see how this follows in general.

It depends, of course, on what the test is testing: if the test is about finding the one incorrect value in a 2000-entry trig table I would expect a computer would be of help, but that's not what a CS test usually looks like, is it?

I expect a good test would in general measure the student's understanding of a topic, not their ability to mechanically carry out repetitive tasks.

So, my question is, are there any advantages in doing written tests on paper, and is this the best approach?

I am going to take a shot in the dark here and make an assumption about your question: your question is about programming and the "test" is writing a program.

Under that assumption I would answer with a loud "yes" - you'd want students to be able to reason about programs and convince themselves of the soundness of their reasoning, not just go by trial and error until they converge to a working program by sheer chance.

That's, after all, no way to program in real life.

On the other hand, I would also have a lab test to measure the student's ability to actually compile and run a program and/or launch a debugger - after all, once you've produced a correct program you still need to compile it to be useful...

Tobia Tesan

Posted 2017-06-13T19:51:53.660

Reputation: 247

@nocomprende to be honest, I don't know how much you can "play around the edges" if the problem statement is "give a procedure to insert an element into a BST"... – Tobia Tesan – 2017-06-15T17:07:56.403

4One sign I notice when students don't quite have the knowledge to solve a problem is that they really double down on things that they DO know. They become very innovative and churn out code that plays around the edges of the problem, enhancing the I/O and things, when a 10 line program would have demonstrated the concept adequately. I guess there would be less temptation to do this on paper. (Or if you are using a batch system with punched cards and only get to submit it a few times a day. Ahem. Er. Was that me?) – None – 2017-06-14T23:56:20.330

5

Depends on the course. For more theoretical CS courses, paper and pencil would be fine. But for the more "hands on" language courses that involve coding, a computer-based exam would be best so the students can concentrate on the thinking part of creating code and not on memorizing all the syntax nuances. There are too many in modern languages for anyone to remember. If all that is expected is to write "pseudocode" to demonstrate knowledge of an algorithm or design pattern, paper and pencil would be fine.

Kumar

Posted 2017-06-13T19:51:53.660

Reputation: 51

1Welcome to CSE! – Ben I. – 2017-06-16T19:59:52.273

4

Some students (like me and my teenager) write slowly and almost unreadably, so nowadays it is common to allow them to use a keyboard for exams.

Computer based exams can be made better than paper, in terms of drilling down to the edge of the students understanding based on their answers.

Dominic Connor

Posted 2017-06-13T19:51:53.660

Reputation: 181

1I like to have students draw flowcharts at some points. This is very easy with a pencil. One student was disabled such that he could not use a pencil, but man, was he great at Visio! Of course, I made the exception during the test for this student. – None – 2017-06-14T20:26:39.450

3

I guess I’m late to the party, but I feel like there’s still a bit missing from the discussion. It’s too easy to say “CS != Programming” and get a round of applause. It’s also too easy to try to generalized from our own experiences: most of our students are not going to grow up to frequent csEducators.stackexchange. I think Winterborne, Ben I, and notquitezeus are right to start by thinking about who our students actually are, and what our course goals are.

Most of our students are at the introductory level, so let’s focus there. At my small, liberal arts college, the current intro CS student does not fit the stereotype of even a few years ago. Most are interested in something other than CS, but want to learn a bit of CS to be more effective at their other major. They are probably weaker on average than students who took CS when CS was a smaller discipline “only for the elite.” I for one, want to welcome these students to the CS world. My main course goals are:

  1. Give them a positive experience.
  2. Convince them that CS is everywhere, and empower them to use it.
  3. Encourage them to take even more CS. The intro course is not meant to be all-encompassing.

I try to avoid outdated teaching methodologies that historically would have pushed these newcomers out of the field (and, unfortunately, historically pushed out women and minorities in particular). I’m talking about things like macho programming challenges or extremely difficult labs that require all-nighters. For me, the jury is out regarding whether exams should be on paper or with a computer. I totally agree with Winterborne’s point that “Writing is not representative of their past experience in the course.” Many students hate the pencil and paper exams, don't feel empowered by them, and don't want to take more CS after the exam, running contrary to my course goals.

While the class focus is on algorithms and computational problem solving, rather than programming, exams still tend to be 10 questions of the form "write a function to compute ...". Many of the answers on this thread seem to go against IDEs, but the OP didn’t mention IDEs at all. I’m not convinced that an exam in something like IDLE would really give me less info than a paper exam. It would give me different info. By allowing students to actually run their code, an exam in IDLE would let me test their ability to quickly come back from a mistake, rather than brute force memorization skills. It would test their ability to understand and respond to error messages. Crucially, I believe working in IDLE on the exam (as they spend most of their time in the class, HW, and labs doing) would reduce test anxiety and stereotype threat, and this does line up with my course goals.

What are the downsides? Sure, error messages will point out their syntax errors so they can fix them. But that’s fine with me, because I’m testing ability to come up with algorithms, not knowledge of syntax. Exams meant to test syntax would run counter to my course goals. How about Ben’s example of confusing items and indices? This is certainly a common mistake. For some, it’s because they are rushed and sloppy. For others, it’s because they really don’t know the difference. I think that, even if they are getting help from error messages, I’ll detect the 2nd group of students. They are the same students who might not even realize a loop is required, and error messages are not going to help with that. Also, if Ben is concerned about detecting just how lost they are, he could have the computers record the history of error messages generated as students take the test, and could automatically browse it for particularly bad errors.

I'm not saying I necessarily think exams in IDLE are better than on paper. My colleagues are all against this idea. But I made a similar switch in Statistics and I'm convinced it was the right move. It allowed me to test students' abilities to synthesize resources I'd given them to quickly conduct a new applied data analysis. It traded rote memorization in return for an environment where students can showcase their skills and can go as in-depth as they want. CS is obviously different, but for me at least, this question deserves more thought.

David White

Posted 2017-06-13T19:51:53.660

Reputation: 484

P.S. I know that students could also "run their code" without IDLE. But, no matter how many times I tell them to “pretend to be the computer” and “write out what your code actually computes”, or make them do this in class, experience has shown that most will not do this on the exam. I'm open to suggestions for more conceptual test questions (at the college level), or for ways to really get the weaker students to actually catch their disasterous mistakes when they only have pencil and paper. – David White – 2017-06-19T18:05:49.963

P.P.S. we mostly hit our course goals via programming labs where students run large-scale simulations of real-world phenomena (often with some starter code). All credit to my colleagues for that course design. Unfortunately, exams remain 50% of the final course grade, and labs/HW don't really prepare students for them. I spend most of my in-class time on practice problems that are meant to be like exam problems, but students spend most of their time in front of IDLE on HW and labs. That's what got me thinking of changing assessment to most closely match student time expenditure. – David White – 2017-06-19T18:07:24.000

You've given me a lot to ponder here. I certainly agree wholeheartedly with the goals you've outlined, but I am not sure I agree with your conclusions - among other things, I don't find it difficult to discount syntactic mistakes entirely except for the occasional (very odd) edge case. – Ben I. – 2017-06-19T18:08:31.587

2

This thread is so long that I'm not sure that I picked up every point made, but I want to add an orthogonal thought that I missed if it is present.

Any timed exam, whatever the technology, old or new, will advantage some students and disadvantage others. Time itself can be the mind killer. I don't have a real answer for how to overcome it, other than to be constantly aware of it, and compromise when needed.

Your students are not like you. They are nothing like you. They are very little like one another, actually, with different learning modalities.

However, toward the end of my career I deemphasized formal exams (even finals when possible) in favor of papers and projects. Yes, I know that there are dangers with that as well, especially plagiarism.

Buffy

Posted 2017-06-13T19:51:53.660

Reputation: 21 033

2

It depends.

As an undergraduate student, based on experience of exams that have been mostly paper based, I'd say:

For more 'theoretical' exam questions (describing algorithms, reasoning about time complexity, designing automata etc) paper is absolutely the way to go. It's the best way to handle mathematical questions and it's easy to clearly lay out an answer.

But this doesn't mean paper is a good way to assess knowledge of programming languages. I've had a few exam questions require me to write over a page of C or Java code, and these are just a pain. As an example, one question asked for a UML diagram to be converted into object oriented code. This would have been better assessed by asking "State the visibility of the field X on the entity Y?" or "Write a method signature for X" - having to write reams of boilerplate code in an exam is not a good use of time and isn't at all a good indicator of the candidate's ability to work in an actual software engineering role (which I assume is what's supposed to be being assessed, because it's most certainly not CS theory).

Adam Williams

Posted 2017-06-13T19:51:53.660

Reputation: 658

1Welcome to CSE! I hope we hear more from you in the future. – Ben I. – 2017-06-19T12:41:47.390

2

Our program teaches top-down design. This is an iterative process in which a text editor is indispensable. You stub in functions and methods, spec them out, and then write test code for them. After this, you begin writing code. If a method or function gets too complex, you refactor into reasonable-sized chunks.

This does not work well on paper. Stuff you put on paper is "dead."

Some of our assessment is paper and pencil; this is fine for the small stuff. A lot of our assessment comes in in the form of projects. For this, we assign complete projects of escalating complexity.

ncmathsadist

Posted 2017-06-13T19:51:53.660

Reputation: 1 658

Isn't the question about tests, though? I don't imagine anyone is arguing that CS projects should be on paper. – Ben I. – 2017-06-20T22:08:41.130

My tests are lab practicals. These involve problems of escalating complexity that demonstrate various levels of proficiency from basic to advanced. I do not expect students to get 90% of this; I am looking for their level of proficiency demonstrated by what they can create. – ncmathsadist – 2018-01-06T01:14:06.280

2

Simply - yes, but not always. Paper-based programming exams make students think differently than they do in front of their editor. What's wrong with that? Nothing (as long as it isn't always that way). What's the benefit? Thinking differently.

Brett Becker

Posted 2017-06-13T19:51:53.660

Reputation: 304

1

One boring answer:

One big advantage of making students sit in a room in silence and write on a piece of paper is the best way of know of to ensure that the student has learnt the material, and is not paying someone to do all their work for them.

Of course this isn't a perfect system, but it is better than most things we have.

The question of why do those exams on paper instead of on a computer is often for boring reasons -- it is still hard to lock down a computer sufficiently to ensure no cheating, and also ensure it won't crash mid-exam. Paper has no such issues.

Chris Jefferson

Posted 2017-06-13T19:51:53.660

Reputation: 281

1

Save the trees

Requiring the exam be completed with pen and paper is a ridiculous anachronism.


  • If you are teaching something practical, like, say, Java 101, there is absolutely no point in asking a student to write Java code on paper. Nobody, not a single person, needs this. Why not use clay tablets if you are going for hard labor?

  • If you teach something like automata theory, then use a language capable of embedding proofs of what you teach, and require that students complete their tests by submitting proofs in that language. There is not a single reason why computer scientist wouldn't know how to use the most widely available form of computer as a tool in her work. And if proofs aren't possible or are too difficult, then use code that illustrates the solution. Anything other than that is "hot air"; feels unjust and discriminating due to the randomness of the opinion of the person responsible for verifying such test.


Some background info

I've studied CS and was briefly a TA. I had to submit exercises in Java written in pen on paper. I also saw the exercises submitted by other students. None had even a correct number of opening and closing brackets, let alone other typos.

My other background is in printing and typography. Having worked for a while in a publishing house, I assure you that the number of mistakes in mathematical and programming-related books is astronomical when compared to regular prose. It is not humanly possible to verify handwritten formulas in reasonable time. Every single textbook I had on subjects like math and CS had dozens of mistakes that I was able to discover, and the denser the material, the more mistakes there were.

There are prominent mathematicians like Vladimir Voevodsky who try to change this situation by developing better foundations for automating proofs. I believe that this is the way to go.

wvxvw

Posted 2017-06-13T19:51:53.660

Reputation: 149

1"Ridiculous anachronism" Is it safe to assume that you believe that paper should be banned in geometry and calculus classes as well? – Ben I. – 2017-06-19T12:32:45.010

@BenI. indeed so. Instead, students could use Geogebra, for example. There are great packages for symbolic math, s.a. Sage, or Maxyma (free ones) or Mathematica, Maple (paid-for). It's hard to imagine something so extravagant in undegrad, or even in graduate studies that wouldn't be covered by those. – wvxvw – 2017-06-19T12:47:12.800

Unfortunately, the kind of mindset which says "use pen and paper" pervades the offices of the ministry of education (at least where I live). Which is partly explained by incompetence, and partly by conservatism. None of which are commendable qualities for educators... nevertheless, instead of investing into improving the schools supplies, the funds are spent elsewhere :( – wvxvw – 2017-06-19T12:54:36.483

1

I can't say I agree with you. Computers don't necessarily make a great cognitive match with human brains, which are persistently analog. Our brains orient around spaces, faces, pleasures, and dangers, and learning is inherently social. I don't think that computers are a panacea, nor that they will ever fully replace analog processes. Use them where they obviously help, but be wary of falling into overly easy technologist thinking.

– Ben I. – 2017-06-19T12:59:24.947

@BenI. Now you resemble my wife when she claims all sorts of bad things about computer games. :) Are ruler and compass social or antisocial? Why would you even bring the subject of being social into this discussion? It is completely orthogonal to the quality of the tool. Do ruler and compass make a good "cognitive match" with our brain? Modern science knows squat about brain, and that would be an exaggeration. How can anyone make claims about what matches it and what doesn't, keeping a straight face, escapes me. – wvxvw – 2017-06-19T13:04:26.243

1Re' the subject of this article: I want to see anyone using Sage or Maxyma at the same speed they can type arbitrary prose... this is not comparing apples to apples. And, surely, the article is not about testing, where different skills are needed... – wvxvw – 2017-06-19T13:10:26.777

-2

Yes. If you use a computer to take a test, for instance, you will have the tendency to code and check the program if the software is already there else you verify your answers on the Internet. As a skilled programmer, you can take tests manually and to the coding part you can shorten it. The school and college tests are just to ensure that you can remember the concept well.

Jalagandeswaran r

Posted 2017-06-13T19:51:53.660

Reputation: 7

1Welcome to Computer Science educators! This answer doesn't exactly answer the question. The question asked about advantages of writing tests on paper, and you answer is addressing disadvantages of doing tests on computers. While that is related, it is not directly answering the question. Could you please edit your answer so that it addresses the advantages, so it fits the question? – ItamarG3 – 2017-06-17T12:39:35.143