"The real question is not whether machines think, but whether men do." -- B. F. Skinner

 

Content

About

Reading code is a common activity; it's not unusual for a developer to spend more time reading code then writing it. However, despite it's importance, readability is one aspect of code quality that is not well understood formally.To do that, we need data from real humans. SnippetSniper is designed to collect this data by asking people to judge short code snippets based on readability.

We hope to use this data to help answer questions like:

  • Do people agree on what readable code looks like?
  • What code "features" are most important in this judgment?

We will not ask you for your name, and will not record any identifying information.  Data obtained in this study will be used to synthesize a readability metric for software.  We anticipate including an evaluation of this tool in an upcoming publication.

SnippetSniper is completely voluntary. If you do choose to participate, you will be asked to rate the readability a set of 100 short Java code snippets on a 1 – 5 scale (from difficult to easy to understand).  No special knowledge or experience is required for participation.  Most people complete the program in less than 30 min, but there is no time limit.

Instructions

Scoring Snippets

You should score snippets based on your estimation of their readability. You might consider naming of variables, spacing, comments, ... whatever is important to you.

Simply give snippets a lower score that you think are hard to understand. Give them a higher score if you think the code it clear.

If you want, you can skip a snippet for any reason. However, you can't go back.

Send Us Your Results, Comments, and Questions

Thanks for your help! Send your comments, and questions to:

Snippet Sniper

When you ready to start, click below.

Launch Snippet Sniper 2.2

 

FAQ

How long does it take?

We designed the experiment to take less than 30 minutes. However, there is no time limit.

How do I know if the code is readable?

Maybe you don't. That's what we're trying to figure out. All we're asking is that you try - we'll do the rest.

Where did these snippets come from?

Various open source projects.

What if I don't know anything about Java?

There is no requirement that you are a master Java coder, or even that you've written any Java code at all. We aren't asking you to actually figure out what the code means; just estimate the work it would take to understand it. Any previous exposure to a structured programming language is sufficient.