Archives for : phd

HCI 2013 publication – cognitive models

I have a paper coming up at BCS HCI 2013 (September 11-13th, London), “Predictive Modelling for HCI Problems in Novice Program Editors” (Fraser McKay and Michael Kölling). It’s a predictive human performance modelling approach to learner programmer tools, carried out with CogTool. Second author is my PhD supervisor. From the abstract:

We extend previous cognitive modelling work to four new programming systems, with results contributing to the development of a new novice programming editor. Results of a previous paper, which quantified differences in certain visual languages, and feedback we had regarding interest in the work, suggested that there may be more systems to which the technique could be applied. This short paper reports on a second series of models, discusses their strengths and weaknesses, and draws comparisons to the first. This matters because we believe “bottlenecks” in interaction design to be an issue in some beginner languages – painfully slow interactions may not always be noticeable at first, but start to become intrusive as the programs grow larger. Conversely, text-based languages are generally less viscous, but often use difficult symbols and terminology, and can be highly error-prone. Based on the models presented here, we propose some simple design choices that appear to make a useful and substantive difference to the editing problems discussed.

This is a short paper, that ties up the loose ends of a previous HCI paper.

EDIT 12/09/13: I’ve posted a link to my copy of the PDF, and it’s also in open-access on the KAR (my university’s institutional repository). ACM and BCS links will (presumably) follow once available.

PPIG Publication, November 2012

In press is a conference paper I’m about to present at the Psychology of Programming Interest Group (PPIG) workshop in London, 21st-23rd November 2012. As I noted on my HCI 2012 post, some of these have now been going on longer than the participants (this being PPIG no. 24). This is a full paper written with my PhD supervisor, Prof. Michael Kölling (well known in the field of computing education). The paper reports on a pilot study where we evaluated the effectiveness of new HCI usability heuristics (which we created), specific to the domain of learner programming systems (e.g. Greenfoot, Scratch, the Mindstorms kits, etc.). The abstract follows below.

EDIT: The paper is now published, and a PDF can be downloaded for free from the Kent Academic Repository, or directly from this link. The Q&A session after the talk gave an opportunity for some interesting discussion with the likes of Thomas Green, Alan Blackwell, and the other cognitive dimensions “grandees”!

Evaluation of Subject-Specific Heuristics for Initial Learning Environments: A Pilot Study

Fraser McKay & Michael Kölling

“Heuristic evaluation is a “discount” technique for finding usability problems in well-established domains. This paper presents thirteen suggested heuristics for initial learning environments (ILEs). To investigate the usefulness of these heuristics to other developers, we conducted a pilot study that compared two groups of evaluators: one using an older, generalised set of heuristics from the literature, and one using our domain-specific heuristics. In this study, we compare not just the number of problems found, but the way in which the problem reports were expressed. There was a significant difference in the length of written comments when problems were found (those from the new set being longer). New-set reviews touch on more themes – many make suggestions about what would improve the problem; many comments refer to a suggested cause-and-effect relationship. As designers, we find this detail helpful in understanding problems. Quantitative data from this study is not large enough to support any robust conclusions about the relative thoroughness of the heuristics at this time, but we plan to use lessons learned from this study in a larger version shortly.”

HCI 2012 publication

I’m presenting a work-in-progress paper at the BCS HCI 2012, next month in Birmingham. HCI (the “People and Computers” series) is one of the big international HCI conferences, now older than some of the participants, in year 26 (I’m 23!). This is a work-in-progress paper entitled “A Prototype Structured but Low-viscosity Editor for Novice Programmers”. The abstract, below, is on the conference programme site. The paper will be published in the proceedings, and on the ACM digital library and the BCS ewics archive - which I’ll link to here later.

EDIT: BCS ewics repository (PDF – open access)
EDIT: ACM digital library (PDF – login required)

“This paper presents work in progress on a prototype programming editor that combines the flexibility of keyboard-driven text entry with a structured visual representation, and drag-and-drop blocks. Many beginners learn with Java, a traditional text-based language. While text entry is ideal for experts desiring speed and efficiency, there is evidence in the literature that a significant portion of novice errors are related to syntax. Some beginners learn with Scratch, Alice and Star Logo, all of which have drag-and-drop, “block”-based interfaces. Validation makes them less prone to syntax errors, but they are very “viscous” – there is resistance to changing or rearranging statements once they have been entered. The new system combines keyboard input with statements that can still be manipulated with the mouse as whole blocks. Standard text idioms can be used – highlighting code by dragging the mouse, copying & pasting (as text), etc. With CogTool cognitive/keystroke models, we show that the new system effectively overcomes the viscosity found in block-based languages, but it retains much of the error-proofing. Work is ongoing, but there are implications for the design of a new novice programming system.”

Interfaces: HCI for Beginner Programmers

This morning, I got my spring copy of the BCS HCI magazine Interfaces. I’m pleased to be published in this quarter’s issue, as part of the “My PhD” series. In the article, I give a brief overview of my general project, and then run through some of the usability heuristics that we’ve been developing for novice programming tools.

McKay, F. HCI for Beginner Programmers. Interfaces, British Computer Society, 90 (2012), 22-23.

Update 19/04/12: an open-access PDF is available from the BCS.

I’ve pasted the introduction here:

My PhD is based in the Programming Education Tools group at the University of Kent, and my project concerns the development of a new way for novices to enter and maintain code. Coding in schools has recently been a more topical issue than usual [1]. Most of my work so far has been about analysing existing systems, but I’ve also begun to work with some early designs. My review of existing systems has been structured with thirteen new heuristics. These are based on previous heuristics [7], the cognitive dimensions [2], and a wider review of the literature on novice programming errors. For specific features, Cog Tool models [4] have exposed subtle differences in the effort a novice might need to exert to make changes in two comparable code editors. Now, at the start of my second year, I am working on prototypes that might make it into a new programming tool.

PPIG, and mini-thesis upgrade

Back in September, I was at my first PPIG conference. I gave a quick talk on my project summary at the doctoral consortium, the work-in-progress paper for which is here:

…The intended outcome of this project is the design and implementation of a new programming language with focus on improved interactions for creating and maintaining novice programs. Evaluating the HCI in current novice programming systems (Scratch, Alice, Greenfoot) has led us to propose new heuristics for these kinds of tools. Cognitive models of viscous interactions in block-based and text-based languages have also been compared. Early prototypes have been created, with some preliminary evaluations being used to refine the ideas…

McKay, F. Work-in-Progress: Design of a Beginners’ Programming Language with a Focus on Novel Interaction Techniques. Doctoral consortium, Psychology of Programming Interest Group. (York, September 6-8th). 2011. [Google Docs].

This month, I also passed the viva based on my minithesis, which includes detailed reviews of the evidence we use in support of the new heuristics, mentioned only briefly in the PPIG paper.

A new language for beginners?

A summary of my PhD proposal has been placed on my research page at the University of Kent:

* * *

This PhD project concerns the design of a new type of programming language for teaching, one that tries to combine the benefits of textual input with those of a structured and flexible graphical code output.
Recently, the three most prominent programming tools for beginners are Scratch, Alice and Greenfoot (discussed in Utting et al. 2010). Both Scratch and Alice are visual blocks-based languages. Greenfoot’s code view is less graphics-rich, resembling more of a traditional code editor. In coding terms, there is still a large “jump” for students from visual tools to more traditional code editors, even in teaching systems.

Visual programming tools do some things very well; they can remove completely the possibility of syntax errors (Maloney et al. 2010). The tool palettes of block languages allow students to work under recognition rather than recall, in interaction terms; it is not necessary to remember all of the commands or structures, since prompts are provided. Unfortunately, these visual languages can also be restrictive. The palette means that there are only a very specific number of available code blocks. Speed of writing is also an issue. Scratch is aimed at users from 8 to 16 (Maloney et al. 2010). It uses the mouse almost exclusively; after the initial learning period, dragging and dropping blocks with the mouse might become quite tedious, especially when writing long programs. Students aged 15 or 16 might reasonably be expected to be proficient enough on the keyboard to find “pure” mouse interaction slow and time-consuming, compared to the speed at which they can type.

This project suggests looking for another option, where text-based input and graphics-based output can be combined. Greenfoot and the Karel-family (starting with Pattis 1981) have graphical output of the program execution. Their code editors, however, are text-based, and that is what is being examined here. Informally, programming editors (not the model) can be sketched on a graph (Figure 1*) of two axes: the balance of mouse and keyboard input, and the corresponding balance from textual to graphical output. Most tools are found at opposite corners of the graph, with little overlap between text-based and graphics-based languages.

Figure 1: Input and output paradigms

The proposal here is for a tool that fits somewhere in the upper-left quadrant of that graph, where we have the speed and flexibility of text input, but the student benefits from the editor visualising the structure of the code to a similar extent as is done with block-based programming, not necessarily in the same way.

* Acknowledgement: Michael Kölling


  • Maloney, J., Resnick, M., Rusk, N., Silverman, B. & Eastmond, E. 2010, “The Scratch Programming Language and Environment”,  ACM Transactions on Computing Education (TOCE), vol. 10, no. 4, p. 16.
  • Pattis, R.E. 1981, Karel the robot: a gentle introduction to the art of programming, 1st edn, John Wiley & Sons, New York, USA.
  • Utting, I., Cooper, S., Kölling, M., Maloney, J. & Resnick, M. 2010, “Alice, Greenfoot, and Scratch–A Discussion”, ACM Transactions on Computing Education (TOCE), vol. 10, no. 4, p. 17.