As I mentioned a week or so ago, we were lucky enough to be back in schools running language experiments with Real Children in July. We visited 4 settings over 5 days, running experiments with 21 preschool-aged children – a fair marathon.
It’s a total privilege to be able to do this because it takes a lot of organising on the part of the childcare providers on top of everything they already do (and, as we’re increasingly aware, early years practitioners get nowhere near the wages they deserve). Children’s time is also, quite rightly, protected, and we can’t simply waltz into their space and take it up.
This means that experimenters have to plan our experiments incredibly carefully to make sure we don’t waste anyone’s time or energy (including our own). That said, no experiment *ever* goes to plan first time, whether you’re working with children, chemicals, or quarks. This means we spent a good part of the week adjusting and refining our experiment. It was a week of *much* reflection so I thought it might be worth sharing our story and some of those reflections here.
What were we trying to do?
As I explained in my previous post, we wanted to get some information about how children produce tag questions (You like flowers, don’t you?) and negative questions (Don’t you like flowers?). These questions are really hard to elicit (that means, to get kids to produce them), so we planned an imitation experiment where we would feed children the question and then see how faithfully they reproduce it. This reveals a certain amount of their linguistic knowledge as they’re more likely to accurately reproduce grammatical constructions they have already learned than ones they haven’t yet mastered, which they will tend to amend or rephrase.
Our original experiment involved using puppets to act out a scene, then asking a child to take one of the roles on. It was based on a study involving 4-6 year old Cantonese speakers and we piloted it with children we knew well.
But when we went into our childcare settings, it was a dismal failure! Children aged 4-5 got distracted by operating the puppet, didn’t want to join in the scene or just got bored halfway through – even the really chatty, outgoing ones. Plus, when they went back into their usual class spaces, they struggled to tell their teachers what they’d been doing. Why did it go so wrong?
Lesson #1: Piloting experiments – don’t keep it too close to home
As I mentioned, we had done a dry run – a pilot – of the study with a child at the bottom end of the age range we were interested in (3 years 11 months, or 3;11) and a child at the top end (6;1). Both children completed the experiment successfully and without getting bored. But we knew both children very well and ran the pilots in their homes.
Of course, we spent a lot of time before running our experiments just playing with our child helpers, and their classmates, and making ourselves familiar. But we had still massively underestimated how much a break from routine, with someone you don’t really know, would affect children’s confidence in taking part in this (admittedly complicated) task.
Lesson #2: All the small details count
A related problem was that in the original study, this act-out task was one of just many tasks that an individual child would take part in, across a series of weeks. The children in the original study were also likely used to meeting strangers and engaging in language tasks, as they were all receiving interventions following a diagnosis of autism (thought the study was a bit skimpy on details about children’s prior experience). In contrast, our children most likely had no experience of this kind of interaction with language specialists, making the background for our respective studies fundamentally very different. We should have taken this into account more seriously from the start.
What we did next; or, lesson #3: Get advice from the other experts in the room!
We had a chat to the nursery teachers about our struggles – they were the ones that told us about the children’s difficulties in describing the task – and ran some new ideas by them. This was invaluable in directing our efforts for the next design.
It was clear that we needed to get children involved in the experiment earlier, rather than making them wait to watch a whole scene before getting involved. We also needed to motivate them a little bit more. There wasn’t really a *reason* to play an act out game with us so they were struggling to understand why they were doing it.
Out of this grew experiment #2 – still an imitation experiment with a puppet – where the puppet became a robot that was learning English. The robot would say sentences in a robot voice (flat, even intonation) and the child would be asked to repeat it back “like a human”. The robot would then say the sentence again, either more human-like or still like a robot, and the child would reward it if it was more human-like (with a strawberry) or give it broccoli to make it “cleverer” if it still sounded robot-like. The teachers we spoke to thought that the robot sounded engaging and that giving the children a clear job to do from the start would help.
And it did… ish.
Lesson #4: Keep it *even* simpler
Experiment #2 worked a lot better – children as young as 4;3 could complete it, they found it entertaining (the robot generated a lot of giggles) and they could describe the experiment back to their caregivers. However, for younger children who weren’t quite 4 yet, there were still too many steps involved – they couldn’t quite connect up the reward system with all the repetition of the sentences.
We were also finding a fundamental problem with the imitation task. As well as the children’s grammar, we were interested in seeing what intonation children would produce these questions with. But rather than “asking” the questions back to the puppet, the children seemed to be repeating them as if they were embedded in a silent “You say it like this…” structure, with the question-ness of the sentence stripped out. This shouldn’t have been surprising – the questions were not connected up by context and there was no real questioner-answerer relationship to support their question-ness either.
So back to the drawing board, via a chat with the teachers, we went…
Lesson #5: Context is everything
I covered this to some extent in lesson #3, but context was key for setting up our third (and final) experimental design. We kept the robot and binned the strawberry/broccoli. We introduced a second puppet, a “human” puppet, who couldn’t understand the robot puppet when he talked. The role of the child now was to be the expert interpreter. As the child could understand the robot when he talked, they had to relay his questions to the “human” puppet so the two puppets could get to know each other.
Now we had set up a questioner-answerer relationship, and this even helped provide context for the questions themselves, which is important for tag and biased questions, which are not used “out of the blue”. The child now had just *one* task to focus on and it was clear and motivated by the context.
Et voilà! Seven children, aged between 3;6 and 5;1, *all* successfully completed the task (with, again, plenty of laughter along the way).
The experiment is still a fragile one – individual children can still choose to take a “You should say…” type approach, where they’re not really re-asking the question, but this just means that we need to ensure we work with plenty of children to be able to group children by individual differences and still be able to say something about the ones who really get into the spirit of re-asking the question. To some extent this is unavoidable: the questions we’re interested in are such a natural part of normal conversation that to shine a light on them at all creates an artificiality that some people just won’t be able to get past. But others can and will, and we can still learn something from them.
What an instructive, humbling, enjoyable, frustrating, spellbinding week. Thanks to all the children that welcomed us into their spaces at Childsplay Cooperative Nursery, Hotspur Primary School, Northumbria University Nursery, and West Jesmond Primary School, and the wonderful staff at all four providers who were so generous and accommodating. We hope to see some of you again in September to run our experiment with some older children!
Oh, and lesson #6: Never forget to pack your puppet…