On test day for my Behavioral Ecology class at UCLA, I walked into the classroom bearing an impossibly difficult exam. Rather than being neatly arranged in alternate rows with pen or pencil in hand, my students sat in one tight group, with notes and books and laptops open and available. They were poised to share each other's thoughts and to copy the best answers. As I distributed the tests, the students began to talk and write. All of this would normally be called cheating. But it was completely OK by me.
Who in their right mind would condone and encourage cheating among UCLA juniors and seniors? Perhaps someone with the idea that concepts in animal behavior can be taught by making their students live those concepts.
Animals and their behavior have been my passions since my Kentucky boyhood, and I strive to nurture this love for nature in my students. Who isn't amazed and entertained by videos of crafty animals, like Betty the tool-making crow, bending wires into hooks to retrieve baskets containing delicious mealworms? (And then hiding her rewards from a lummox of a mate who never works, but is all too good at purloining the hard-won rewards of others?)
Nevertheless, I'm a realist. Almost none of my students will go on to be "me"—a university professor who makes a living observing animals. The vast majority take my classes as a prelude to medical, dental, pharmacy, or veterinary school. Still, I want my students to walk away with something more than, "Animals are cool." I want them to leave my class thinking like behavioral ecologists.
Much of evolution and natural selection can be summarized in three short words: "Life is games." In any game, the object is to win—be that defined as leaving the most genes in the next generation, getting the best grade on a midterm, or successfully inculcating critical thinking into your students. An entire field of study, Game Theory, is devoted to mathematically describing the games that nature plays. Games can determine why ant colonies do what they do, how viruses evolve to exploit hosts, or how human societies organize and function.
So last quarter I had an intriguing thought while preparing my Game Theory lectures. Tests are really just measures of how the Education Game is proceeding. Professors test to measure their success at teaching, and students take tests in order to get a good grade. Might these goals be maximized simultaneously? What if I let the students write their own rules for the test-taking game? Allow them to do everything we would normally call cheating?
A week before the test, I told my class that the Game Theory exam would be insanely hard—far harder than any that had established my rep as a hard prof. But as recompense, for this one time only, students could cheat. They could bring and use anything or anyone they liked, including animal behavior experts. (Richard Dawkins in town? Bring him!) They could surf the Web. They could talk to each other or call friends who'd taken the course before. They could offer me bribes. (I wouldn't take them, but neither would I report it to the dean.) Only violations of state or federal criminal law such as kidnapping my dog, blackmail, or threats of violence were out of bounds.
Gasps filled the room. The students sputtered. They fretted. This must be a joke. I couldn't possibly mean it. What, they asked, is the catch?
"None," I replied. "You are UCLA students. The brightest of the bright. Let's see what you can accomplish when you have no restrictions and the only thing that matters is getting the best answer possible."
Once the shock wore off, they got sophisticated. In discussion section, they speculated, organized, and plotted. What would be the test's payoff matrix? Would cooperation be rewarded or counter-productive? Would a large group work better, or smaller subgroups with specified tasks? What about "scroungers" who didn't study but were planning to parasitize everyone else's hard work? How much reciprocity would be demanded in order to share benefits? Was the test going to play out like a dog-eat-dog Hunger Games? In short, the students spent the entire week living Game Theory. It transformed a class where many did not even speak to each other into a coherent whole focused on a single task—beating their crazy professor's nefarious scheme.
On the day of the hour-long test they faced a single question: "If evolution through natural selection is a game, what are the players, teams, rules, objectives, and outcomes?" One student immediately ran to the chalkboard, and she began to organize the outputs for each question section. The class divided tasks. They debated. They worked on hypotheses. Weak ones were rejected, promising ones were developed. Supportive evidence was added. A schedule was established for writing the consensus answers. (I remained in the room, hoping someone would ask me for my answers, because I had several enigmatic clues to divulge. But nobody thought that far afield!) As the test progressed, the majority (whom I shall call the "Mob") decided to share one set of answers. Individuals within the Mob took turns writing paragraphs, and they all signed an author sheet to share the common grade. Three out of the 27 students opted out (I'll call them the "Lone Wolves"). Although the Wolves listened and contributed to discussions, they preferred their individual variants over the Mob's joint answer.
In the end, the students learned what social insects like ants and termites have known for hundreds of millions of years. To win at some games, cooperation is better than competition. Unity that arises through a diversity of opinion is stronger than any solitary competitor.
But did the students themselves realize this? To see, I presented the class with one last evil wrinkle two days later, after the test was graded but not yet returned. They had a choice, I said. Option A: They could get the test back and have it count toward their final grade. Option B: I would—sight unseen—shred the entire test. Poof, the grade would disappear as if it had never happened. But Option B meant they would never see their results; they would never know if their answers were correct.
"Oh, my, can we think about this for a couple of days?" they begged. No, I answered. More heated discussion followed. It was soon apparent that everyone had felt good about the process and their overall answers. The students unanimously chose to keep the test. Once again, the unity that arose through a diversity of opinion was right. The shared grade for the Mob was 20 percent higher than the averages on my previous, more normal, midterms. Among the Lone Wolves, one scored higher than the Mob, one about the same, and one scored lower.
Is the take-home message, then, that cheating is good? Well … no. Although by conventional test-taking rules, the students were cheating, they actually weren't in this case. Instead, they were changing their goal in the Education Game from "Get a higher grade than my classmates" to "Get to the best answer." This also required them to make new rules for test-taking. Obviously, when you make the rules there is no reason to cheat. Furthermore, being the rule-makers let students behave in a way that makes us a quintessentially unique species. We recognize when we are in a game, and more so than just playing along, we always try to bend the rules to our advantage.
Morally, of course, games can be tricky. Theory predicts that outcomes are often not to the betterment of the group or society. Nevertheless, this case had an interesting result. When the students got carte blanche to set the rules, altruism and cooperation won the day. How unlike a "normal" test where all students are solitary competitors, and teachers guard against any cheating! What my class showed was a very "human" trait: the ability to align what is "good for me" with what is "good for all" within the evolutionary games of our choosing.
In the end, the students achieved their goal: They earned an excellent grade. I also achieved my goal: I got them to spend a week thinking like behavioral ecologists. As a group they learned Game Theory better than any of my previous classes. In educational lingo, "flipping the classroom" means students are expected to prepare to come to class not for a lecture, but for a question-and-answer discussion. What I did was "flip the test." Students were given all the intellectual tools beforehand and then, for an hour, they had to use them to generate well-reasoned answers to difficult questions.
The best tests will not only find out what students know but also stimulate thinking in novel ways. This is much more than regurgitating memorized facts. The test itself becomes a learning experience—where the very act of taking it leads to a deeper understanding of the subject.
This article originally appeared on Zócalo Public Square and was republished with permission. Peter Nonacs is a professor in the Ecology and Evolutionary Biology Department at UCLA.
At work the dedicated goal must be met. My boss doesn't ask me how I did it, he just want the goal to be accomplished. In the real world, just no stealing or breaking laws is allowed, other wise use any recourse you can find and just get the job done.
My bosses does not try to trip me up towards accomplishing his goal with bizarre questions and miss information, they only do this in schools. I always thought that to be strange, how a test and its questions could lead you away from the correct answer; in life, my wants boss me to succeed so communication is encourage for clarity.
The schools or teachers would say they are helping you with all the miss information, odd questions, challenging me.
All I know, a person who would pay a person to accomplish a particular goal for them, would have no desire to add cost to accomplishing the project with distracting information.
I also thought many school test was a farce to the reality of life, but a means to help justify the importance of the teacher. I come to school to learn, not to be tricked by the teacher illustrating they are smarter than me. Hello! I am the student and I am trying to learn to accomplish a goal here!
This article was GREAT! As a long time student of game theory and strategy, this was an OUTSTANDING example of both disciplines in action. I hope the students got as much out of the exam - and the cheating - as I did reading the article!
The article gave me tons of really really good ideas about game theory and strategy. The idea of being able to cheat was outstanding - in fact it caused everyone to think strategically and to work as a team (at least for the mob). The added spice of offering to return the test or shred it was brilliant! For me, I definitely would have wanted my test back!
As an employer for many years, it's great to see the pursuit of the best answer in a class. For me, that's the best part of the article and what I hope is replicated in many other classrooms around the world.
The only open question for me is why this prof isn't in the math department, or teaching game theory full time!
Excellent article, and excellent execution of the strategy on the exam!!
Discussing and sharing ideas is all part of the learning process.
I don't think the concept of an open notes/book/laptop/etc test is all that novel. As an engineer, there were few tests (once you hit upper levels) that didn't allow for a notes sheet at a minimum. Some tests even permitted the text books and laptops, depending on the material. I think this what animie was getting at: education is preparation for life. I work as an engineer now, and I certainly am allowed (I think "encouraged" is probably the better word) to use outside resources to get the best answers. I'll admit I never took a "group test", but I've definitely heard of them being administered before.
Nevertheless, this is probably one of the best PopSci articles I've seen recently, because the way that the professor turned his exam into a lesson relevant to the class is absolutely brilliant. The only thing I disagree with is his insistance on saying he allowed the students to "cheat". Cheating means that rules were broken. As the professor, it is up to him to determine what the rules are. So, when he decided to administer an exam where the rule was "there are no rules" (aside from prohibiting illegal behavior), he effectively made "cheating" impossible. It was a brilliant idea, a great example of effective education, and really fun to read about, but there was no cheating.
As a final thought on game theory and test-taking, I would be extremely curious to see the results of an exam where the professor gives a set of literal rules that will be enforced, but encourages students to try things to see what they can get away with (ie, what is not prohibited explicitly is permitted implicitly).
Nisi credideritis, non intelligetis.
Life does not consist mainly, or even largely, of facts and happenings. It consists mainly of the storm of thoughts that are forever blowing through one’s mind.
This article reminded me of another I read a while back in which the author was trying to explain why South Korea and Finland were topping the world OECD rankings, especially in the maths and sciences. The author mentioned how the philosophy towards education differed in these countries - there being a greater emphasis on promoting equity and cooperation through education - and how it was benefitting the students. The success of these UCLA students seems to support that explanation, and perhaps North American students could benefit if there was a greater emphasis on cooperation in schools as opposed to competition.
One of the best articles I've read in a long time. I believe that critical information should not be memorized word for word, but instead memorized where to find the information. That way it's impossible to make mistakes and if the data is updated, you have the information at hand. In my opinion, that is the best way to learn.
Here is my answer to your exam question:
Natural selection can be regarded as a game. However, we are not the players, but merely the pieces on the board.
The game is solitaire, and nature is the only player.
There are no teams, and the only rules are the laws of physics.
The only objective is the continuation of the game, and the outcome on a personal level is that we are all sacrificial pawns.
I wish people were as cool about cheating as this guy. Also, finally someone realizes the importance of gaming huh? And the results of Game Theory were also impressive. Smart man.
I think in the article, though, he marginalized the importance of scroungers. They are an important part of any continued collaboration as they are the traders of ideas they did not make themselves. He takes an idea from party A (as a debt), but to get an idea from party B he trades it for the idea from party A, whom he can now reimburse. All are wealthier than had the lazy scrounger "played fair" and done his own work.
I applaud this professor's effort to teach his students by experience. I, too, am constantly searching for new ways to engage students and to inspire them to actively participate in their education. But let's be clear--the only REAL goal of this experiment was to produce the best answers to a test. Everything else is drawn from inference and is not necessarily supported by the results.
The students as a GROUP learned game theory better than previous classes. What does this mean for the individual? Not every member has to contribute in the intellectual process in order to still be useful to the group. The issue of "parasites" aside, a member could conceivably act as a "secretary" of the group, or a go-between to facilitate communication between sub-groups, never engaged in the intellectual process itself, but still contributing to the overall efficiency of the group. In fact, this is how societies work on both a macro and micro level. Leaders emerge to direct a nation; CEO's manage companies; and beneath all the hegemony and innovation is a team of "workers" whose value to the system is immense, but who are directly engaged with little, if any, conceptual or innovative activity.
The above experiment replicates this scenario, and the three "lone wolves" provide evidence in support of this. Having prepared for the exam on their own, two were able to score just as high or higher than the group, and one scored lower. The two who scored on par with the group undermine the value of the group's success to the individual. Meanwhile, the one who scored lower raises the question whether every member of the group would still be able to score high, even after benefitting from the group dynamic. Things get more complicated when we consider that the "lone wolves" could see and hear the actives of the group, and so benefit from them to some degree.
There is also evidence in the fact that the students hesitated to see the results of their test. The hesitation comes from an internal deliberation over whether or not to trust the group--that is, to leave one's fate in the hands of others. Some will be eager to do so, knowing full well that they could not score as high on their own. Others will pause, wondering to themselves whether their scores were brought down, either because they felt some of their fellow group members were less prepared or simply because they still don't agree with some of the answers on the test. Each of these scenarios points to the possibility that the students did not absorb the material on an individual basis.
The real test of this experiment would be if the instructor issued another test similarly designed to apply their knowledge of game theory, and see if indeed scores improve without the support of a group.
Of course, this is all interpretation drawn from inference, but so is anything else that we say about this experiment aside from one fact alone: the group produced a higher than average score on a test. The issues I've raised are important, in my opinion, because they raise questions about the teaching philosophy of the university, especially one that declares itself to be an elite school with a highly selective enrollment process. Are these institutions designed to train workers or leaders? Do we want students to innovate or to be wholly subsumed in a system? The reality is that most university student will not find themselves in positions of leadership or as innovators, but should that influence the university's teaching philosophy? My answer is "No." And so an experiment such as the one above, while certainly exciting, requires a bit more scrutiny to see if it fits the objectives of university education. Indeed, it's precisely because of how exciting it is that I believe it deserves close examination.
As much as I'd like on the 'education is broken and isn't anything like the real world' bandwagon with you...
While it is probably true that a boss or client wouldn't intentionally hinder your progress, your argument fails with the assumption the motive is significant. Just because someone isn't trying to trick you or make things difficult, doesn't mean that they won't change their mind or have a failure in communication. Being adaptable through adversity is an incredibly valuable skill that would not be learned if educators were holding your hand and not introducing new challenges.
I think 'he marginalized the importance of scroungers' because only 3 people weren't with the group. When almost everyone is already openly working together a scrounger doesn't have value. That said, being a scrounger earned me my engineering degree.