Since 2009, Utah has used computers to grade essays on a state student-assessment test. And testing companies use essay-evaluating software as one of two graders on graduate-school admissions exams such as the GRE. But how well, really, can a computer grade an essay?
To find out, Mark Shermis, an education researcher at the University of Akron, ran 22,029 standardized middle- and high-school essays through software from eight companies (plus one open-source algorithm). The programs, which generally track content, organization and style, generated results indistinguishable from those of humans—just much faster. With that kind of efficiency, robot graders could mean more homework for students everywhere.
140 years of Popular Science at your fingertips.
Each issue has been completely reimagined for your iPad. See our amazing new vision for magazines that goes far beyond the printed page
Stay up to date on the latest news of the future of science and technology from your iPhone or Android phone with full articles, images and offline viewing
Featuring every article from the magazine and website, plus links from around the Web. Also see our PopSci DIY feed
Engineers are racing to build robots that can take the place of rescuers. That story, plus a city that storms can't break and how having fun could lead to breakthrough science.
Also! A leech detective, the solution to America's train-crash problems, the world's fastest baby carriage, and more.


Online Content Director: Suzanne LaBarre | Email
Senior Editor: Paul Adams | Email
Associate Editor: Dan Nosowitz | Email
Contributing Writers:
Clay Dillow | Email
Rebecca Boyle | Email
Colin Lecher | Email
Emily Elert | Email
Intern:
Shaunacy Ferro | Email
Eeeewwwwwwww.... I soooo don't like that idea. Even though I taught (years ago), I still prefer to grade them myself... and am more than happy to have a computer lose its job than a fellow human being. We biological critters gotta look out for each other, ya'know?!? :-) Yes, I know there are those who would argue that consistency of grading and standardization of bla bla bla...(yeah, whatever, right?)... but give me the human factor any day; if my students want to contest a grade, let them come to me and convince me that they deserve a better score. There's more to be said for human drive and interaction than in an algorithm, in my fallable only-human opinion. ;)
fck human, long live machines. Lets free ourself from labor, irrationality and inefficiency.
@ JayArr,
You at least can't dispute using them as one of the graders in a redundant grading scheme. Cuts down time by half, and still leaves a real person in place, in case there is a one-in-a-million discrepancy between the program's grade and the human's grade.
And seeing as how money going towards grading tests could then be used instead to further prepare kids for the test, I don't see how a teacher of any form can be against it. As far as "humans" first goes, I'd rather assert "students first". More efficient teaching, more money for kids, and more consistent standardized test results. I see the kids coming out ahead on this.
I don't see how they can't come out better on this (because if the grading was questionable, they wouldn't use it). So if that is the case, why would anyone be against it, unless driven by other motives? And who can defend motives other than students first? They're the future in our public schools, and our customers in the private ones. Moral pecking order is pretty clear.
Well, I hope that the robo graders do a better job than whatever method popsci uses to assign hyperlinks -- not actually too relevant.
Would like to see a lot more detail about this. Too bad there's no link to a source.
I want our US government to come up with a simple major, standardized in a computerized fashion, so that anyone can go to college online for free. Perhaps it may only be of Majoring in English and Mathematics. But once a person completes this Major, imagine how it would help them the rest of life, not to mention our current college system needs an incentive to lower their prices.
Also they same thing could be offered, who did not graduate from high school as well or simply wish to brush up on their skills.
Education should be free and available and yes accredited to be respected.
How can a computer grade imagination. This is only for sentence structure and grammar. Where does imagination kick in. This will not be ok until we have AI.
What variety of idiots are grading papers these days?
A robot with no imagination, comprehension, sense of humor or wisdom of any kind cannot possibly grade an essay.
Unless that essay was multiple choice or something.
The "content" of an essay is more than just the sum of the characters on the pages. A robot would see nothing BUT characters on a page.
Or are the essays so dumb they are easily graded by mindless automatons?
Can you imagine the absurdity once teachers figure out the algarithm and teach students to "write to the test" without bothing to make the writing useful to people.
topic-artical-vocab-verb-vocab-prepasition-topic-punctuation. topic-vocab-verb-prepasition-vocab-conjuntion-vocab-prepasition-vocab-punctuation.
transition-topic-vocab-verb-vocab-punctuation.
etc.
If it scores well, and teachers are graded by scores, that IS what they will teach (since it is what they get paid for).
"Or are the essays so dumb they are easily graded by mindless automatons?"
When they are writen by mindless automatons with no real life experiences (because they were raised on internet and TV, not real world interactions), no imagination (TV and internet), no attention span (TV and internet), and poor modeling (no books + TV and internet) - then they are easily graded by the same.
Of course, if the robot can grade the assignment of the student, why involve the student at all? Replace the person's role in life with the robot and have the child do something robots are not as good at - like cleaning bathrooms or gutting fish.
This would be great for me (teacher in Germany) but where can you get it - I googled essay machine grading open source - and got nothing useful!!
Wow robots programed with acceptable error percentages, fascinating. Add the new AI interface links and it becomes a war machine with acceptable losses.
robot teachers maybe a good thing but what if it malfunctions it could go crazy and blow up.