New App Uses Smartphone Sensors to Automatically Tag Photos With Names, Places and Emotions

By Rebecca Boyle Posted 07.01.2011 at 3:58 pm 9 Comments

Faces in iPhoto sekimura via Flickr

A new app can automatically tag your smartphone photos with a wide range of attributes, picking out not only the people but the context of the picture, including emotions, weather conditions and type of activity.

Dubbed TagSense, the new app was developed by students from Duke University and the University of South Carolina who combined smartphones’ many sensors into one all-encompassing tag suite. The technique goes way beyond GPS technology to recreate a photo’s location and context.

A phone’s built-in accelerometer could tell whether a person is standing still, dancing or engaged in some other activity, according to a Duke news release. Light sensors in the phone, normally used to dim or brighten a display screen, can be used to tell whether the picture is inside our outside; weather conditions can be checked against the phone’s location; and it can even use a microphone to tell whether the subject of the photo is laughing or quiet.

All these attributes are assigned to each picture, and a user can search according to various categories, the news release says.

“So, for example, if you've taken a bunch of photographs at a party, it would be easy at a later date to search for just photographs of happy people dancing,” said Chuan Qin, a visiting graduate student from the University of South Carolina.

The students tested the system using eight Google Nexus One mobile phones, snapping more than 200 photos at various spots on the Duke campus, and found it was more sophisticated than Google’s Picasa or Apple’s iPhoto tagging systems, according to Romit Roy Choudhury, assistant professor of electrical and computer engineering at Duke.

The team unveiled the app at the ninth Association for Computing Machinery's International Conference on Mobile Systems, Applications and Services (MobiSys), being held in Washington, D.C.

Previous Article: Human Belly Button Is Home to Hundreds of Never-Before-Seen Species

Next Article: $2.7 Billion Later, the Army’s Intelligence-Sharing Computer System Still Doesn’t Work

9 Comments

Link to this comment

minisquid

07/01/11 at 8:11 pm

The one night stand is gonna die when this goes mainstream...

Link to this comment

lohengrim

07/02/11 at 10:14 am

eden of the east?

anyway, spammer at popsci? gasp! at the lack of internet-ID we are doomed...

it's not the mod's fault they're not blocking it's IP-address or upgrading their captcha to make it harder for image-recognition software to recognize..

it must definitely be our fault for not jumping in on the bandwagon and that this current trend will serve us a gentle nudge for us to reconsider our opinions..

or not..

Link to this comment

lohengrim

07/02/11 at 10:10 am

oh yea.. as a side note.. they seemed to have removed the captcha for posting comments on popsci.. is that normal?

hmm.. intriguing..

Link to this comment

lohengrim

07/02/11 at 10:12 am

and so exploitable.. lol..

Link to this comment

lohengrim

07/02/11 at 10:25 am

what the.. my first post was 10:14am.
second post was 10:10am..
and the third post was 10:12am..

i just traveled 4 minutes backwards in time..

hahaha... awesome..

Link to this comment

cholin3947

07/02/11 at 11:44 am

lohengrim,

Captcha does not work as well as people think it does. There are spammer bots programs that can defeat it most of the time, and when that does not work there are always Chinese cyber sweet shops.

Also I still have to input the captcha code, so maybe you are special.

Link to this comment

lohengrim

07/02/11 at 1:45 pm

weird. i posted comments over several topics hours ago and the captcha was missing. and yea, captcha isn't perfect, as discussed here: http://caca.zoy.org/wiki/PWNtcha

although i don't see that as an excuse not to continue further developing it.. this guy has some nifty suggestions: http://www.mperfect.net/aiCaptcha/

i also have several suggestions but i'd only end up digressing more than necessary..

but the sudden appearance of the spammer and the captcha suddenly missing awhile ago made me think that something fishy was going on..

or maybe popsci were just doing maintenance..

paranoia.. lulz..

Link to this comment

gizmowiz

07/02/11 at 4:05 pm

Until they make an app or a web site to ORGANIZE your photos into multiple level nested albums what good is having digital pictures at all? It's just a mess of photos that no one can find and NOT A SINGLE WEB SITE that's still in business has any organizational tools worth two cents. Phanfare had the best but stubbornly refuses to allow users to embed the site you pay for inside your own web site and refuses to allow frames or iframes and to allow organizing by anything other than dates. Myphotoalbums had the best organizational tools then folded up in bankruptcy. It also allowed frames and embedding.

Even the famed google picasa web albums gives you no way to organize your albums in any kind of hieracrchy so good luck keeping track of more than about 1000 pictures online. It's just not possible without resorting to Jalbums and running your own server from home.

Stupid photography online it's as dumb as they can get by refusing to allow organizational tools that EVERYONE has on their own PC's.

What kind of cloud system is that!

What's amazing is that Google Picasa is a darn good organizational app but just try that online!

Popular Tags

Technology

All Tags All Photos All Videos

Technology

+ More Photo Galleries

June 2013: American Energy Independence

Five amazing, clean technologies that will set us free, in this month's energy-focused issue. Also: how to build a better bomb detector, the robotic toys that are raising your children, a human catapult, the world's smallest arcade, and much more.