What do you see here?

A giraffe, right? Isn't it incredible how fast you went from coloured pixels on the screen to the full concept of a long-necked mammal living in Africa? Our brain is extremely powerful, right?

And now looking at this image...

...still a giraffe, but, well, not the same, eh? For instance, we find it harder to say that it is an animal, to recall that it eats leaves from trees. Doesn't it make you want to know more about how our brain process images and categories?

So here we are...

What do we need? Lots of images!

Understanding biological intelligence and making progress towards artificial one go hand in hand. To study how our brain process visual information, and to train algorithms to do the same, one key ingredient is a good set of images. Traditionally, experimental psychology and cognitive neuroscience have relied on highly controlled, carefully selected, dataset of stimuli, which have critical limitations. First, the stimuli lack ecological validity: you don't usually see a hammer isolated, on a white background, but rather immerse in natural scenes (e.g., on a carpenter table). Second, the size of the datasets is typically restricted to a few dozen stimuli or, in the case of larger databases, the labels are verified only for few of them. This kind of datasets cannot be used to develop tasks for machine learning as hundreds of properly labeled images are needed to train an artificial agent.

How do we need them? Free & labeled!

To address these issues, we seek to create a large multimodal set of ecologically valid stimuli and to provide curated human labels. Moreover, we want to ensure they are free from copyright issues so that anyone in the world can download them and used them for scientific purposes. Your help is critical in ensuring that the correct label (and classification) is associated to each image. Please use English words, but remember: there is no right or wrong answer: just tell us what you see! If you find it hard to isolate the main item of a given subject, feel free to skip it. Or to try your best and name the most prominent one according to you.

How can you help? Naming & Classifying!

You will be asked to name the main item in each image. Please write one single concept in English, but remember, there is no right answer! We will also give you the option to add another answer, should there be another appropriate name, a synonym. For instance, it could be impossible to tell an alligator from a crocodile, or a camel from a dromedary. Again, no wrong answers - and you can skip if there is no alternative name that comes to mind. Finally, for each image, we will ask you to classify it as belonging to a few categories. For instance, is a guitar a musical instrument, a tool, or a mean of transportation? Sometimes this might be tricky: is it a fruit a living or nonliving thing? A wool scarf is made of natural materials, but we'd likely say it is an artificial, man made object. Again, no wrong answers, just let us know how you would assign it to a category or the other. If/when there are multiple correct options, pick the most representative for that image.

Who will benefit from this? Everyone!

Our dataset will be unprecedented (in size and content) and we expect it will be vastly adopted by the international scientific community leading to broad impact. Its applications will range from the study of cognitive phenomena with neuroimaging techniques, to the exploration of differences and similarities between computations and representations in biological and artificial networks. Oh, yes: you will be contributing to major scientific discoveries with something as simple as labeling an image!

Will we always agree? How will the data be analyzed?

Some images might be harder than others to name and/or categorize, for a variety of reasons: maybe there are too many items in the picture and it is impossible to chose "the main one", or maybe the name of the main item is just very difficult to recall, like for a very rare animal. This is exactly why we need your help! Based on your responses, researchers might want to eliminate from future experiment the difficult images, or add the variable disagrement to their analyses. Feel free to point out to particularly hard images in the dedicated talk board and to check out our analysis plan and preliminary results in this other one.

Can we see an overview of the classification scheme?

Sure! Here is a representation of the series of choices you'll be asked to make:

Important notes

Generally speaking,

• there is no need for details, e.g., instead of "striped red and yellow socks", just "socks" works!
• plural or singular won't matter, e.g., either "sock" or "socks" will work just the same
• UK or US English spelling and choice of words are both fine
• some images might be best described by two words (or compound words), that is why we specify that it should be a single concept, but not necessarily a single word.