Finished! Looks like this project is out of data at the moment!
We've updated our Results page! Click on About and take a look at how you've helped Phish Finders learn more about cybercrime!
Results
When Phish Finders launched in January 2021, we had no idea how successful our project would be. 1,429 registered volunteers classified our 1,895 images in a little under a week. We were incredibly surprised by how quickly volunteers like you completed this task. For that, we thank you for all of your hard work! Since then, we’ve been analyzing the data and we’re here today to give you an update on what we’ve been doing and where our research is now.
Our first job was to compare your classifications with certain classifications made by experts. We did this by using a set of 34 images classified by experts (our gold standard images) inserted into the data set. What we found with this analysis was that your classifications on those images were not statistically significantly different from the classifications made by the experts. This means that we can use the insights you shared on all of the other images with a pretty good level of confidence. We called this our proof of concept and presented a paper discussing these findings, “Phish Finders: Improving cybersecurity training tools using citizen science”, at the 2022 International Conference on Information Systems. Looking forward, our team is exploring how different data labeling strategies and the amount of noise those methods create can impact volunteer performance and data accuracy.
We’ve also taken a lot of time looking at all of your comments in “Other Phishy Findings”. In fact, our paper, “Phish Finders: Crowd-powered RE (requirements engineering) for anti-phishing training tools”, not only discusses how your comments have led us to identify new types of phishing cues in general web design, aesthetics, and content but it also talks about how citizen science can help build better anti-phishing tools by including those cues you identified here on Zooniverse. This paper was just the tip of the iceberg, and we’re also working on another paper that describes the specific cues you mentioned in your comments and how those cues may affect social computing systems.
So, the future is bright for Phish Finders! As we discover new ways to curate data for labeling, keep your eye out for the project to return with fresh images or new questions for you to help us answer.