The following projects are available for GSOC 2018 students.
Related repo: https://github.com/WildbookOrg/Wildbook
Mentor bios: http://www.wildme.org/contact/. All mentors are full-time Wild Me staff.
Summary: Use Java programming to listen for reply comments on whale shark-related YouTube videos, and run those comments through Google Cloud Vision, Google Machine Translation, and Natural Language Processing to help qualify whale shark sightings with relevant information about when and where the animal was sighted. This is a great opportunity to get exposure to multiple forms of A.I. (computer vision, machine translation, OCR, NLP, etc.) while developing unique software that aids wildlife researchers in the field.
Difficulty: Moderate (High concept, intricate work but low volume)
Description: In June 2017, we deployed a novel use of our existing computer vision and basic artificial intelligence tools (see wildbook.org) to pro-actively data mine YouTube for whale shark sightings and replace human labor with automation and an A.I. that interacts with YouTube posters. The software performs the following tasks:
Expected outcomes: We need your help with this step above: “Automatically post questions (where? when?) to posters if metadata is missing.” Currently, we have no automated way of listening to those comments and must manual review them for date and location information. We need you to create a listening service that monitors identified videos for comment replies and feeds those into our natural language processing pipeline automatically, allowing needed information to be automatically fed into the whale shark research community and then thanking OPs for their follow up to our questions!
Figure 1. Individual whale sharks can be identified from tourist videos
Skills required/preferred: Java programming, Google Machine Translation and YouTube API experience.
Possible mentors: Jason Holmberg (founder) and/or Mark Fisher, PhD. We will orient you to the existing APIs and help you build your listening service.
Summary: Some of our Wildbooks are used to study multiple species as they co-occur in nature. For each species, researchers may record different types of measurements and different genetic values. Help us better support researchers by converting Wildbook from key-value pair configuration to a hierarchical format with YAML that allows for configuration by species.
Possible mentors: Jason Holmberg and/or Drew Blount (Flukebook.org lead developer).
Difficulty: Easy. Best done with Project 2 above (YAML configuration).
Description: The behavior and configuration options for Wildbook are currently defined in properties files, which are generally not understood by nor well described for biologists seeking to use Wildbook. Use your programming skills to develop a slick, question-based interface that describes needed configuration choices to biologists first starting Wildbook and guide them through the process of making good choices before saving their results to the configuration resources for persistence.
Expected outcomes: When Wildbook first starts, your configuration wizard quickly and easily helps biologists configure Wildbook for their wildlife species and research techniques of interest.
Possible mentors: Jason Holmberg, Colin Kingen
Difficulty: Moderate (High concept, moderate coding complexity)
Possible mentors: Jon Van Oast
Summary: Wrestle D3.js to the ground and make it render animal co-occurrence diagrams for search results in Wildbook.
Difficulty: Hard (but awesome!)
Expected outcomes: When an Individual Search is executed in Wildbook, a D3.js force-directed graph of co-occurrence relationships is visible as a results option in Wildbook.
Possible mentors: Jason Holmberg
Summary: Apply your Java skills, learn the Flickr API, and blend multiple forms of A.I. to create an intelligent agent that automatically helps a community of whale watching naturalists in Flickr rapidly identify the humpback whales they photograph and share online.
Difficulty: Hard (but seriously…A.I. experience is a great thing for a resume), integrating computer vision calls (existing APIs), NLP, neural machine translation, OCR, etc. Google Cloud Vision and Machine Translation APIs are used for processing visual and textual data. We have built similar agents for YouTube and Twitter, so we know we can help you succeed.
Description: The Flukematcher Flickr community is a group of humpback whale watching enthusiasts manually comparing photos pg humpback flukes, which are individually identifiable by their white and black contrast and by the unique, soundwave-like signature of their trailing edges. We already have the computer vision technology to match these flukes in images. We need you to apply your programming skills to build an intelligent agent that listens for new photo posts to the Flukematcher community and to run those posts automatically through our computer vision and NLP pipelines, quickly responding back to this community with the answer to “Which whale does this fluke belong to?” Social media APIs can be squirrely, so you'll need to build original code that connects Flickr to Wildbook, but the result will be a truly amazing and interactive blend of humans and A.I. studying whales!
Expected outcomes: As a result or your amazing coding skills, the Flukematcher FLickr community has an intelligent agent suggesting the identities of the humpback whales sighted by a community of whale watching naturalists and guides.
Possible mentors: Jon Van Oast and Jason Holmberg, who have succeeded in deploying a similar agent for YouTube videos.
Summary: Review i18n static code analysis results and help us find and fix embedded strings, locale-sensitive methods, and other i18n issues to improve the usability of Wildbook in multiple languages.
Difficulty: Easy (or difficult if you want bonus points for helping us support right-to-left languages, such as Arabic)
Expected outcomes: Your awesome efforts have fixed i18n bugs in Wildbook, giving biologist users in non-English languages a great, localized experience with the software.
Possible mentors: Jason Holmberg
Description: The hierarchy of objects in Wildbook is long, and can be challenging to navigate and visualize in complex projects. We will implement a “breadcrumbs” style UI at the top of main data pages to improve this. At its greatest depth, a user should be able to see the chain of relationship from a individual animal to an “Encounter” with an animal at a certain time and place, to an “Occurrence” or group sighting of animals, all the way back to the survey where the data was recorded. A the top level, a user should be able to see from the Survey all the associated data points, i.e. this survey has 14 Occurrences, 39 Encounters, 11 Identified individuals and provide navigation options. This simple UI will reside at the top of high usage pages and improve workflow for developers and researchers across multiple Wildbook implementations.
Skills required/preferred: Java, CSS, HTML. Nice to have: SQL (Postgres), Datanucleus.
Possible mentors: Colin Kingen, Wildbook Engineer.