Gaurav Vaidya

TDWG 2011, day two, part two: Global biodiversity informatics initiatives update

My post on TDWG 2011, day two, part two, starts at 2pm with the symposium on "Global biodiversity informatics initiatives update". This symposium was hugely useful to me as a biodiversity noob, giving me a broad overview of the major #biodinfo initiatives, which problems they are tackling, and where they are headed. I don't need to repeat that these are all based on my very wonky notes; all errors here are mine, and -- when you spot them -- please let me know in the comments section. The talks in this symposium were:

• Eamonn O'Tuama talking about the Global Biodiversity Information Facility (GBIF) and its recent improvements, including:

  Switching to using DarwinCore Archives as an input format: spreadsheets are easier for people to work with than XML! It now takes a month for new data to go from upload to being published; GBIF's goal is to get that down to a week.

  GBIF has also revamped their taxonomic validation, adding other authority checklists their current taxonomic authority, the Catalogue of Life. In fact, they're now assembling a database of taxonomic checklists! I wonder if they'll show the changes in checklists over time.

  A brand new GBIF Online Resource Center has been set up to make it easier for people to find out about and upload data to GBIF.

  There are some really exciting projects which could potentially use GBIF data and TDWG standards: the Wallace Initiative, the ICLEI and the IAIA.

• William Ulate talked about the Biodiversity Heritage Library, with a presentation explaining how the BHL network looks today, with nodes spread out across the planet. Books, journals and periodicals are uploaded through BHL nodes, and are then distributed to the others (and to the Internet Archive). Prime among these were BHL Europe and BHL-Australia (a part of the Atlas of Living Australia); he particularly noted BHL-Au's fantastic user interface, which will form the basis of UI improvements to the US website. He also mentioned BHL's Egyptian collaborator, Bibliotheca Alexandrina (a project older than BHL itself!).

  William also pointed out that anybody could suggest books or journals for BHL to scan, and they would try their best to do it as fast as possible. Also, BHL's source code -- including the code that powers their website, APIs, and Citebank -- is publicly available.

  BHL is also developing CiteBank, a database of biodiversity citations. Put that together with Rod Page's iTaxon idea, and you can see some really amazing -- and amazingly useful! -- interlinks between taxonomic names and literature happening.

• The Taxonomic Name Resolution Service was mentioned. It is a really cool project to standardize plant taxonomic names. And yes, an API is available!

• Cyndy Parr presented on updates from the Encyclopedia of Life (EOL) project. She pointed out that EOL is all about aggregating and curating data; it's never going to be as smooth as Wikipedia, where all the data is in one place. However, it can have better import processes: all new names on ZooKeys/PhytoKeys are immediately added to EOL; it can also get feedback and corrections back to the original publishers much more efficiently than Wikipedia can.

  Cyndy also showed off EOL collections of taxa, across any kingdom of life and collected by any criteria, from birds at the Smithsonian's National Zoo to the colour blue. Collections can have EOL communities form around them.

  Another exciting development: EOL is going multilingual, with a little help from TranslateWiki. And they're planning collaborations with iNaturalist, one of the best websites for naturalists I know.

  EOL also plans on moving to DarwinCore Archives flat-files, as an easier way of moving data in and out of their system. They have released a the largest possible phylogenetically-organized tree, representing the state of our phylogenetic knowledge.

• Donald Hobern spoke about developments at the Atlas of Living Australia (ALAu) focusing -- much to my excitement -- on taxonomic issues.

  He began by talking about descriptive taxonomy. He mentioned IdentifyLife, a database of identification keys, including -- get this -- a plan to build a single, gigantic Key to All Life. The idea is to support citizen science "in the broadest sense". ALAu is working on a project to port the DEscriptive Language for TAxonomy (DELTA) software suite into Java. Finally, he mentioned the Taxonomy Research and Information Network (TRIN), which hosts a pretty well-organized wiki.

  Donald pointed out that ALAu's taxonomy arm had two primary communities to serve: the one interested primarily in geographical data, and the one interested primarily in taxonomic/collection data. I think he demonstrated this division by comparing the Australian National Herbarium datasets with Australia's Virtual Herbarium, but to be honest, my notes are somewhat in disarray at this point. This is because Donald is an excellent presenter, and so I wasn't focusing on my notes so much. Sorry about that!

• The symposium ended with Dave Roberts talking about ViBRANT. He spoke mainly about the exciting new changes coming in Scratchpads 2.0, which -- based on Drupal 7 -- will have a prettier user interface, snappier access times, and much better scalability. There are ambitious plans to plug in Scratchpads to biodiversity publishers (such as PenSoft) as well as biodiversity producers and consumers (such as the Encyclopedia of Life and LifeDesks). Even more interestingly, they're setting up a job-processing backend for Scratchpads on the Oxford Batch Operations Engine (OBOE).

  A sandbox Scratchpad is available, if you want to play around with the technology in its current state. Their source code is available, including Scratchpads 2.0. And, being on Git, all that code is easy to play with.

TDWG 2011, day two, part one

After a short trip to meet Walter Jetz at Yale, I'm back in Boulder, have seen Paul Simon in Broomfield, CO and Wikipedia Loves Libraries! at Norlin last week, and am making a start at putting together my research ideas. Fun times ahoy!

But first, I'm still working through my notes from TDWG 2011. Here's a first installment of my treasure trove from day two; obviously, many awesome presentations and parallel talks were missed, probably because I was too busy enjoying them to take down notes. All errors are mine, and when you spot them, please do let me know in the comments!

• Islandora (p. aye-lan-do-ra) is a slick, open-source digital asset management system based on the popular Drupal CMS. Islandora was developed by the University of Prince Edward Island's Robertson Library. Islandora's goal is to steward digital assets in the long term, with an emphasis on long term stability, and to manage the entire research life cycle. Interestingly, the developers of Islandora provide paid support for their software through a private company, DiscoveryGarden, which I can think can be a valuable way of supporting scientific software development and maintenance. The presentation and demo (which I unfortunately missed, because of parallel sessions) was given by Mark Leggott, Islandora's Lead.
• If you want to play around with Islandora, their Sandbox instance is automatically reset regularly.
• The Field Book Project at the Smithsonian aims "to create one online location for scholars and others to visit when searching for field books and other field research materials". Sounds interesting! Unfortunately, I missed their presentation at TDWG; I'm hoping the So You Think You Can Digitize blog will cover them soon.
• FishNet 2 has a pretty slick interface, allowing search by a number of common fields or by drawing polygons on a map. Search results allow taxonomic checklists to be quickly assembled and downloaded as a CSV, among other features. You can definitely see the basis for a really fantastic data exploration tool here, and - being part of VertNet - definitely opens the possibility of a really first-rate data exploration tool for all those vertebrate records. And yes, there's an API as well. A good idea: FishNet 2 saves all search queries, so that developers can analyze what users are looking for.
• Aaron Steele and John Wieczorek demoed the Darwin Core Engine, the heart of VertNet. The plan is to create an application on Apache CouchDB, a "document-oriented database that can be queried and indexed in a MapReduce fashion using JavaScript", using the magic of HTML5 as the controller to CouchDB's model. The end result: speed, maintainability, scalability. I'm definitely looking forward to more about that. And - I only just noticed they - they credited "Dennis Ritchie for C and Unix, Steve Jobs for our Macs". Aww.
• If you want to play around with this, try IRIS Couch, a CouchDB hosting service.
• Adrian Thomas Hine at the NHM Department of Entomology had an interesting talk on Species-level Metadata collection in museum; his argument in a nutshell being that even assembling species-level (rather than specimen-level data) could be valuable for research. He argued that this was also more sustainable, since maintaining this species-level index was a daily job for curators, and would answer a key question: what does a museum actually have?
• Cam Webb is working on modeling botanical data for the semantic web, based on DarwinCore.
• Filtered Push is a way to push annotations around between online repositories. My understanding of this tool is: if an analysis tool finds a specimen entry from GBIF suspicious, the dream is to allow the tool to "push" an annotation to GBIF, which GBIF can in turn "push" to the distributing museum for examination and verification by a curator. In this way, data can be improved across the interconnected databases we are now building.
• Ontobrowser, an "interactive ontology navigator and ontology-based image retrieval system", is a pretty good way of navigating Morphster, a "demonstration prototype of a service-oriented architecture enabling and supporting morphologically based phylogenetic studies". Try it out on the Hymenoptera!
• Chris Freeland talked about BHL's new Darwin Library Collection, "a digital edition and virtual reconstruction of the surviving books owned by Charles Darwin". They used detailed notes from di Gregorio and Gill, Charles Darwin's marginalia vol. 1 to map Darwin's own comments in his own hand onto books already scanned for the BHL, or - in some cases - Darwin's own annotated copies scanned directly into the BHL. Chris gave us a fantastic example of the latter, where Darwin wryly commented on Charles Lyell's thoughts on intraspecific variation that "if this were true, adios theory".
• Donald Hobern talked about annotation and ALAu. There were a lot of useful technical details about ALAu's implementation of annotations, but there are also some really interesting side projects being developed. The Online Zoological Collections of Australian Museums is a collection of animal data from museum across Australia, using the TAPIR protocol to aggregate the data from museum repositories. Identify Life, meanwhile, is a database of identification keys. Their Key to All Life project hopes to build one single gigantic identification key - for everything. How awesome is that? Give it a whirl yourself!
• Bob Morris introduced the Annotation Ontology, and stressed the importance of linking to the proof of what you're saying.

Part two will be out as soon as I can get around to it. Also: getting more and more excited for Life and Literature, just a week away tomorrow. It'll be great to see familiar faces from TDWG so soon after TDWG 2011 New Orleans!

TDWG 2011, day one

Yesterday (Monday) was the first full day at TDWG 2011. This post is an extraordinarily incomplete summary of some of the things I learned that day; a lot happened on that day, and I really don't have time to write about it all here (although I might post my thoughts in future posts). I'm writing down some URLs for my own interest. If I missed something you found interesting, it's likely that I was just too fascinated by the presentation to take notes (see my single sentence describing Michael Donoghue's keynote speech for a perfect example). All errors are mine, and when you spot them, please do let me know in the comments!

• Hank Bart announced the Tulane University Biodiversity Research Institute.
• TDWG Annual Conference 2012 will be at Beijing, People's Republic of China!
• Since 1988, TDWG has been a part of the International Union of Biological Sciences (IUBS), a 92-year old biological organization which inexplicably has scrolling text on its homepage.
• Everybody working at TDWG is a volunteer - including everybody working hard to make TDWG 2011 a success. That's crazy!
• Nozomi James Ytow is the representative for Asia on the TDWG Executive.
• Since 1935, the Index Herbariorum has maintained a list of herbaria around the world. But it does not have a Wikipedia page (as of Oct 19, 2011, anyway). Interestingly, this database has been set up in KE EMu, the popular collections management system, which is also missing a Wikipedia page.
• Audubon Core is "a set of vocabularies designed to represent metadata for biodiversity multimedia resources and collections". Its specification, interestingly enough, is based on a wiki "developing a range of new, much easier and paper-free identification tools, for use within schools and universities across Europe". Interactive/digital keys have come up before, and will come up again, in this conference.
• One good way of helping support TDWG is by collaborating with them on a grant proposal. Who better to work with on biodiversity standards?
• Michael Donoghue had an excellent keynote address to start the conference with.
• So far, 39 awards have been made under the NSF's Advancing Digitization of Biological Collections (ADBC), including InvertNet, and the Consortia of North American Lichens and Bryophytes.
• The TDWG Taxonomic Names and Concepts Interest Group had a meeting, where I found out about a bunch of taxonomic information aggregation and exchange projects around the world. Discussions included:
• A comparison of the Taxon Concept Schema (TCS) and Darwin Core (DwC). If we want to move on to ontologies, the TDWG Ontology came up.
• TDWG's own Global Names Architecture project, which has some exciting things ready already (Global Names Index) and some in the works (GNUB).
• Naim Matasci also mentioned the iPlant Taxonomic Name Resolution Service (TNRS), a pretty utility "for correcting and standardizing plant names". And the source code is available on GitHub, released under a modified BSD license.
• I know how awesome Pensoft Publishers are. But I didn't know about IPNI.
• The BGBM's Biodiversity Informatics Section (BDI) has some fascinating resources available on their website.
• Javier de la Torre showed off the Geospatial Conservation Assessment Tool (GeoCAT). As you would expect from Vizzuality, the website is fast, slick, efficient and easy to understand. And yes, it's on GitHub. Hard to tell if it's open source, though: there ain't no license.
• Denné Reed described a "federated system of data collection" he uses for paleoanthropology. And yes, he uses Darwin Core.

I'm dreading writing tomorrow's blog post, since today (Tuesday) was an incredibly valuable day for finding out about projects, websites, ambitions and niches. Luckily, I don't have to worry about that for a day at least, maybe more.

TDWG 2011, day zero

Here's some fantastic stuff I discovered during the TDWG icebreaker yesterday:

• Mandala, a cross-platform, FileMaker® Pro database "designed to track specimen-based biodiversity data, loans, and detail complex nomenclatural histories with their associated literature". A book chapter describes the schema used, but if you prefer to just jump in and try it out, there is a live database of 130,229 specimens across 2,658 (possibly invalid) taxa citing 952 pieces of literature from the Therevidae (Insecta; Diptera).
  • (Bit of a coincidence there: I co-authored a chapter in the same book! How cool is that?)
• mx, a Ruby-based platform for "gathering and working with data", as per Matt Yoder's blog post explaining his approach to writing software for science, which is a great read if you write any kind of software for any area of science. And mx is available on Github, making it perfect to play around with.
• 3I Interactive Keys and Taxonomic Databases, a "set of software tools for creating on-line identification keys, taxonomic databases, and virtual taxonomic revisions". There are plenty of sample datasets here, too.
• Species File, "both a database and a website: The database provides the detailed information about all taxa contained within the scope of the "apex taxon" and the website provides the means to interact with and modify the information contained in the database", principally developed by David Eades. There are a lot of example 'species file' websites available.

And, off of science, Ray's Site, a blog about a blind cat, and how he navigates a world without his sight.