From Filtered Push Wiki
Jump to: navigation, search

Etherpad for meeting notes: http://firuta.huh.harvard.edu:9000/FP-2014Dec16



  • Publications
    • Progress
      • Paul/James: Collection Objects
      • Bob: Refactoring Dup finding cluster analysis
      • Other papers?


  • QC work
    • Adding agent authority file to Symbiota - harvest to solr index - use in actor.
    • JSON to XLS into Kurator
    • Generate QC results for each SCAN collection.
  • State of Deployments
    • FP2.acis
      • Status of InvertEBase setup
    • FP3.acis
      • State for harvest for NEVP
  • Annotation Processor


  • Paul
    • Developed a small (10) set of test cases to exersise the ScientificNameValidator for Kurator
    • Set up a developers email list for the Symbiota developers.
    • Developed timeline with Ed for rollout of Agent functionality in Symbiota in early January.
    • Have implemented content negotiated delivery of RDF/XML or RDF as Turtle for occurrence records in Symbiota.

Example: http://symbiota4.acis.ufl.edu/scan/portal/collections/individual/index.php?occid=12771364 (ttl below is from deployed code to date, more implemented but not deployed yet).

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix foaf: <http://xmlns.com/foaf/0.1/> .
@prefix dwc: <http://rs.tdwg.org/dwc/terms/> .
@prefix dwciri: <http://rs.tdwg.org/dwc/iri/> .
@prefix dc: <http://purl.org/dc/elements/1.1/> . 
@prefix dcterms: <http://purl.org/dc/terms/> . 
@prefix dcmitype: <http://purl.org/dc/dcmitype/> . 
    a dwc:Occurrence  ; 
    dwc:institutionCode "TAMU" ; 
    dwc:collectionCode "ENTO" ; 
    dwciri:inCollection <http://biocol.org/7c429a87-93ff-406a-9c33-7a6ed387d3c9> ; 
    a dcmitype:PhysicalObject ; 
    dwc:basisOfRecord  "PreservedSpecimen" ; 
    dwc:catalogNumber "X0632931" ; 
    dwc:family "Membracidae" ; 
    dwc:scientificName "Umbonia crassicornis" ; 
    dwc:scientificNameAuthorship "(Amyot and Serville)" ; 
    dwc:genus "Umbonia" ; 
    dwc:specificEpithet "crassicornis" ; 
    dwc:eventDate  "1956-12-23" ; 
    dwc:year  "1956" ; 
    dwc:month  "12" ; 
    dwc:day  "23" ; 
    dwc:startDayOfYear "358" ; 
    dwc:lifeStage "Adult" ; 
    dwc:country "United States of America" ; 
    dwc:stateProvince "Texas" ; 
    dwc:county "Cameron" ; 
    dwc:locality "Brownsville" ; 
    dwc:decimalLatitude "25.90917" ; 
    dwc:decimalLongitude "-97.4946" ; 
    dwc:geodeticDatum "NAD27" ; 
    dwc:coordinateUncertaintyInMeters "5451" ; 
    dwc:georeferencedBy "D. Allen Dean" ; 
    dwc:georeferenceProtocol "named place/site" ; 
    dwc:georeferenceSources "BioGeomancer Workbench" ; 
    dwc:georeferenceVerificationStatus "georeferencing complete" ; 
    dcterms:modified "2013-08-22 11:16:18" ; 
    dc:rights "http://creativecommons.org/licenses/by-nc-sa/3.0/" ; 
    dcterms:rightsHolder "Texas A&M University" ; 
    dcterms:accessRights "Not-for-profit use only".
    owl:sameAs <7c429a87-93ff-406a-9c33-7a6ed387d3c9> .


Present Paul, Bob, Jim, Tianhong, Tim, David Agenda Non-Tech

  • Publications
    • Bob: oops. I owe a look at what Jim added to gdoc entry
    • Progress
      • Paul/James: Collection Objects

Paul: James is making progress, hasn't circulated draft yet.

      • Bob: Refactoring Dup finding cluster analysis

Bob: Waiting to hear from Brazilians and NCSA. Thinking about drafting a paragraph about why the approach may be more general than dup finding. Not yet at a point to report results.

      • Other papers?

Paul: Up priority in early January. Tech

  • QC work
    • Adding agent authority file to Symbiota - harvest to solr index - use in actor.

Paul: Looking at rollout into production symbiota instances in mid january.

    • JSON to XLS into Kurator

Tim: Nothing blocking. Not clear on how to write tests for this as the end product is an xls spreadsheet.

David: Some tests on generation, but nothing substantive.

    • Generate QC results for each SCAN collection.

David: Harvest about a month old. Paul: Can do a fresh harvest, then walk through each collection generating a QC report, transforming to xls and sending out to relevant contact.

Tianhong: Collection defined as?

Paul: Distinct institutionCode in the mongo collection scan_prod_occurrences.

Paul: David to act as point person for sending out the QC reports, then to collect feedback and add into NCSA tools (confluence, jira) as issues etc.

David: Have generated DMNS spreadsheet, will be sending to Paula (71k records).

  • State of Deployments
    • FP2.acis

David: Tomcat deployment from war of node is working. Working on configuring icinga for monitoring. Testing communication with Symbiota. Have updates to roll out into Symbiota, coordinating with Ed.

      • Status of InvertEBase setup

Paul: on symbiota4 in /var/www/invertebase

David: Need to deploy and configure client helper.

David: Need user account on invertebase portal for testing. PauL: Send email to Petra.

    • FP3.acis

David: Tomcat deployment from war of node is working. Some testing of ports needed.

      • State for harvest for NEVP
  • Annotation Processor

David: Planning on using the Spring data JPA to make the data access objects and then write busness logic for those for the communication that is needed between the annotation processor and the driver - letting the data access object provide an object to the annotation processor that the annotation processor can augment and send back. We should be able to write a configurable driver for Specify and Specify-HUH, with spring handling the details of CRUD.