2014Sep30

From FilteredPush
Jump to: navigation, search


Etherpad for meeting notes: http://firuta.huh.harvard.edu:9000/FP-2014Sep30

Agenda

Non-Tech

  • Publications
    • Progress
      • Paul: Collection Objects
      • Bob: Refactoring Dup finding cluster analysis
  • TDWG Symposium
    • Abstracts, Registrations.

Tech

  • Firuta server move.
    • Cleanup of artifacts in Archiva
      • Tim is looking for more cleanup here, and working build on clean machine.
      • Can we open archiva to public read only access?
    • Update deployed apps in Tomcat and Apache (Symbiota, Morphbank, Annotation Processor)
    • Distribution upgrade
  • Status of Mongo on FP2
  • QC for SCAN
    • Feedback from Neil
  • QC work
    • Adding agent authority file to Symbiota - harvest to solr index - use in actor.
  • Deployments
    • Access point updates
    • Bringing Annotation Processor up-to-date
    • Deploy and re-run harvest of occurrence records
    • Status of fp2 and fp3
  • SVN re-organization/cleanup

Reports

  • Paul
    • Some cleanup on Firuta.
    • Firuta needs a distribution upgrade. Have been taking a workstation through that.
    • TDWG abstracts in, accepted.
    • TDWG registration/travel in.


Notes

FilteredPush Team Meeting 2014 Sept 30

Present: Jim, Tianhong, David, Paul, James.

Non-Tech

  • Publications
    • Progress
      • Paul: Collection Objects

Paul: Bit further on draft, haven't circulated yet.

      • Bob: Refactoring Dup finding cluster analysis
  • TDWG Symposium
    • Abstracts, Registrations.

James: Workflows and services session, all invited abstracts accepted, session moving forward, other submissions, may add a session or put in general session.

Jim: Please review the presentations:

James: (1) Session on workflows and services, a talk given by Bertram on Kurator (2) iDigBio session on digitization - a talk given by Paul on precapture in TCNs (3) Bob talk on duplicate detection.

Paul: (4) annotation interest group working session.

Tech

  • Firuta server move.
    • Cleanup of artifacts in Archiva
      • Tim is looking for more cleanup here, and working build on clean machine.

David: Have added some artifacts that tim had pointed out as missing, need to test doing a clean build.

      • Can we open archiva to public read only access?

David: No issue doing this, will need credentials.

Paul: Artifacts from other repositories?

David: fedora fc-repo.org artifacts, no sources.

Paul: adding sources should manage the issue if the licencing is GPL or similar.

    • Update deployed apps in Tomcat and Apache (Symbiota, Morphbank, Annotation Processor)

David: Last week deployed symbiota and morphbank from current trunks, but haven't configured yet. Which data to use?

Paul: Let's use NEVP from symbiota 4 (CNH data set).

    • Distribution upgrade

Paul: Will schedule for tomorrow morning and notify.

  • Status of Mongo on FP2

Paul: Working smoothly since cleanup?

David: Haven't seen any issues.

Tianhong: no

  • QC for SCAN
    • Feedback from Neil

David: Working on scheduling a call with Paul Henrich to review this.

  • QC work
    • Heap Space

David: Tianhong reported heap space issues running akka on the full data set on a local machine.

Tianhong: One actor produces too many things for the queue - perhaps running an actor on a second machine would help.

Discussion: Rate limiting within the actors, or chunking the data set. If there is a mechanism in akka for rate limiting the flow into slow actors that would scale most robustly, for now chunk the data as a workaround.

    • Adding agent authority file to Symbiota - harvest to solr index - use in actor.

Paul: Haven't yet, need to soon.

  • Deployments
    • Access point updates

David: Finished updates to the spring/camel access point. Ready to deploy onto FP2 and FP2. Will add activemq as JMS broker for monitoring the system.

    • Bringing Annotation Processor up-to-date

David: Next step. Will start looking at driver code on maureen's machine.

    • Deploy and re-run harvest of occurrence records

David: Same state as last week - investigating broken bit.

    • Status of fp2 and fp3

David: Ready to do redeployments this week.

  • SVN re-organization/cleanup

David: Removing the Java-SOA project (tag and remove from trunk), looking at a few other probably no longer in use projects.