From Filtered Push Wiki
Jump to: navigation, search

Etherpad for meeting notes: http://firuta.huh.harvard.edu:9000/FP-2012Aug29


  • iDigBio and SCAN integration
    • Test VM for SCAN Symbiota-FP integration. See: FP-Lite_deployment_for_SCAN
    • General overview of plans for rollout to production for SCAN.
    • VMs for Annotation System (FP-Lite) and FP Node development.
    • Annotation system integration with Morphbank and iDigBio portal.
    • Number and scaling of Annotation Stores.
    • Planned FilteredPush NE Vascular Plant TCN support and differences in security model from SCAN.


  • Annotation Paper
  • Collaborations
    • Kepler
    • Apiary
    • Specify/Symbiota
    • SCAN TCN
    • NEVP TCN


  • Paul
    • Working on ABI proposals.
    • Tested Lei's fixes to 6K limit in Kuration, adding timeout seems to fix the issue.


Filtered Push team meeting 2012 Aug 29 Present: Bob, Maureen, Paul, David, Jim, Ed, Gil, Alex, Andrea. Agenda:

Paul: Overview of FP-Lite, FP-Medium (SCAN), FP-Heavy (ApplePie, NEVP).

Ed: Test VM requested and in the works.

Paul: Test bed for integration prior to deploument.

    • General overview of plans for rollout to production for SCAN.

Paul: Graduated rollout over the fall. Into test first, then into production:

Bob: Do the arthropod systematists have different practices that will affect our deployment. I'd be surprised if we wouldn't be able to.

Jim: Ed has worked with this group and can provide more insight. It is a very distributed network of systematists, we should be able to handle that.

Ed: Arthropod collections environment is somewhat new, they are trying to organize their workflows - they aren't as organized in the informatics as the plant folks in the region, lots of learning each others needs.

Gil: Workflows for entomologists probably not significantly different than for botanists in the areas where FP might be involved.

Gil: Will this implementation have annotations redirected back to Specify instances.

Paul: Yes, but one specify instance in SCAN.

Bob: Workflows-two use cases perhaps? (1) locate an expert who can provide an identification, (2) once such an expert is using the software, to provide them with resources to make the identifications useful.

Ed: Yes. All of this sounds like usecases to support SCAN. Not quite clear yet of how they are going to provision the images. Workflow and management of this around the images (including image storage) needs further definition.

Ed: Symbiota VM setup within next couple of weeks. Would like to do setup FP bits.

    • VMs for Annotation System (FP-Lite) and FP Node development.

Alex: Credentials for both machines set up, Maureen has indicated credentials changed for all accounts, ready to store.

Maureen to start on FP deployment onto larger.

Next week David to start on FP-Lite deployment onto smaller.

    • Annotation system integration with Morphbank and iDigBio portal.

Gil version of portal to implement:

Alex: Had annotations in list for next version of portal, now might fit well with public release in June. Can start looking around the first of the year.

Alex: How many VMs?

Paul: Three: (1) FP-Lite then FP-Medium for SCAN as test bed for SCAN. (2) FP-Medium, FP-Heavy for upstream filtered push development and testing. (3) FP-Lite annotation system for testing.

Bob: VM Platform?

Alex:Citrix Xen server, using puppet for management.

    • Number and scaling of Annotation Stores.

Paul: We'll need to worry about scaling in production.

Maureen: The fewer annotation stores the better.

Bob: Concur. Main purpose for testing multiple stores in to make sure that we haven't got hidden assumptions about single central resources. Not clear if we have time to do that.

Alex: Part of the iDigBio integration work has an internal annotation data type that we use relatively frequently within the system - primary method for users to initiate changes from the web interface. Integration plans with FP probably involve bidirectional synchronization. Probably need to look at tighter integration.

Bob: Scalability issue is not keeping syncronized is that an annotation may be in one store but not in another.

Alex: Those types of issues are preciecly why ACIS is invloved in the iDigBio project - core research area in CS for our group.

Bob: That's good. If we can formulate that problem properly, we may be able to hand off that problem to your. I like this discussion.

Maureen: Current design involves single central annotation store, not reasoning across distributed annotation stores.

Paul: Examination of OA and AOD/AO annotation examples perhaps as a starting point for discussion.

Alex: Thought was using FP annotation data structure, likely serialization as JSON.

Bob: Thus our involvement in W3C community in on OA.

David: Annotation generation service can take JSON and create annotation RDF/XML.

Bob: Can framework emit JSON?

David: Not sure yet.

Alex: Should be relatively trivial to serialize between JSON and RDF/XML.

    • Planned FilteredPush NE Vascular Plant TCN support and differences in security model from SCAN.

Gil: Where does the NEVP tcn fit.

Paul: Concept understood, details of deployment to be worked out.

Paul: Major difference with other systems is closed shop of ApplePie to support distribution of endangered species locality information within the network

Ed: Desire in Lichens and Bryophyte TCN to deploy FP for annotations - using symbiota, and a number of collections using specify.

Discussion: Lots of folks busy in October.

  • Future coordination:


Ed and Gill to join on weekly FP calls.

Contact Alex as needed for VM issues.


  • Annotation Paper

Bob: Working on OA examples. Rewriting the paper backwards (starting at the end)...

  • Collaborations
    • Kepler

Work this weekend.

    • Apiary

Call at 3 eastern today.

    • Specify/Symbiota

Nothing new.

    • SCAN TCN

Nothing other than covered in call. Maureen: Will have new set of users providing requirements, need mechanism to document.

    • NEVP TCN
Startup meeting Sept 6, good to be able to show demo on iDigBio infrastructure.