From Filtered Push Wiki
Jump to: navigation, search

Etherpad for meeting notes: http://firuta.huh.harvard.edu:9000/FP-2011Mar08

User: David Lowery | User: Zhimin Wang | User: BertramLudaescher | User: James Hanken | User: Paul J. Morris | User: James Macklin | User: Lei Dou | User:
"User:" cannot be used as a page name in this wiki.
| User:
"User:" cannot be used as a page name in this wiki.
| User:
"User:" cannot be used as a page name in this wiki.
| User:
"User:" cannot be used as a page name in this wiki.



  • Paul
    • Refined Use Case diagrams, began adding in Requirements as notes on diagrams.
    • Began creating example annotation ontology documents as examples for mapping.
  • Zhimin
    • With Bob, added detail to the architecture diagram
    • With Lei, worked on the update part of schema mapping
  • James
    • Continued to work on the Canadensys+ proposal which rather generally includes reference to the Filtered Push (the proposal does not include much technical discussion).
    • Will attend a workshop this Friday and Saturday on the Flora of the Canadian Arctic project. I will present on Filtered Push and digital floras (using the concept we have in the grant as a demonstration) as this will be a key component of the project.
  • Lei and Bertram
    • Worked on update (especially insertion) exchange between two database schemas by using data exchange idea from Clio system

Meeting Notes

Filtered Push Team Meeting Tuesday 2011 Mar 08

Present: Bertram, Lei, Jim, James, David, Zhimin, Paul.

Report from Friday Tech Meeting: Mapping update/insert queries

Paul: Continuing look at how to map annotations into inserts and updates into local databases. Lei: Clio able to handle transformation of queries from global schema to an intermediate structure matching that of the local schema. Then we need to ourselves solve the transformation from that structure to the local database, with rules to determine when to do inserts and when to do updates. James: Network or Client? Zhimin: On the boundary of the two.

Prepare Video for Amanda for Field Museum SI2 meeting.

James: Paul, James, and Bob will all be in a semantic meeting in Arizona at the same time as this meeting in Chicago. We (and the organizers in Chicago) feel strongly that FP should be part of the discussion. Amanda came up as a possible person to give a FP demostration, best way to do this is probably a canned video (screenshot video with narration).

Need to find out how long it will be.

Action Item: James and Zhimin to start on this on Thursday afternoon.

Jim: What's the meeting about at Chicago? Sponsored by?

James: Meeting about infrastructure about efficient datacapture. Sponsored by the synthesis center.

Jim: Well made video to promote FP a good idea for generic use. Harvard-Smithsonian center for Astrophysics has a very good team for helping with this, timescale probably too short for this, but we should plan to use this capability later.

===FP SPNHC Abstract

  • (end of May meeting, abstract for demo-camp March (18?))
  • (==> also: when/who should we be there!? Do we need to register? URL?)

James: Workflows would be a good target. Bertram: This would be great. Lei? Lei: Show how the workflow can easily find and visualize errors, and improve the data qualtiy, with agents for cleaning the data and for visualizing the results. Paul: Could also do duplicate finding. Zhimin: Workflows for georeferencing textural data. (Paul: with agents that use existing services). Bertram: Other pieces than abstract? James: Abstract, then demonstrate tool live.

Wed, Thur, Fri at http://research.calacademy.org/spnhc Thur: Demo-camp (aside: Is Harvard running this: http://www.spnhc.org/ (we were)?)

Bertram: Lei to coordinate abstract!? Starting with a google doc (need url w/ submission instructions)

Bertram: How about provenance. James: People are very afraid of loosing chain of attribution in data aggregation. Jim: Provenance also a key issue in EOL, yes, provenance is a key issue for the professional scientists.

Provenance a very good focus.

James: Bob put up a doodle calendar for us to figure out when we might have a FP meeting in california before or after SPNHC. Perhaps a short FP full team meeting during SPNHC, then a couple day meeting on one side to do some focused work together.

iEvoBio: Data Integration Challenge

Jim: At the evolution society meeting. James: Bob intends on going. Paul: Multiple ducks that are getting placed in line to make this submission, review of source code, copyright statements, and licences on the UBM codebase, then review of UMB-Harvard subcontract agreement, then Harvard technology office form submission. Bertram: Bob to coordinate abstract!? [yes]

Path for client library: mapping, dummy RDB, then to Specify and MCZbase.

Jim: My sense is that we shouldn't have to customize against specific (e.g. local foxpro databases). James: Correct. Paul: Specify as widely available client, then library for others with programming capability, then web interface for others with their own local solutions and their Zhimin: Two parts to client library. (1) generic for all platforms, (2) adaptor for specific platforms. The general use case can thus be separated from specific local requirements.

From Paul (earlier in an email concerning a road map for mapping in the client library)

  • (1) a tool that allows a DBA to construct a set of rules for mapping (stored in a configuration file),
  • (2) client library functionality that allows the ingest of annotations from the network, accept/reject/ignore decisions (and attendant annotations) on those, and the use of the mappings in the configuration file to generate the data exchange query,
  • (3) a GUI for (2), and
  • (4) client library functionality that allows the merge of this data exchange query into the local database, with co-commitent changes to the first 3 pieces.