Scaling Workflows and Provenance

From Filtered Push Wiki
Jump to: navigation, search

Lessons from SPNHC 2013 Demo

  • Issues using Kepler
    • Running Kepler Headless
    • Workflow throughput and parallellization in Kepler/COMAD/Kuration
  • UI
    • Providing adequate data from harvested data back to user in spreadsheet
    • Visualizing provenance in summary and in detail.

Challenges

  • performance issue; a million records, .. analysis in a reasonable amount of time; for demo: need small subsets of data
  • what happens with similar analysis on overlapping kinds of data, receiving contradictory / multiple annotations