The majority activity today was reviewing the proposals we received for the Fall 2014 edition of our Data Science Incubator. We received 26 fantastic proposals ranging from Astronomy to Statistics (sadly, no Zoology), and we are probably going to have to cut it down to 5 or 6. Lots of ...
read moreThere are comments.
Next week, I’ll see if the incrementalization actually helps us scale.
Only had a tiny bit of time today; I worked more on the least common ancestor query. Here is what new work contributed to better scaling:
Incrementalizing the code (duh) did in fact let me scale it farther ...
There are comments.
In between meetings, I spent most of today continuing yesterday’s work on the citation use case. Further query rewrites and testing exposed an interesting bug in the optimizer due to a mismatch between logical algebra representation and the actual system implementation behavior — the optimizer assumed the system could perform ...
read moreThere are comments.
Today I picked up some of the work that Sandra Anderson did in her summer internship, namely trying to find common citations (transitively) between pairs of papers in Jevin West‘s data sets.
Once again I identified a number of nice optimization opportunities:
There are comments.
Today we held the information session for the second installation of our Data Science Incubator projects which we will hold in the Spring. It was fairly well attended; maybe 20—25 people came and many of these indicated that they will be submitting proposals.
Over the weekend and today I ...
read moreThere are comments.
Today I met with UW CSE undergrad York Wei and Brandon Myers for demos and code review of York’s summer Project. York and Brandon have adapted our Myria Web frontend to extend the current support for Myria with the ability to issue queries to two new backends:
There are comments.
Today’s highlight was another hack session with Sophie Clayton in the Armbrust Lab. The same query to compute the cytometric diversity (“richness”) that we ran a month ago on a smaller dataset now ran out of memory after 12 minutes; after an hour of futzing I got it to ...
read moreThere are comments.
The big news today is that Myria now has support for user-defined aggregations. I’m excited that we can soon put this new feature to work and asymptotically speedup Sandra’s citation graph analysis work, which is currently throttled by memory pressure.
read moreThere are comments.
We had our monthly SeaFlow/eScience group meeting meeting. For this grant the oceanographers have been doing lots of new science using tools like SQLShare, Myria, and popcycle, our software for storing indexing and analyzing SeaFlow data. We discussed needed improvements to popcycle and to the seaflow-viz web dashboard (see ...
read moreThere are comments.
Today we met with UW-IT to talk about the future of SQLShare. We will be starting some sprints in the fall to prepare it to be transitioned to them as an on-campus service.
Spent a fair bit of time reading and discussing Brandon Myers’ paper using Raco to compile ...
read moreThere are comments.