In the morning, Andy Becker and I met with Andy Connolly, who was apparently not in London at the time, to give him an update on our KBMOD progress this quarter. We talked about the database designs, the scaling lessons, the technology tradeoffs, and the potential of our new approach ...
read moreThere are comments.
More work with Andy Becker on the KBMOD database and query design today. We have been running into issues loading all the pixels in the database — O(Millions) of images with O(Millions) of pixels each means 10^12 records, which would take years to load into our Postgres database ...
read moreThere are comments.
Today was day 1 of Incubator week 3.
Sophie Clayton exposed a locking issue in SQLShare: if a running query reads from table T and a user changes the sharing on table T, SQLShare basically locks the entire system until the query finishes. We need to revisit this issue, but ...
There are comments.
Over the weekend, both Sophie Clayton and Andy Becker worked independently on their Data Science Incubator projects; I spent some time then and today answering emails :).
Sophie has been loading underway data (GPS, temperature, salinity, etc. from ships in motion) into SQLShare for cleaning. Every research vessel is its own ...
read moreThere are comments.
I spent today in the Data Science Incubator working with Andy Becker on KBMOD. We are trying to port the key components of the moving object detection into PostgreSQL using PostGIS.
On his own, Andy designed a schema for the various data, including spatial columns and indexes. Side by side ...
read moreThere are comments.