… and points beyond

mostly about data

Browsing Posts published in February, 2007

Neat data viz project.

Video Demonstration and Download Page

The ideas in this paper will be incorporated into the Vertica database product. And unfortunately it won’t be open source. At least that’s what one company employee commented on Slashdot.

In the same way that RAID design options (e.g. 1, 5 and 10) can accommodate multiple drive failures, the Vertica system will distribute the same slice of the database to several servers. A grid of commodity hardware can act as a high-availability system and Vertica’s shared-nothing architecture enables this feature without complex design or execution.

We call a system that tolerates K failures K-safe. C-Store will be configurable to support a range of values of K.

Inserts and updates are performed on a separate data store and merged in batches. Deletes are marked with bitmasks. Rather than building a complex locking scheme for grid members, data in the read-only and write stores is stamped with a time “epoch”. Queries specify an epoch. It’s an elegant implementation that is very well suited to a data warehouse.

Started by a major contributor to the Ingres and Postgres projects, Vertica is implementing a read-optimized database that is an excellent fit for the data warehouse world. Given the founder’s support of open-source, I expect this company will follow the hybrid commercial/FOSS model of MySQL and others. Some core design features include highly compact storage, total ad-hoc read optimization, and using a shared-nothing grid design that is dead easy to implement with commodity (not High-Availability) hardware. Via Slashdot.

New database company raises funds, nabs ex-Oracle bigwigs – Network World

Master Foo Defines Enterprise Data

I’m a bit of a latecomer to the Singularity concept. As I understand it, the singularity occurs when technological evolution is able to reproduce human intelligence, likely by simulating a human brain. It must happen, but when? Kurzweil gives a date of 2045 and gives a strong justification. My favorite part is right after that when he explains that, even if his estimate of the computing power required is off by a factor of a billion, that’s only 15 years in the exponential worlds of computation and storage. It’s coming, just wait.

Ray Kurzweil’s The Singularity Is Near

Building the Cortex in Silicon — Models of the brain built from specially designed computer chips could reveal the secrets of our cerebrum.

Mimicking How the Brain Recognizes Street Scenes — First Computer Model Based On The Brain Works Well For Artificial Vision