Data Mining for a New Age.

It all starts with the data. And here, at the dawn of the internet age,
there is certainly no shortage of data. Now we just need to make sense of it.

Google Storage.

As internet use is growing exponentially, we now have an incredible amount of data on the internet. An amount that is also growing exponentially.
Luckily, Google and others help us keep track of the avalanche.

Which, obviously, is not easy. So, the Google Platform has changed a lot since the beginning. See: The 1999 Google Platform.

Now, in 2010, rumours have it, that the Google service are made available with the help of more that 450,000 servers, arranged in racks, that are located in clusters in cities around the world.

According to Google, their global data centers combined processing power might reach from 20 to 100 petaflops (an awful lot of processing power...), they handle more than 3 billion searches each day (2012), and store more than 425 million Gmail accounts.
That is just a lot of data ...

Kaggle.

Is a website that host competitions for data prediction.
NewScientist (December 1st 2012) writes:

Companies, governments and researches present data sets and problems, and offer prize money for the best solutions. Anyone can enter.
...
Now, there are already more than 64.000 registered users.
Taking part in all sorts of competitions that rely on techniques like data mining and machine learning to predict future trends from current data etc.

Interestingly, Kaggle have discovered (rather controversially) that creative data scientists can solve problems in every field better than experts in those fields can...

Well, go Kaggle!

Mining the Mobile Life.

A wealth of data can be provided by smartphones, that is, if society allow the information to be mined.

According to Scientific American, December 2012:
Smartphones sends a steady stream of location data back to centralized centers, because few users bother to opt out of such data collection, or are even aware that they can.
And a lot of information can be extracted from such location data:
Companies that parse location data, can now accurately predict, where each of us will be at any point during the day...
And, obviously, phone records can be used to tell who our friends, family and co-workers are. And when we meet them. But now there is more:
It should also be possible to figure out, when we are likely to get the flu, and what the demographics of any major metropolitan street corner will be at any moment.
Work in Haiti allowed relief agencies to send text to cell phone users whose location histories indicated that they might have been exposed to cholera.
Insights into consumer behaviour will also increase:
Consumers most likely to click on an smartphone ad - and therefore who offer the highest payoff to advertisers - are those who are sitting in a movie theater before a film has begun...

According to data mining expert Alex Pentland: Finally, humanity can sense what humanity is doing..
I.e. On the bright side: It should be possible to enhance public health, transportation and the electric grid. Privacy concerns, well, that is another matter ...

EyeWire.

Interesting article in Wired Magazine, dec. 2012, about EyeWire:

By playing a game of colouring neurons, amateur neuroanatomists trace the wires of the retina, working together to find a neuronal ''wiring diagram''. Such a map, also known as the connectome, will help us understand how the retina serves visual perception.

Anyone can sign up to play; the only qualifications are curiosity and a zest for careful observation ...
This is a new age of exploration. By recuiting enough amateur and professional scientist, we will be able to make significant breakthroughs in our understanding of the human brain.

Lead scientist on the project, Sebastian Seung, says that ''NeuroScientists have long hypothesized that our memories are encoded in our connectomes, because each experience leaves a trace on the brain by altering neural connections.
We will be able to test this hypothesis by attempting to read memories from connectomes''.

Exciting times ahead!
Start playing here (Use: Chrome or FireFox).


After playing just a short while, it dawns on you that science can be painstakingly slow and boring work... !

WordPress: [1].