Cool new Middleware from Twitter for distributed data

http://bit.ly/a6ZwyV

"Twitter last night offered up the code for Gizzard, an open-source framework for accessing distributed data quickly, which Twitter built to help the site deal with the millions of requests it gets from users needing access to their friends and their own tweets. It could become an important component of building out web-based businesses, much likeFacebook’s Cassandra project has swept through the ranks of webscale startups and even big companies.
Gizzard is a middleware networking service that sits between the front-end web site client and the database and attempts to divide and replicate data in storage in intelligent ways that allows it to be accessed quickly by the site. Gizzard’s function it to take the requests coming in through the fire hose and allocate the stream of requests across multiple databases without slowing things down. It’s also fault-tolerant, which means if one section of data is compromised, the service will try to route to other sections. From the Twitter blog post:"
[...]

Human Computing - Distributed Thinking - new age of Citizen Science

http://www.isgtw.org/?pid=1002401

By using the strengths of distributed computing technologies, both specialized researchers and citizens have the opportunity to participate in a new way of doing science.

We live in a time when nearly all information is available to nearly all people everywhere.

We are entering an age where all types of people can also contribute to many types of information. A school bus driver in rural Romania may be part of a biomedical research project. Or a banker in Los Angeles might moonlight as a collaborator in an astronomy project – classifying galaxies in her spare time.

This new movement in science, called “citizen science,” allows non-specialist volunteers to participate in global research. The projects are as diverse as backyard insect counts (the Firefly citizen science project), studies of how malaria develops and is transmitted (MalariaControl.net) or prime numbers searches (through PrimeGrid).

The marriage of distributed computing techniques with citizen science represents a potential revolution. It gives scientists access to more resources and makes “cybercitizens” participants in the research process. With a few mouse clicks and 20 minutes to spare a person can elect to aid scores of projects. They can aid as many or as few projects as they like, and their involvement does not damage the performance of their own computer.

Considering the average desktop is idle about 80% of the time, its spare computing cycles represent a large resource. After downloading the needed software, a computer’s spare analytical power is harvested to work on small pieces of a large problem that has been sent from the project’s server. Once completed, the results are sent back to the project. By sharing out large tasks to many computers a distributed “grid” of computers can reduce the time needed to solve complicated problems.

In the Galaxy Zoo project, everyday citizens can help astronomers do things such as catch exploding stars, or supernovae. Data for the site is provided by an automatic survey in California, at the world-famous Palomar Observatory. Image courtesy Galaxy Zoo Supernovae

Where to start

Many of these projects use the common software platform BOINC (Berkeley Open Infrastructure for Network Computing). The BOINC webpages point to nearly 50 projects, coming many domains including climate change, astrophysics, earthquake monitoring, epidemiology and searches for extraterrestrial life. These count among them Rosetta@home, Climate prediction.net, Einstein@home, LHC@home, Chess960@home including the well-known SETI@home. BOINC’s volunteers, number about a third of a million people, donate an average of 4,540.83 TeraFLOPS in 24 hours.

Other portals leading to multiple applications include World Community Grid and EDGeS. World Community Grid, sponsored by IBM, with nearly half a million members, collects humanitarian and medical applications such as Nutritious Rice for the World and FightAIDS@home.

The EDGeS project, or Enabling Desktop Grids for e-Science, allows information to pass between desktop grids based on BOINC, and service grids (publicly funded grids of connected computing clusters) such as Enabling Grids for E-sciencE (EGEE.) This makes it possible both for volunteers to contribute to applications on service grids and for researchers to put their service grid applications on volunteer, desktop grid systems.

Human computing – distributed thinking

An intriguing sub-variety of volunteer projects call for “volunteer thinking.” These projects that share out tasks which require human intelligence for accurate processing.

Through Galaxy Zoo, volunteers classify images of the near quarter million galaxies that have been collected through the Sloan Digital Sky Survey. The human brain is able to recognize shape and type much more quickly and accurately than any computer. This work helps astronomers understand how galaxies form.

AfricaMap, a UNOSAT project (the United Nations Institute for Training and Research Operational Satellite Applications Program), will give volunteers satellite images of rural Africa, who will mark roads, bridges, human settlements, rivers, agriculture fields, barren fields and more. This will update old maps and create maps for areas where they did not exist before.

This project, and others like it, are being collected under the umbrella of the Citizen Cyberscience Centre, a partnership of the University of Geneva, the UN Institute for Training and Research and CERN, Europe’s center for physics research, to help regional authorities, humanitarian workers and scientists. Accurate maps will help aid workers reach needy areas and will track the progress of climate change.

Specialized researchers and volunteer citizens can now collaborate on some of the world’s most serious problems. The internet and the Web led to a revolution in the way we access and use information. These tools, coupled with distributing computing technologies, may be ushering in a research revolution as well.

—Danielle Vention, EGEE, is a former iSGTW editor

Tom Friedman on innovation and National Lab Day

From: Ed Lazowska 'Tom Friedman on innovation and National Lab
Day'

Tom Friedman has a wonderful op-ed in today's New York Times. http://www.nytimes.com/2010/01/24/opinion/24friedman.html


Then go to the National Lab Day website, and register!
http://www.nationallabday.org/

"What the country needs most now is not more government stimulus, but more
stimulation. We need to get millions of American kids, not just the geniuses,
excited about innovation and entrepreneurship [...]

Introduced last November by a coalition of educators and science and engineering associations, Lab Day aims to inspire a wave of future innovators, by pairing veteran scientists and engineers with students in grades K-12 to inspire thousands of hands-on science projects around the country.

Any teacher in America, explains the entrepreneur Jack Hidary, the chairman of N.L.D., can go to the Web site NationalLabDay.org and enter the science project he or she is interested in teaching, or get an idea for one. N.L.D. will match teachers with volunteer scientists and engineers in their areas for mentoring.

“As soon as you have a match, the scientists and the students communicate directly or via Skype and collaborate on a project,” said Hidary. “We have a class in Chicago asking for civil engineers to teach them how to build a bridge. In Idaho, a class is asking for a scientist to help them build a working river delta inside their classroom.”
[...]

You may view the latest post at
http://www.cccblog.org/2010/01/24/tom-friedman-on-innovation-and-national-lab-day/

Anthony Doesburg: Idle time put to work, all in the cause of science

http://www.nzherald.co.nz/business/news/article.cfm?c_id=3&objectid=10602226

....in a field referred to as cyberscience, a subset of what has come to be called citizen science. To Bill St Arnaud, head of research at CANARIE, a high-speed network connecting 50,000 Canadian researchers, citizen science is a way of democratising science.

It has echoes of another internet-borne phenomenon, citizen journalism, the idea of members of the public reporting news from their communities without the filters of the corporate media.

Citizen science, however, is not about second-guessing the work of scientists, but can contribute to their efforts by giving them many sets of eyes with which to make observations and measurements, and by lending them computer power.

[...]

Community Remote Sensing and Citizen Science

[Here is a great concept that integrates citizen science, cyber-infrastructure and web 2.0 tools -- BSA]

Community Remote Sensing
http://www.igarss2010.org/CommunityRemoteSensing.asp

What is Community Remote Sensing?

Remote sensing is the sensing or collection of information ‘from a distance’. Community remote sensing is a new field that combines remote sensing with citizen science, social networks, and crowd-sourcing to enhance the data obtained from traditional sources. It includes the collection, calibration, analysis, communication, or application of remotely sensed information by these community means.

The Earth information needs of our society are vast. Until now, we have relied on government-sponsored satellites and observing systems as the foundation for this information. The rapid emergence of citizen science and social networks introduces an exciting new means for augmenting this knowledge.

The Vision for Community Remote Sensing

Information technologies will provide the foundation for society’s rapid progress in the 21st century. Information about the environment (both natural and human-built) is central to this progress. The enormity of the required undertaking – observing and understanding our world at all space and time scales – takes your breath away.

Accomplishing it will be enabled in part by citizens who contribute to ‘remotely sensed’ versions of the world around them. Governments will depend on such information to understand local details of climate change and respond to natural disasters. The private sector will use it to build online maps and virtual worlds that make commerce more efficient and accessible. Within just a decade or so, the influence of community remote sensing will be as profound for understanding our Earth as the satellite revolution has been over the last five decades.

If you are working in this area, your participation in the Collaboration will benefit both your project and the greater community. Further information, including a detailed description of the Collaboration, can be obtained from the Conference Plenary Chair Bill Gail (plenarychair@igarss2010.org, 1.303.513.5474). Limited funding may be available to help support selected projects.


An International Community Remote Sensing Collaboration

The Organizing Committee of the 2010 IEEE International Geoscience and Remote Sensing Symposium (IGARSS) is sponsoring a novel project exploring the emerging field of community remote sensing and invites your participation.

IGARSS 2010 will spotlight this emerging field with a plenary session entirely dedicated to the topic, supporting the conference theme “Remote Sensing: Global Vision for Local Action”. During the year leading up to the plenary, the Organizing Committee is identifying and highlighting existing projects that embody the plenary theme. Participating projects, selected for their promise to create either new knowledge or new technologies associated with community remote sensing, are highlighted here on the website (see links in table below). Progress is being tracked throughout the year, and results will be presented during the IGARSS 2010 plenary session. Plenary speakers will be selected from major organizations that reflect public sector, private sector, academia, and NGO perspectives on community remote sensing.

CERN launches citzen cyber-infrastructure project

http://www.unitar.org/citizen_cyberscience_centre_project

UNITAR, UNIGE and CERN Collaborate on a Citizen Cyberscience Centre

The United Nations Institute for Training and Research, UNITAR, the University of Geneva, UNIGE, and the European Organization for Nuclear Research, CERN (hereafter referred to collectively as "the Partners") have formally agreed to collaborate on a Citizen Cyberscience Centre Project.

Citizen Cyberscience
The Partners have a common interest in, and experience of, developing Citizen Cyberscience applications for humanitarian and fundamental research.

Citizen Cyberscience provides scientists with an inexpensive form of distributed computing power that is complementary to Grid technology. This is especially true for processing-intensive problems, as illustrated by the LHC@home project developed by CERN and partners for Large Hadron Collider beam studies.

[...]

The areas of collaboration include:


* use of Citizen Cyberscience to help local authorities and humanitarian workers in developing regions to use earth imagery acquired from space for emergency response and improved territorial planning and management, including disaster risk reduction and adaptation to climate change;
* use of Citizen Cyberscience to help scientists discover new drugs for infectious tropical or neglected diseases using computer-aided screening of potential drug compounds, as well as to predict the impact of using such drugs in large-scale epidemiological simulations.
* use of Citizen Cyberscience to enable scientists in developing countries with limited resources to contribute in meaningful ways to international collaborations in fundamental science, such as particle physics and astrophysics.

The age of Citizen cyber-infrastructure

[For more examples of citizen cyber-infrastructure please see http://citizen-science.blogspot.com/. Some excerpts from iSGTW--BSA]
http://www.isgtw.org/?pid=1001877
Opinion - The age of citizen cyberspace
________________________________________

Using LHC@home, particle beam dynamics can be studied with volunteer computing. Image courtesy CERN
(François Grey, one of the key people behind the founding of the present-day iSGTW and a frequent contributor to these pages, argues that with volunteer computing, we are about to embark upon a new era of “citizen science.”)

….PrimeGrid, is tackling a host of numerical challenges, such as finding the longest arithmetic progression of prime numbers (the current record is 25). Professional mathematicians now eagerly collaborate with Rytis to analyze the gems that his volunteers dig up. Yet he funds his project by selling PrimeGrid mugs and t-shirts. In short, Rytis and his online volunteers are a web-enabled version of a venerable tradition: they are “citizen scientists.”
There are nearly 100 science projects using such volunteer computing. Like PrimeGrid, most are based on an open-source software platform called BOINC (see 15 October 2008 iSGTW, “Reaching for the Exa-scale”) with volunteer computing. Many address topical themes, such as modelling climate change with ClimatePrediction.net), developing drugs for AIDS with FightAids@home, or simulating the spread of malaria with MalariaControl.net. (See 7 May 2008 iSGTW, “WISDOM unplugged: Malaria drug-leads graduate to the wet lab”)

Fundamental science projects are also well represented. Einstein@Home (iSGTW 14 May 2008) analyzes data from gravitational wave detectors, MilkyWay@Home simulates galactic evolution, and LHC@home studies accelerator beam dynamics. Each of these projects has easily attracted tens of thousands of volunteers.

[..]
A new wave of online science projects, which can be described as “volunteer thinking,” takes the idea of participative science to a higher level. A popular example is the project GalaxyZoo, where volunteers can classify images of galaxies from the Sloan Digital Sky Survey as either elliptical or spiral, via a simple web interface. In a matter of months, some 100,000 volunteers classified more than 1 million galaxies. People do this sort of pattern recognition more accurately than any computer algorithm. And by asking many volunteers to classify the same image, their statistical average proves to be more accurate than even a professional astronomer.

[..]
Going one step farther in interactivity, the project Foldit is an online game that scores a player’s ability to fold a protein molecule into a minimal-energy structure. Through a nifty web interface, players can shake, wiggle and stretch different parts of the molecule. Again, people are often much faster at this task than computers, because of their aptitude to reason in three dimensions. And the best protein folders are usually teenage gaming enthusiasts rather than trained biochemists.

Who can benefit from this web-based boom in citizen science? In my view, scientists in the developing world stand to gain most by effectively plugging in to philanthropic resources: the computers and brains of supportive citizens, primarily those in industrialized countries with the necessary equipment and leisure time. A project called Africa@home, which I've been involved in, has trained dozens of African scientists to use BOINC. Some are already developing new volunteer-thinking projects, and the first African BOINC server is running at the University of Cape Town.

A new initiative called Asia@home was launched last month with a workshop at Academia Sinica in Taipei and a seminar at the Institute of High Energy Physics in Beijing, to drum up interest in that region. Asia represents an enormous potential, in terms of both the numbers of people with internet access (more Chinese are now online than Americans) and the high levels of education and interest in science.

To encourage such initiatives further, CERN, the United Nations Institute for Training and Research and the University of Geneva will establish a Citizen Cyberscience Center on 2 July. This will help disseminate volunteer computing in the developing world and encourage new technical approaches. For example, as mobile phones become more powerful they, too, can surely be harnessed.
There are about one billion internet connections on the planet and three billion mobile phones. That represents a huge opportunity for citizen science.