NASA application of Big Data

Just wanted to pass along some information about how a well-known government agency is using Big Data applications, found at the site: http://open.nasa.gov/blog/2012/10/04/what-is-nasa-doing-with-big-data-today/. NASA is using big data in countless projects, as the amount of information they take in and store is almost unfathomable. In fact, NASA uses Amazon's cloud computing services the same way we do!

The challenge for NASA is that they are receiving data from various satellites orbiting Earth as well as deep-space exploration vehicles. Currently, they utilize radio frequencies to transmit data to their systems, which means each satellite can send data on the order of GB per second while the deep space crafts transmit on the order of MB per second due to the longer distance the waves must travel to reach Earth. They currently have projects in the works that will increase their outer space capacity to approximately 24 TBs of data a day. In addition, they obtain data from various other ground-based antennae. One of these, the Australian Square Kilometer Array Pathfinder. In 2016, they predict this antennae field alone will be able to generate as much as 700 TB of data a second. These numbers are almost impossible to wrap our minds around, as NASA alone is projecting to have 350 petrabytes of data in storage by the year 2030. According to the site linked above, this would be equivalent to 70 years worth of letters delivered by the United States Postal Service. It is/will be an enormous challenge for NASA to manage all of this data and use it to gain knowledge about both the universe and Earth itself.

NASA is working on many different projects to utilize the data they collect, which are summarized on this site: data.nasa.gov. Some of these applications include airline safety, where they try to predict when safety issues would occur in airliners so that the problems could be rectified quickly, and in analyzing global climate change. However, of particular interest to me was the Kepler project, which is a project attempting to find evidence of life in our universe outside of Earth. There are billions of stars in the universe of which the Sun is just one. NASA collects data about each star and any planetary bodies orbiting them in order to try and predict which planets, if any, have the ability to contain water, which is essential to life on another planet. They analyze the distance from a planetary body to its orbital star and the size of the planet. In particular, they are only looking for planets between one half to two times the size of Earth. It is very hard to collect this data due to the shear number of planetary bodies in the universe and a limit on how far our technology can see into outer space. This is useful because it enables scientists to limit the scope on which planets they should focus their search for life. By using big data capabilities, they are able to parse the list of planets they need to search from billions to perhaps in the hundreds or thousands.  In fact, the image below is from this site http://www.space.com/19044-alien-earth-exoplanets-2013.html claiming we will find the first "Alien Earth" in the current year. This link also has more information on what the Kepler project has discovered.



Finally I wanted to post this last link about a NASA-sponsored Big Data competition they talked about in the first link I posted. Although the NASA competition is closed, I used it to link to this site http://challenge.gov/search, which is a set of government-sponsored Big Data competitions similar to the Kaggle site. I would have to check with Fadel on this, but I am sure doing these would count the same as a Kaggle competition for a class grade, just something to think about.