Need to analyze big data (i.e. large computations analyzing data from a large sample)? Your own computer may now be able to help thanks to new software developed by computer scientists at Carnegie Mellon University.
What was once the domain of large computer clusters or supercomputers may now be possible with your own laptop or desktop computer. The new software performs graph computations, which are used by many companies to analyze customer data, such as on social media or search engines. But the computer scientists who developed the software hope it helps to spring innovation across several fields.
“A researcher in computational biology could do large-scale computations on their PC; a developer working on a data-center algorithm can test it on their laptop before pushing it to the cloud,” says Carlos Guestrin, according to MIT’s Technology Review. Guestrin is the codirector of Carnegie Mellon Select Lab where the new software, called GraphChi, was created. “Big data is everywhere now, but some big data isn’t as big as it once was, relatively speaking. Tools like GraphChi will let many companies and startups solve all their graph-computing needs on a single machine. It’s cost effective, and it drives innovation, too.”
Instead of relying on a computer’s RAM, GraphChi relies on the computer’s hard drive that can hold a good deal of information as the graph computation is performed. To overcome slower speeds associated with using the hard drive, Aapo Kyrola, a PhD student in the Carnegie Mellon Computer Science Department and a student of Guestrin’s, developed a method to speed up the process. Guestrin notes, according to Technology Review, that a Mac Mini utilizing GraphChi may analyze Twitter’s 2010 social graph in just under an hour compared to roughly 400 minutes using a large computer cluster consisting of about 1,000 computers.
The new software is sure to create discussion among the technology community who frequent related Usenet newsgroups. Tech savvy users often discuss new advancements in the field, and many of them are active participants in the industry. Some users may even find that the new software helps them in their personal or professional pursuits. And, as mentioned, GraphChi has applications across several fields, so don’t be too surprised to see the topic pop up elsewhere, too.
