Go Back > Great Internet Mersenne Prime Search > Math

Thread Tools
Old 2006-08-21, 20:09   #1
ewmayer's Avatar
Sep 2002
República de California

13·29·31 Posts
Default Distributed Internet Cryptography

A Move to Secure Data by Scattering the Pieces -- New York Times

Describes an open-source distributed cryptography scheme largely based on the ideas in Adi Shamir's 1979 paper, "How to Share a Secret", independently reported by G. R. Blakley that same year ("Safeguarding cryptographic keys", Proc. National Computer Conference, 48, pp 313–317, 1979.)

Article text, in case the above link eventually becomes inactive:
Published: August 21, 2006

Chris Gladwin, a software designer and businessman in Chicago, had time on his hands after selling his company, the online music store Music Now, in 2004. So he decided to digitize all of the music, photos and paper detritus that he had been meaning to organize for years.

After he was finished, he discovered that he had 27 gigabytes of data — equivalent to a library of 22,000 books — that he was eager to protect. “I wondered, ‘what are my options?’ ” he said, “and I realized that none of them were that good.”

But he had been reading histories of early encryption research, and he saw a germ of an idea in the work of cryptographers who kept information secure by dividing it into pieces and dispersing it.

So what began as a home improvement project culminated in a system called Cleversafe, with potential applications far beyond Mr. Gladwin’s memorabilia. For companies and government agencies trying to secure networked data, it offers a simple way to store digital documents and other files in slices that can be reassembled only by the computers that originally created the files.

The idea of distributed data storage is not new. But Cleversafe is significant because it is an open-source project — that is, the technology will be freely licensed, enabling others to adopt the design to build commercial products. That approach may contribute to Cleversafe’s potential to lower the cost of reliably storing data on the Internet.

“If we distributed data around the world this way, it would be a pretty resilient way to store data,” said David Patterson, a computer scientist at the University of California, Berkeley, who is a pioneer in designing distributed data storage techniques.

Mr. Gladwin contends that Cleversafe can store data at a lower cost and make it more secure than current Internet services. The group is counting on a continuing explosion of consumer digital data of all types, including new generations of high-definition still and video cameras that will create demand for secure and private backup capabilities.

Computer scientists argue that projects like Cleversafe are an indication that the broadband Internet will soon have the same impact on data storage that it has had on computing and communications technologies. Dozens of commercial Web storage services are already used to back up data safely. In addition, Amazon’s S3 and other services are intended to enable an array of digital Internet services to operate without any local storage capacity.

But the current design of such services generally involves making as many as five or more complete copies of the original data and storing them at multiple locations to ensure that information is not lost through a drive failure or other catastrophe. The Cleversafe design will cut the amount of storage space needed for secure backup by more than half.

Mr. Gladwin, 42, said he was deeply influenced by a seminal paper, “How to Share a Secret,” written in 1979 by Adi Shamir, a designer of the encryption algorithm known as public-key cryptography. The paper describes how a message can be broken into pieces and then reassembled from a subset of those pieces without revealing the message.

Mr. Gladwin developed a set of software routines that would copy the data stored on his PC into a large number of fragments, or slices. The mathematics of his solution had an additional benefit: the original data could be reconstructed from a majority of the slices. The design made it possible to retrieve a complete set of his original data even if some of the disks that held portions of the data failed or went offline.

The design of such “distributed file systems” is already a rich area of computer science research, and commercial systems are widely available in the software and data-storage markets. But Mr. Gladwin argues that his new standard offers security and efficiency features not easily available either to information technology managers or to individual computer users.

The experimental Cleversafe research grid is located at 11 storage sites around the world, but Mr. Gladwin is hoping that a commercial network will evolve, composed of tens of thousands or hundreds of thousands of storage sites that will be accessible at low cost.

The Cleversafe design could lead to a communal Internet storage system that Mr. Patterson called “hippie storage.” The idea is similar to SETI@Home, the shared computing system that allows PC users to contribute idle time on their machines to create a distributed supercomputer.

Today most distributed storage systems work by making multiple copies of data at multiple locations and then using various mechanisms to keep the copies synchronized. Examples include distributed file systems from Microsoft and Google as well as a system designed by software developers at Stanford known as Lockss — Lots of Copies Keep Stuff Safe — that is used to preserve the digital versions of academic journals.

The Cleversafe project uses a different approach based on dispersing data in encrypted slices rather than copying it. That approach shares some design similarities with a Berkeley research project known as OceanStore, which is also intended to create a globally distributed computer storage system.

“They’re not making a commercially implemented solution,” Mr. Gladwin said of the Berkeley project. “Our focus is something that people can use.”

A storage industry analyst said that such an approach had significant potential.

“The great thing about storage is that it’s always a moving target,” said Michael Dortch, principal business analyst at the Robert Frances Group, an industry consulting organization. “The I.T. industry is littered with the bodies of people who have said solution X will never fly.”

The Cleversafe project, with 25 employees, is housed on the campus of the Illinois Institute of Technology in Chicago. Mr. Gladwin said the school had been an ideal technology incubator because of the ready availability of student technical talent.

One company considering the Cleversafe software is Univa, a developer of grid computing software and systems. “The potential to be able to geographically distribute data over the Internet has very nice properties,” said Steve Tuecke, a founder and chief technology officer of Univa, in Lisle, Ill.

An early financial backer of the project, Stewart Alsop, argues that Cleversafe is an indication that the open-source software movement is shifting from merely reusing existing designs to becoming a force for innovation.

“Data storage on the Internet is one of the most brutally competitive markets in the world,” he said. “But nobody is using this architecture, and the logical benefits of this are remarkable.”
ewmayer is offline   Reply With Quote
Old 2006-08-21, 20:53   #2
garo's Avatar
Aug 2002
Termonfeckin, IE

22·691 Posts

As someone who has done some research in this area I can say that the idea is not exactly new. Erasure codes have been widely known for about 5 years now and they do precisely what Cleversafe claims - divide data into chunks such that only some chunks are necessary to reconstruct the original data. Just type "digital fountain" in google to get the original paper that discussed "erasure codes". The only difference is that erasure codes were intended to be used to send large quantities of data across lossy channels.

Oceanstore provides an example of a file storage app that does the same thing. Yes Cleversafe commercializes this technology but is that a good thing Anyway, I'll follow the company's success or lack thereof with interest.

I do find the way NYT starts of all its science articles rather tiresome. There has to be a "human interest" angle which lesser mortals can understand. You can't have a scientist doing stuff for the love of knowledge or even money. It has to be some minor thing that led to something great.

PPS: Methinks Stewart Alsop is talking through his hat - or giving the reporter a tasty soundbite when he claims that "Cleversafe is an indication that the open-source software movement is shifting from merely reusing existing designs to becoming a force for innovation". What the hell was Napster and everything that has happened since?

Last fiddled with by garo on 2006-08-21 at 20:58 Reason: Added PPS
garo is offline   Reply With Quote

Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
..New algorithm shakes up cryptography"? jwaltos Operation Kibibit 5 2016-01-30 09:33
CUDA and cryptography ET_ GPU Computing 10 2012-09-24 10:27
Metrosexual/Bi-Curious Cryptography article ewmayer Math 4 2012-09-09 06:43
GIMPS and cryptography Rodrigo Lounge 4 2012-07-31 16:03
PCs not connected to internet kdelisle2005 Software 3 2011-06-22 03:09

All times are UTC. The time now is 19:49.

Tue Jan 18 19:49:40 UTC 2022 up 179 days, 14:18, 0 users, load averages: 1.74, 1.77, 1.70

Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, Jelsoft Enterprises Ltd.

This forum has received and complied with 0 (zero) government requests for information.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation.
A copy of the license is included in the FAQ.

≠ ± ∓ ÷ × · − √ ‰ ⊗ ⊕ ⊖ ⊘ ⊙ ≤ ≥ ≦ ≧ ≨ ≩ ≺ ≻ ≼ ≽ ⊏ ⊐ ⊑ ⊒ ² ³ °
∠ ∟ ° ≅ ~ ‖ ⟂ ⫛
≡ ≜ ≈ ∝ ∞ ≪ ≫ ⌊⌋ ⌈⌉ ∘ ∏ ∐ ∑ ∧ ∨ ∩ ∪ ⨀ ⊕ ⊗ 𝖕 𝖖 𝖗 ⊲ ⊳
∅ ∖ ∁ ↦ ↣ ∩ ∪ ⊆ ⊂ ⊄ ⊊ ⊇ ⊃ ⊅ ⊋ ⊖ ∈ ∉ ∋ ∌ ℕ ℤ ℚ ℝ ℂ ℵ ℶ ℷ ℸ 𝓟
¬ ∨ ∧ ⊕ → ← ⇒ ⇐ ⇔ ∀ ∃ ∄ ∴ ∵ ⊤ ⊥ ⊢ ⊨ ⫤ ⊣ … ⋯ ⋮ ⋰ ⋱
∫ ∬ ∭ ∮ ∯ ∰ ∇ ∆ δ ∂ ℱ ℒ ℓ
𝛢𝛼 𝛣𝛽 𝛤𝛾 𝛥𝛿 𝛦𝜀𝜖 𝛧𝜁 𝛨𝜂 𝛩𝜃𝜗 𝛪𝜄 𝛫𝜅 𝛬𝜆 𝛭𝜇 𝛮𝜈 𝛯𝜉 𝛰𝜊 𝛱𝜋 𝛲𝜌 𝛴𝜎 𝛵𝜏 𝛶𝜐 𝛷𝜙𝜑 𝛸𝜒 𝛹𝜓 𝛺𝜔