Bittorrent File System

This is a paper to discuss an idea for a distributed cloud computing system. This system would use nodes to distribute and hold the data that are non co-located.

From the front end, the system would be identical to any other cloud computing solution. The user could make calls to retrieve or store data from the web (or other applications).

On the back end, the system would differ from a standard cloud system in that instead of being an array of centralized servers, the system would use a P2P method of distributing the data and loads throughout itself.

Let’s take the example of incoming data.

Data is sent to the cloud. This data is then processed by the RAID software and divided as required (RAID 5, 7, ETC). The distributed part then takes each part of that raid split, call them bits (not those bits) for this paper.

Now each bit will be distributed to multiple clients via a bittorrent like P2P system. One bit would then be copied on 2, 4, 8 nodes (however many deeded needed for reliability).

Then when the data is called, the raid will call the data just as it would normally. However, the system will retrieve the data in this bittorrent fashion by calling it from the available nodes. Nodes that have dropped out or are slower will be tagged and recorded, the data will be pulled from the better sources.

Once assembled by the bittorrent, the bit will be presented back to the RAID as desired. The RAID will assemble the file as required and present it back to the web request.

As stated, the nodes themselves will be tagged and recorded for performance when needed. Highly reliable nodes will be called on more frequently than lower reliable drives. The system would ensure that bits lived on a certain percentage of higher reliability than on the lower reliability. The bittorrent client would use this data to shift bit data onto different nodes, populating data as required.

Lower reliability nodes are not completely useless. They can be used to help with this repopulation as well as for storage of lower requested data. This logic would be based on available cloud space, amount of traffic, even peak times and peak availability.

Now the question is: Why? Why divide this system up into all of these nodes and introduce another step in the process over the current system? The answer is that this is not the current system. This is where the ‘distributed’ part comes in.

For the nodes we create a client program. This is installed on a computer and allows for configuration of amount of space, location, etc. Then this computer is now a node. So instead of setting up a huge server farm, node software can be installed on multiple computers spanning anywhere there is an internet connection.

Imagine if all the computers in a college computer lab donated just a single gig of space to being a node. Then maybe all of the computers in an Apple store or a Best Buy. A PS3 or XBox client could be made to contribute. Even an iPhone or Blackberry client offering as little as 10megs of space could be used.

This would significantly reduce outside influences on data availability. Things like power outages, natural disasters, even high traffic load due to sporting events could be worked around by spreading the data geographically both on an individual node level and on a node collection level.

The space would be tallied, prioritized by various parameters and prepared for use.

Security will be a concern of people using this. How do I know that my data is safe out there on someone’s personal PC? First only a small part of any one file would be located on any one node. The next upstream information available to the user is the bittorrent client which only knows where other copies of the bits are. The user would have to go up an additional step into the RAID and then back down through the bittorrents to find a usable chunk of any one file.

This is the same argument for the security of the node provider as well. For example, should a pirated movie be uploaded to the cloud, the nodes themselves would get parts so small that none of them would have enough for a reasonable argument that it was known to be there.

Extra security could be imposed by making the node into an encrypted image. This would further ensure the security, but may have negative impact on the speed of the node. This would need to be investigated.

This distributed cloud computing allows for a more robust system by decentralizing the hardware as well as allowing for expandability beyond boundaries such as building size and electrical power. It would take the one last part of open source cloud computing, the cloud itself, and allow it to be open as well.

A system such as this could be used in a grand scale, one large cloud, or in smaller forms, several small clouds that could be specialized. Just like bittorrent itself, there could be multiple gateways (torrent trackers, or cloud cars?) into the cloud.

How Twitter could Kill SMS (and save us all)

I have a blackberry. I pay $30 for a data plan that will let me send as many emails as I want. Full emails. Long emails. Emails with content, pictures, punctuation and correctly spelled words even. This data plan does not cover SMS though. For that I have to pay extra, or pay 15 cents per text or a $10, $15 or $20 a month plan on top of my unlimited data.

Here is a great article on SMS bandwidth charges.

http://gthing.net/the-true-price-of-sms-messages/

The short version: we are getting ripped off.

Here is where Twitter comes in. Or could, I should say. The intranets were abuzz a few months ago because there was a rumor that Apple was going to buy Twitter. (and about eight other companies, but topic at hand) This got me thinking. What in the hell would Apple want with Twitter?

SMS.

And no, Apple didn’t buy them, and no, I don’t think they are planning this, but this was the idea I had, the thing I reasoned out as to what value Twitter could have to Apple. Here is the idea: Twitter replaces SMS. Instead of sending a text message to someone, you send a direct message to someone.

Now, granted, this is a better idea for the data plan phones. Blackberries, iPhones and Androids all have a data plan and could add on a Twitter client to have this functionality (and should, damnit!). The normal phones would need some help.
 
The idea would be that a normal phone, one that would have to use SMS anyway, could SMS Twitter, something that can already be done, to bridge the gap between an older SMS phone and a newer Twitter phone. This isn’t as easy as it could be though.

The connection between the address book and Twitter still needs to be made on most of the platforms. I need to be able to find “mom” in my phone book, and hit message. Not what I’d need to do currently, which is find “twitter” and then type “d momstwitter message” If twitter clients could make this link, then sending a DM would be as easy as sending an SMS.

Then this provides all the advantages of SMS and all the advantages of Twitter at once. Need to contact someone? Just send a DM. It will act as a tweet and as an SMS at the same time. You can text someone (twext?) from your computer, and likewise they could answer from there. (something Google Voice is working on)

You’d have a telephone number independent way of mobile communication. Walk away from your carrier, your phone, hell, even dare to go out of the country without impunity. Your twitter account is still yours. 

Then as more and more phones have data plan options, more phone users can get off of SMS. With SMS plans as high as $20 a month and data as low as $30, the advantages that SMS brought once are slipping away. New phones will be more internet capable and features that were once only for expensive phones will be as common as cameras are today.

And maybe this won’t save us all, but it would be a step in the right direction.

http://help.twitter.com/forums/59008/entries/14014

Oh, and if you have a smart phone, email, IM, anything but SMS when you can.

The New In This Moment Album

(oops! this was supposed to be posted a while ago. sorry!)

When I first found out about In This Moment I thought I had found a small slice of heaven here on this Earth. A metal band with a chick singer. And not just a chick singer, but an “I’M GONNA RIP YOUR FACE OFF” chick singer. I heard Daddy’s Fallen Angel on the radio and couldn’t get to the record store fast enough.

I got the album and was not disappointed (well, except for the fact it was short). Daddy’s Fallen Angel definatly woke up my neighborhood a few times, and Circles became my own ironic workout song.

So when The Dream came on Tuesday (30 September, 2008), I got it. No questions, no reviews, walked in and bought it without thinking about it. I put it in my car stereo and turn it up ready for it, waiting for it. The first song is an intro similar to what “Beautiful Tragedy” had. This was a bit more of an Indian/Middle Eastern sound, like something you’d hear on a Delerium album. I was ready for it.

And what I got was a bit different than what I expected. Rather than the deep crunch of the first album what came out took me by surprise. My first reaction was “what is this?” The album was slower, softer, and seemingly more radio friendly. This was not what I was expecting, not what I wanted. Don’t get my wrong, it is still metal, but more of a Metallica metal, rather than a Lamb of God one.

I didn’t just dismiss it, though. I left it in my disc payer and listened to it a few times more. I found a solid, well made metal album with a much more refined sound than its predecessor. The band that made “The Dream” had come into itself more than the one that made “Beautiful Tragedy”. I thought to check changes in writers or producers, but in the end it is what comes out of the speakers that is important. “The Dream” delivers, even if it is a different package than “Beautiful Tragedy”. I found myself turning it up loud, and perhaps even shifting for emphasis during several of the songs.

The verdict? Buy it. Really, it was great. Yes, I still put in “Beautiful Tragedy” and wish for its spiritual successor, for ITM to turn it up again, but “The Dream” delivered an album experience I can’t turn off.