Essentially the core protocol is a gossip overlay. This generates random connections between peers in the overlay. Over these connections peers transmit information like recently seen torrents, recommendation lists, download/upload totals for other peers etc. Each peer proactively distributes known torrent files so that every peer builds up a database of torrents. Then search just works by broadcasting queries to nearby peers.
The algorithms used have dozens of applications outside of filesharing. I wrote my MSc dissertation on a similar subject:
The code for Tribler is naturally very specialised to their purpose. At some point I want to sit down and reimplement them in such a way that they can easily be reused and recombined for other applications. I have some vague ideas about building a prototype trust-based distributed database for dot-p2p.org based on similar principles.
> Essentially the core protocol is a gossip overlay.
Is this a broadly similar idea to NNTP 'flood fill', where once you have a peer (discovered in some way), you exchange articles (or in this case torrents) you each have - so they propogate to all connected nodes in the graph?
If so, doesn't this have problems scaling? (All nodes receive all torrents)?
Each node has a subset of all torrents. You flood searches as well. There is also a second overlay on top of the first which groups peers by their similarity in taste, with the aim of reducing the number of hops necessary to find what you're looking for. The taste similarity is also used to actively propagate recommendations so that stuff you're interested in is more likely to move towards your immediate vicinity.
I would recommend scanning through the paper to get a more accurate idea of how it works. My knowledge of Tribler is pretty out of date.
wasn't the biggest issue with the first gnutella networks that search through flooding basically does not work? Is this somewhat accomodated by the fact that now we, well, can basically keep much larger databases?
Bigger databases, proactive distribution of torrents and similarity based clustering of users. Tribler is too small as yet to know if it will work but it certainly looks like an improvement on gnutella.
Very similar, except that Skynet was a vast, distributed, self-aware intelligence controlling most of the worlds machines and Tribler is a file-sharing network. But they both run on computers ...
Most of the underlying protocols are described here, though it is a bit out of date:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.78....
Essentially the core protocol is a gossip overlay. This generates random connections between peers in the overlay. Over these connections peers transmit information like recently seen torrents, recommendation lists, download/upload totals for other peers etc. Each peer proactively distributes known torrent files so that every peer builds up a database of torrents. Then search just works by broadcasting queries to nearby peers.
The algorithms used have dozens of applications outside of filesharing. I wrote my MSc dissertation on a similar subject:
http://scattered-thoughts.net/one/1283/644001/538941
The code for Tribler is naturally very specialised to their purpose. At some point I want to sit down and reimplement them in such a way that they can easily be reused and recombined for other applications. I have some vague ideas about building a prototype trust-based distributed database for dot-p2p.org based on similar principles.