A private bit torrent tracker written in python
Eric Urban 2d99c9bd3a make rtorrent reseed more robust, set private 4 years ago
html fixed front end and misc stuff 4 years ago
screenshots new images 4 years ago
test expounded upon test 4 years ago
.gitignore Initial commit 5 years ago
BTL.py moved licenses for specific files into the comments of those files themselves 5 years ago
LICENSE added license 5 years ago
README.rst correct typo 4 years ago
adduser.py updated to make 'addUser' script create a user with invite and role changing privileges 4 years ago
auth.py change to expose user id in tracker 4 years ago
bencode.py moved licenses for specific files into the comments of those files themselves 5 years ago
building_dependencies updated list of dependencies 5 years ago
chpasswd.py silly script to change an arbitrary users password 5 years ago
computeInfoHashExample.py milestone! now authenticating against pgsql db 5 years ago
convOldTorrentStore.py ported from filesystem based store to GDBM based store. Added support for extended data when uploading torrents 5 years ago
example.conf.json updated example config file 4 years ago
fairywren.py testing for new stats stuff 4 years ago
fairywren.sql working on new swarm tracking stuff 4 years ago
monotonic.py hackish implementation of monotonic clock 5 years ago
peers.py working on new swarm tracking stuff 4 years ago
permissions.sql updated permissions to match new table 4 years ago
postgres_transition.py transition scripts to move everything postgresql 4 years ago
postgres_transition.sql transition scripts to move everything postgresql 4 years ago
restInterface.py don't return the whole value for bigass parameters 4 years ago
roles.sql added example SQL files for roles and permissions 5 years ago
rtorrentReseed.py make rtorrent reseed more robust, set private 4 years ago
runDbTest Swarm stuff with test, now to fix the WSGI side 4 years ago
runTest README updated. All tests passing except swarm resource 4 years ago
standalone_tracker.py fixed front end and misc stuff 4 years ago
standalone_webapi.py fixed front end and misc stuff 4 years ago
swarm.py added log messages 4 years ago
torrents.py add code to show seed/leech coun ton torrent info request 4 years ago
tracker.py support for arbitrary function invocation after a callback 4 years ago
upload.py fixed wrong exception type 4 years ago
users.py add ability to retrieve only username 4 years ago
vanilla.py refactor to pass connection pool to Peers object 4 years ago
webapi.py repaired swarm backend, now to tackle the javascript 4 years ago

README.rst

=====
Fairywren
=====

Fairywren is a private BitTorrent tracker written around Postgresql and
Eventlet.

Features
=======

- Upload torrents through web interface
- Search for torrents by title
- View number of seeders and leechers in real time
- Create one time invites to send to new users
- HTML5 & Javascript written using Bootstrap, displays well on phones and tablets
- Backend of web interface is entirely RESTful

Screenshots
=====

Browsing and creating torrents
-------

.. image :: screenshots/newest.png

.. image :: screenshots/upload.png

.. image :: screenshots/search.png

Inviting new users
-------

.. image :: screenshots/account.png

.. image :: screenshots/register.png

Software Dependencies
=======

To run fairywren you'll need the following

.. _Stackless: http://stackless.com/wiki/Download
.. _Eventlet: http://eventlet.net
.. _Psycopg2: https://pypi.python.org/pypi/psycopg2
.. _Multipart: https://github.com/hydrogen18/multipart
.. _redis-py: https://pypi.python.org/pypi/redis
.. _Redis: http://redis.io/download
- Stackless_
- Eventlet_
- Psycopg2_
- Multipart_
- Postgresql
- Redis_
- redis-py_
- A web server that supports proxying. I use lighttpd.


The unit tests also require

.. _wsgi-intercept: https://pypi.python.org/pypi/wsgi_intercept

- wsgi-intercept_

Configuration
=======

The configuration file is a JSON file which configures both the tracker
and the API. The basic outline is shown in example.conf.json. The JSON
file is a dictionary. The keys are

trackerUrl
The external URL that the web server proxies to the tracker

pathDepth
An integer specifying the depth at which the API and the tracker are proxied from. This is
used to allow the code to be independent of the website it is hosted
on. For example the URL http://a.com/b/d/f/api/torrents with this
configuration value set to 4 causes the first four parts of the path
to be ignored and just 'torrents' to be matched against when processing
the request.

salt
A string used to salt users password before storing them in the database.
This value should be random, long, and guarded as secret. Changing this
value after adding users is equivalent to setting all users passwords to
random values.

.. _webapi:
webapi
Configuration values specific to the API. See webapi_.

.. _tracker:
tracker
Configuration values specific to the tracker. See the tracker_.

redis
------
All keyword arguments to pass to the construct for Redis connections

tracker
------

postgresql
A dictionary of values. These are passed to the constructor of
psycopg2.connect verbatim

webapi
------

postgresql
A dictionary of values. These are passed to the constructor of
psycopg2.connect verbatim

secure
A boolean indicating if sesssion cookies issued should be flagged
with the 'Secure' option. Used when running behind an HTTPS proxy.

Adding users
====
The script adduser.py takes a single argument which is the same JSON configuration
file as used by the HTTP servers. Please note you must run this script after
you have have launched standalone_webapi.py at least once. There is a small
amount of bootstrapping that has to on before users can be created.

You are prompted for the username and password of the newly created user.
Users created with this script have permission to
create invites. Creating invites, which are one time user hyperlinks,
and sending them to new users is the preferred method for adding
new users after the first user is created. Eventually, I'll get around to
creating a web interface to add and remove permissions from users.


Architecture
=======

HTTP
------
Two seperate Python instances are launched. Each hosts a single HTTP
server. One instance is the tracker, which is used by BitTorrent clients
to exchange peers. The second is the web interface, which is a RESTful API
for interacting with the private tracker. The HTML5 & JavaScript
web interface is best served by a traditional web server.

Each instance is ran behind a HTTPS server(lighttpd in my case) which
proxies requests to them.

SQL
----
The PostgreSQL server is used by both server instances.

The tracker uses the database to authorize specific torrents and users.
There is no writing to the database by the tracker. Peers are stored only in memory.
At first this seems silly, but given that there is rarely a reason to restart
the tracker it works well. If the tracker is restarted, it only takes
until all peers have announce'd to rebuild the complete list of peers. If
someone comes up with a use case where the tracker is consuming too
much memory, the intent will be to move the peer lists into a Redis
instance.

The web server uses it to allow users to login and upload new torrents.
Torrents themselves are completely stored in the database. The actual
uploaded BitTorrent files are decoded from their bencoded form, then pickled and stored in the gdbm databse. Any
extended information for a torrent is stored as a pickled object as well.
Initially, I was lead to believe this is a bad idea. I learned that PostgreSQL
implements TOAST which allows large entries to be stored outside of the row
they are part of. This mitigates the performance impact if the entry is seldom
accessed. For now this is an appropriate solution. If scalability becomes
an issue, I will move to implementing a LRU type cache in the application.

The tables needed are specified in fairywren.sql. The roles needed
are in roles.sql. The permissions for the roles are granted in permissions.sql.

Two users are used in my configuration, a read only user for the tracker
and a read-write user for the webapi. The example roles and permissions
are shown in roles.sql and permissions.sql. Obviously, a single user
with global permissions could be substituted.

Redis
----
The Redis instance is used by both servers. The tracker uses it store and retrieve
lists of peers for each torrent. The webapi uses it to retrieve peer counts on torrents.