How to automaticly track train/airplane tickets

Hello everyone! Today I'm gona tell you about a common situation in life:

• you need to buy a ticket for a precise date but there are no available (oh crap!)
• you know 95% for sure that some tickets will appear for sale suddenly, you want to get them!
• you have no time and patience to track site from time to time to check for those tickets
• you can write a programm that'll do this for you
• ...
• profit!

The idea is to perform repeatable GET requests for your specific demands

• From city
• To city
• Date
• e.t.c.

..after you find this "bookmarkable GET" all you need is to parse the income HTML for information you need.

So let's go!

Requirements

1. Programm must track the site for available tickets and try to find those you need
2. It's notify you fast (!) right after it'll find the appropriate ticket

Prerequisites

1. My Your favourite text editor or IDE
2. JSOUP library for HTML parsing
3. JavaMail library for sending you an email (optional)

Analyse

First thing you need to do is to find a site, which you're going to track for tickets.

I'd chosen this one because it's in Russia and have a funy name :)

Now find a bookmarkable GET request and get common with it's structure. In my case it was:

http://www.tutu.ru/poezda/rasp_d.php?nnst1=2010280&nnst2=2004000&date=19.08.2012

Here as you can see, from-city, to-city and date are configurable and easy to change.

Next, analyse the structure of the site and it's HTML layout.

Think about "what I want to track for" and create a model of it:

In this example we have the <table> of available train and tickets for them </table> with static id=schedule_table, so we can parse it and retrieve the information we need to proceed. In the table there are <tr> elements </tr> that represents trains, we can just iterate thru them. In trains there are <td> columns </td> that we can iterate just for their index, because they have static layout.

Let's assume that I want to know when new seat places will appear for sale. See the following algorithm to understand how we can identify this case.

Structure and algorithm:

So in our programm we are saying the following:

If the number of <p> elements is more than 1 that means that new places appear for sale

Ok, lets code this.

Code

What you need is a fast solution, so forget all this stuff about OOP principles, clean code or architecture :)

1. Create a new class with a main method

public class TrainDaemon {
 
 public static void main(String[] args) {
  
 }

}

2. Establish a HTTP connection

URL url = new URL("http://www.tutu.ru:80/poezda/rasp_d.php?nnst1=2010280&nnst2=2004000&date=19.08.2012");
HttpURLConnection uc = (HttpURLConnection)url.openConnection();
uc.connect();

or via proxy

Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxyhost", 12345));
URL url = new URL("http://www.tutu.ru:80/poezda/rasp_d.php?nnst1=2010280&nnst2=2004000&date=19.08.2012");
HttpURLConnection uc = (HttpURLConnection)url.openConnection(proxy);
uc.connect();

3. White a parsing solution with the help of JSOUP

StringBuffer tmp;
try {
 URL url = new URL("http://www.tutu.ru:80/poezda/rasp_d.php?nnst1=2010280&nnst2=2004000&date="+date);
 HttpURLConnection uc = (HttpURLConnection)url.openConnection(proxy);
 uc.connect();
 String line = null;
    tmp = new StringBuffer();
    BufferedReader in = new BufferedReader(new InputStreamReader(uc.getInputStream()));
    while ((line = in.readLine()) != null) {
      tmp.append(line);
    }
} catch (MalformedURLException e1) {
} catch (IOException e) {}

Document doc = Jsoup.parse(String.valueOf(tmp));

Elements table = doc.getElementsByAttributeValue("id", "schedule_table");
for (Element el : table) {
 Elements tbodys = el.getElementsByTag("tbody");
 Elements trains = tbodys.get(0).getElementsByTag("tr");
 System.out.println(trains.size() + " trains");
 for (Element train : trains) {
  int i = 0;
  for (Element td : train.getElementsByTag("td")) {
   if (i == 9) {
    Elements ps = td.getElementsByTag("p");
    System.out.println(ps.size()+" places");
    if (ps.size() > 1) return true; // TADA!
   }
   i++;
  }
 }
}
return false; // another time...

4. Notify you if hit (if TADA!)

Here is a good tutorial of how to send emails in Java or you can use any other notifier (just console log for example).

5. Put all together in a loop

// PERIOD = 60000 = 1min
long currentTime = System.currentTimeMillis() - PERIOD;
while (true) {
 while (System.currentTimeMillis() - currentTime < PERIOD) {}
 System.out.println(System.currentTimeMillis() + " attempt:");
 for (String date : DATES) {
  System.out.println("date: " + date);
  String result = checkForDate(date);
  if (result != null) {
   System.out.println(result);
   pw.println(result);
   pw.flush();
  }
 }
 System.out.println("attempt end in: " + (System.currentTimeMillis() - currentTime - PERIOD));
 currentTime = System.currentTimeMillis();
}

Result

As a result you have very bad, not optimized, but working programm that frees you from routine hand-checking tickets and automaticly notifies you if something you need just appeared.

My console log looks like this

here you can see that it just appeared 2 places for 19.08.2012
so I'm gona finish writing and go buy the tickets :)

Have a happy codding!

P.S.: You can also imrove the programm to automaticly reserve (buy) you tickets after successful hit.