Merge pull request #3 from linkedconnections/development

Rewrite

.gitignore

@@ -0,0 +1,29 @@
+logs
+*.log
+
+# Runtime data
+pids
+*.pid
+*.seed
+
+# Directory for instrumented libs generated by jscoverage/JSCover
+lib-cov
+
+# Coverage directory used by tools like istanbul
+coverage
+
+# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
+.grunt
+
+# Compiled binary addons (http://nodejs.org/api/addons.html)
+build/Release
+
+# Dependency directory
+# Deployed apps should consider commenting this line out:
+# see https://npmjs.org/doc/faq.html#Should-I-check-my-node_modules-folder-into-git
+node_modules
+
+arrivals
+dates
+departures
+stop_times

LICENSE

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2014 OpenTransport
+Copyright (c) 2015 Linked Connections contributors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -1,29 +1,40 @@
-# Guide: GTFS -> Linked Connections
+# Guide: GTFS → Linked Connections
 
-This document explains how you can transform your own GTFS feed into "[connections](https://github.com/linkedconnections/arrdep2connections#2-a-connection-object)".
-
-Note that the code in this repository is depreciated.
-
-## Introduction
+Transforms a GTFS file towards a directed acyclic graph of "connections".
 
 A connection is the combination of a departure and its successive arrival of the same trip. 
 Our goal is to retrieve a list of connections that is sorted by departure time, better known as a Directed Acyclic Graph. This way, routeplanning algorithms can be performed.
 
-First, we calculate two sorted lists: one for arrivals and one for departures.
-After that we can easily calculate connections out of these lists.
+## Use it
+
+First, __unzip__ your GTFS file to a place on your disk using e.g., `unzip gtfs.zip /tmp`
+
+Now, we need to make sure that a couple of files are ordered in a specific fashion, not required by the GTFS spec. You can do these orderings through bash as follows:
+ * __stop_times.txt__ must be ordered by `trip_id` and `stop_sequence`. You can do this using this command: `{ head -n 1 stop_times.txt ; tail -n +2 stop_times.txt | sort; } > stop_times2.txt ; mv stop_times2.txt stop_times.txt`. Mind that the number of the columns are also not standardized by GTFS and you might need to tweak the sort command.
+ * __trips.txt__ must be ordered by `trip_id`.  You can do this using this command: `{ head -n 1 trips.txt ; tail -n +2 trips.txt | sort; } > trips2.txt ; mv trips2.txt trips.txt`.
+ * __calendar.txt__ must be ordered by `service_id`.  You can do this using this command: `{ head -n 1 calendar.txt ; tail -n +2 calendar.txt | sort; } > calendar2.txt ; mv calendar2.txt calendar.txt`.
+ * __calendar_dates.txt__ must be ordered by `service_id`.  You can do this using this command: `{ head -n 1 calendar_dates.txt ; tail -n +2 calendar_dates.txt | sort; } > calendar_dates2.txt ; mv calendar_dates2.txt calendar_dates.txt`.
+
+If you've ensured this, you can install this library using: `npm install -g gtfs2lc` and use it as follows:
+
+```bash
+gtfs2lc /path/to/extracted/gtfs
+```
+
+## How it works (for contributors)
+
+We convert `stop_times.txt` to a stream of connection rules. These rules need a certain explanation about on which days they are running, which can be retrieved using the `trip_id` in the connection rules stream.
+
+At the same time, we process `calendar_dates.txt` and `calendar.txt` towards a binary format. It will contain a 1 for the number of days from a start date for which the service id is true.
+
+In the final step, the connection rules are expanded towards connections by joining the days, service ids and rules.
 
-### Step 1: Calculate arrivals/departures
-The code and instructions are available at:
-https://github.com/brechtvdv/gtfs2arrdep
+## Not yet implemented:
 
-This process can take up a lot of memory when there are thousands arrivals/departures in a day.
-It is recommended to test your dataset by setting a startDate and endDate in step 3.
+At this moment we've only implemented a conversion from the Stop Times to connections. However, in future work we will also implement a system for describing trips and routes, a system for transit stops and a system for transfers.
 
-This generates JSON-LD streams of arrivals and departures that you use in step 2.
+Furthermore, also `frequencies.txt` is not supported at this time. We hope to support this in the future though.
 
-### Step 2: Calculate connections
-The code and instructions are available at:
-https://github.com/linkedconnections/arrdep2connections
+## Authors
 
-### Done
-With the retrieved JSON-LD stream of connections you can now setup your own [server](https://github.com/linkedconnections/server.js) and start experimenting with the [client](https://github.com/linkedconnections/client.js) and [Connection Scanning Algorithm](https://github.com/linkedconnections/csa.js).
+Pieter Colpaert - [email protected]

bin/gtfs2lc.js

@@ -0,0 +1,22 @@
+#!/usr/bin/env node
+
+var program = require('commander'),
+    Mapper = require('../lib/gtfs2lc.js');
+
+console.error("GTFS to linked connections converter use --help to discover more functions");
+
+program
+    .version('0.1.0')
+    .option('-p, --path <path>', 'Path to sorted GTFS files (default: ./)')
+    .parse(process.argv);
+
+if (!program.path) {
+    program.path = './'
+}
+
+var mapper = new Mapper();
+mapper.resultStream(program.path, function (stream) {
+  stream.on('data', function (connection) {
+    console.log(JSON.stringify(connection));
+  });
+});