sangster / gtfs_reader

GTFS Reader is a gem designed to help process the contents of a "GTFS Feed"

Personal Project Stats

gtfs_reader's Issues

CSV formating error

Gtfs_reader produces that error:
Encoding::UndefinedConversionError: "\xEF" from ASCII-8BIT to UTF-8
from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:994:in `encode' ...

Im not sure why this happends.
My code is similar to the example:

GtfsReader.config do
  return_hashes true
  sources do
    madrid do
      url "https://servicios.emtmadrid.es:8443/gtfs/transitemt.zip"
      before { |etag| puts "Processing source with tag #{etag}..." }
      handlers do
        #Data Exports To DB code must be here.
        stops  {|row| puts row }#createStation(row)}
        calendar {|row| puts row }#createService(row)}
        #calendar_dates {|row| createService(row)} <= This services are special
        trips {|row| puts row }#createtrip(row)}
        stop_times {|row| puts row }#createStoptime(row)}
      end
    end

  end
end
GtfsReader.update :madrid

Complete Error:

Processing source with tag false...
Encoding::UndefinedConversionError: "\xEF" from ASCII-8BIT to UTF-8
    from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:994:in `encode'
    from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:994:in `block in <class:CSV>'
    from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:2168:in `[]'
    from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:2168:in `block (2 levels) in convert_fields'
    from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:2166:in `each'
    from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:2166:in `block in convert_fields'
    from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:2165:in `map'
    from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:2165:in `with_index'
    from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:2165:in `convert_fields'
    from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:2206:in `parse_headers'
    from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:1882:in `block in shift'
    from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:1774:in `loop'
    from /home/vagrant/.rvm/rubies/ruby-2.1.0-preview1/lib/ruby/2.1.0/csv.rb:1774:in `shift'
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/gtfs-reader-0.2.8/lib/gtfs_reader/file_reader.rb:27:in `initialize'
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/gtfs-reader-0.2.8/lib/gtfs_reader/source_updater.rb:62:in `new'
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/gtfs-reader-0.2.8/lib/gtfs_reader/source_updater.rb:62:in `block (2 levels) in check_columns'
... 6 levels...
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/gtfs-reader-0.2.8/lib/gtfs_reader/source_updater.rb:60:in `check_columns'
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/gtfs-reader-0.2.8/lib/gtfs_reader/core.rb:52:in `block in update_verbosely'
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/gtfs-reader-0.2.8/lib/gtfs_reader/core.rb:47:in `instance_exec'
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/gtfs-reader-0.2.8/lib/gtfs_reader/core.rb:47:in `update_verbosely'
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/gtfs-reader-0.2.8/lib/gtfs_reader/core.rb:36:in `block in update'
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/gtfs-reader-0.2.8/lib/gtfs_reader/log.rb:44:in `quiet'
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/gtfs-reader-0.2.8/lib/gtfs_reader/core.rb:36:in `update'
    from /vagrant/transit/lib/gtfs_retriever.rb:26:in `updateData'
    from (irb):12
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/railties-4.1.1/lib/rails/commands/console.rb:90:in `start'
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/railties-4.1.1/lib/rails/commands/console.rb:9:in `start'
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/railties-4.1.1/lib/rails/commands/commands_tasks.rb:69:in `console'
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/railties-4.1.1/lib/rails/commands/commands_tasks.rb:40:in `run_command!'
    from /home/vagrant/.rvm/gems/ruby-2.1.0-preview1/gems/railties-4.1.1/lib/rails/commands.rb:17:in `<top (required)>'
    from bin/rails:4:in `require'
    from bin/rails:4:in `<main>'

source_updater combines business and view logic

The file source_updater.rb combines both logic for downloading and parsing zip files and the logic for pretty-printing information about the process on the console. These two aspects should be split into two classes.

[Bug] NameError: undefined local variable or method `parent_station'

May be related to #4, but the GTFS feed for the Nashville MTA throws an exception when attempting to parse it.

NameError: undefined local variable or method `parent_station' for #<GtfsReader::ParserContext:0x007f993f8a40c8>
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/file_row.rb:63:in `method_missing'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/config/defaults/gtfs_feed_definition.rb:38:in `block (4 levels) in <module:Defaults>'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/file_row.rb:31:in `instance_exec'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/file_row.rb:31:in `[]'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/file_row.rb:45:in `block in to_hash'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/file_row.rb:45:in `each'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/file_row.rb:45:in `inject'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/file_row.rb:45:in `to_hash'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/file_reader.rb:45:in `each'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/feed_handler.rb:16:in `handle_file'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/source_updater.rb:176:in `process_from_temp_file'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/source_updater.rb:78:in `block in process_files'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/source_updater.rb:76:in `each'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/source_updater.rb:76:in `process_files'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/core.rb:54:in `block in update_verbosely'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/core.rb:48:in `instance_exec'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/core.rb:48:in `update_verbosely'
  /Users/yeargism/Sites/wego-bus-map/vendor/bundle/ruby/2.4.0/gems/gtfs_reader-1.2.0/lib/gtfs_reader/core.rb:34:in `update'
  bin/updateGtfsData:47:in `<top (required)>'

The file in question has null values for parent_station.

New gem release needed for Ruby 3 compatibility

Hello!

I noticed #17 got merged 🙇, however it seems the latest version of the gem still doesn't include the change.

I didn't notice it before because I was using either a manual fix or my own server.

Are there any plans to release a new one?

Thank you!