rubycocos / csvreader

csvreader library / gem - read tabular data in the comma-separated values (csv) format the right way (uses best practices out-of-the-box with zero-configuration)

Comma-Separated Values (CSV) Tabular Data Readers Incl. CSV <3 JSON, CSV <3 YAML And More

Gem Family

csvreader - read tabular data in the comma-separated values (csv) format the right way (uses best practices out-of-the-box with zero-configuration)

csvjson - read tabular data in the CSV <3 JSON format, that is, comma-separated values CSV (line-by-line) records with javascript object notation (JSON) encoding rules

csvyaml - read tabular data in the CSV <3 YAML format, that is, comma-separated values (CSV) line-by-line records with yaml ain't markup language (YAML) encoding rules

csvhuman - read tabular data in the CSV Humanitarian eXchange Language (HXL) format, that is, comma-separated values (CSV) line-by-line records with a hashtag (meta data) line using the Humanitarian eXchange Language (HXL) rules

tabreader - read in tabular datafiles in text in the tabular (TAB) format

csvpack - tools 'n' scripts for working with tabular data packages using comma-separated values (CSV) datafiles in text with meta info (that is, schema, datatypes, ..) in datapackage.json; download, read into and query CSV datafiles with your SQL database (e.g. SQLite, PostgreSQL, ...) of choice and much more

Typed Structs & More

csvrecord - read in comma-separated values (csv) records with typed structs / schemas

(Command Line) Tools & More

csvutils - tools 'n' scripts for working with comma-separated values (csv) datafiles - the world's most popular tabular data interchange format in text

csvreader's Issues

Remove puts

Is not good to puts everything.

Remove banner

Thank you for the great gem. It is really useful, especially when combined with CsvRecord.

Incidentally, as this gem has already reached version 1, would it be possible to remove the banner? It is quite noisy on the console.

https://github.com/csvreader/csvreader/blob/bf4015721a3668d648bd3dd323191daff9f28c55/lib/csvreader/base.rb#L168

$. in Csv.foreach doesn't provide accurate file line number

$. value isn't accurate

Does not work with Ruby 3

Due to Ruby 3 changes about positional and keyword parameters we need to patch reader_hash.rb to send an exploded hash instead of relying on Ruby's 2.7 automatic conversion, here is how to test the issue:

CsvHash.strict.parse("")

Generates:

.rbenv/versions/3.0.2/lib/ruby/gems/3.0.0/gems/csvreader-1.2.4/lib/csvreader/parser_strict.rb:64:in `parse': wrong number of arguments (given 2, expected 1) (ArgumentError)

I'm adding a PR to address it as well.

Maybe further tests are required but the system I'm working on relies heavily on csvreader and our test suite passes with this simple change

[Enhancement] Allow passing options to Csv.strict.foreach

There is no convenient way to pass "strict" options such as quote to the foreach method:

#> Csv.strict.foreach(file, sep: "-", quote: "#") { |row| pp row }
ArgumentError: unknown keyword: quote

The only way I've found to do this is using the following:

#> CsvReader::Builder.new(Csv::ParserStrict.new(sep: "-", quote: "#")).foreach(file) { |row| pp row }

This works, but it is very verbose and not at all intuitive.

Returning an Enumerator?

Hi @geraldb, we use the standard library CSV quite a bit, so this effort you are making is of some interest to us.

I was wondering if and/or how this library interacts with Ruby Enumerator and Enumerable classes? For example, I noticed a non-standard #foreach method which delegates a given block for processing, but doesn't appear to have a non-block form that returns an Enumerator?

By contrast, the standard library CSV has the #each method, which does return an Enumerator when no block is given, making it extremely friendly for building lazy computational pipelines?
https://ruby-doc.org/stdlib-2.3.4/libdoc/csv/rdoc/CSV.html#method-i-each

Thanks for all your work here!
-Marc

CsvHash.foreach fails on Ruby 3.x

this works on Ruby 2.7.5, but fails on 3.0.x and 3.1.x

require 'csvreader'

CsvHash.foreach("beer.csv", {header_converters: :symbol}) do |row|
  puts row
end

csvreader v1.2.5

Platforms: Windows 10, WSL, Ubuntu, Rasbian

error message:

user@JUPITER:/mnt/d/src/Ruby/csv$ ruby csv_test.rb
/home/user/.rvm/gems/ruby-3.1.0/gems/csvreader-1.2.5/lib/csvreader/reader_hash.rb:58:in `foreach': wrong number of arguments (given 2, expected 1) (ArgumentError)
        from csv_test.rb:3:in `<main>'

Any advise what to change?

How do I get different file encodings?

C:/Ruby25-x64/lib/ruby/gems/2.5.0/gems/csvreader-1.2.4/lib/csvreader/parser_std.rb:324:in sub': invalid byte sequence in UTF-8 (ArgumentError)`

CsvReader::ParseError: found >1 (49)< - FS (,) or RS (\n) expected!!!!

Hello! First, thank you for the gem, I use it on one of my project in production for our csv upload feature.

test.txt
When I try to read attached file using CsvReader.read , I got following error:
CsvReader::ParseError: found >1 (49)< - FS (,) or RS (\n) expected!!!!

(The alphabet on the original file has been replaced by "1" to protect the content, but the error remains)
The standard ruby library CSV.read is able to read this file.

How can I handle this error?

How is this gem different from csv11?

Is csv11 the core logic, and this the configuration sugar on top?

Record, and therefore CsvRecord, not Ruby3 compatible.

Hello!

So, CsvRecord fails on any record instance creation in Ruby 3+, hitting me with an ArgumentError: wrong number of arguments (given 1, expected 0). I traced the issue to your gem Record, specifically:

--- record-1.2.0/lib/record/base.rb
+++ record-that-new-style/lib/record/base.rb
  def initialize( **kwargs )
    @values = []
-    update( **kwargs )
+    update( kwargs )
  end

  def update( kwargs )
    pp kwargs
    kwargs.each do |name,value|
      ## note: only convert/typecast string values
      if value.is_a?( String )
        send( "parse_#{name}", value )  ## note: use parse_<name> setter (for typecasting)
      else  ## use "regular" plain/classic attribute setter
        send( "#{name}=", value )
      end
    end

I'd put a PR against Record, but trouble is the repo is gone! I see that you've got something called Records but it looks substantively different. I tried to just sub one for the other naively but Records lacks a .define method.

I wouldn't mind taking a crack at resolving it, but I'd appreciate a lil guidance on what you're intending with Record and Records.

Appreciate,