paulspringett / csv_shaper Goto Github PK
View Code? Open in Web Editor NEWBeautiful DSL for creating CSV output in Ruby & Rails
License: MIT License
Beautiful DSL for creating CSV output in Ruby & Rails
License: MIT License
It's ideal for converting database backed models with attrbiutes into CSV output.
Should be attributes
csv_shaper is a nice library/DSL, but performance about an order of magnitude slower than creating my own CSV. My simple test with about 5,000 rows in the resulting CSV file took 6 seconds creating the CSV manually as below, and took 70 seconds using csv_shaper! I used the same code to generate the @Entities result set which was < 1 second. All remaining time was in rendering.
my controller method
def entities
@entities = Entity.all.includes(:state)
@headers = ['ID', 'State' ,'Name']
@data = []
@entities.each do |e|
@data << [e.id, e.state.short_name, e.name]
end
content = CSV.generate { |csv| csv << @headers; @data.map { |x| csv << x }; }
...
end
csv.headers :id, :state_id, :name
csv.rows @Entities do |csv, entity|
csv.cells :id, :name
csv.cell :state_id, entity.state.short_name
end
I'm sure I've got this set up correct, but is not returning an actual CSV file for download - what am I missing? I can inspect the csv_string and it has the values I expect in there, just the render csv command seems to be doing nothing?
csv_string = CsvShaper.encode do |csv|
csv.headers :user_id, :total_leave_requested
csv.rows @results do |csv, result|
csv.cell :user_id, result.user_id
csv.cell :total_leave_requested, result.total_leave_requested
end
end
render csv: csv_string
Hi there, this seems like a pretty likely scenario:
csv.headers do |csv|
csv.columns :state, :building_name
end
csv.rows @warranties do |csv, warranty|
csv.cells :state
if warranty.building
csv.cell :building_name, warranty.building.name
end
end
At the moment I get a undefined method
building_name' for #Warranty:0x007fbc0890ece8` error.
I guess it's because the code is calling the building_name
method on the warranty
even though we're not going to be using that method to get the actual value?
I have a user case for .csv
footers. I'd be willing to try to work up a PR for this if it's something you think other might be interested in.
Have you considered this before?
setting @filename didn't work for me as specified in the doco.
I'm in a bit of a hurry so I just set the response header myself and didn't look into it further - just thought you would like to know.
I was generating a CSV of about 300,000 rows and seeing a lot of memory usage.
Probably the best approach is to support 'streaming' (writing one row at a time) for large files, rather than generating the whole document in memory and then writing it out. I may have a crack at this when I get time, and send a pull request.
In the meantime I was in a hurry, so I made a very hacky monkey-patch to reduce the memory footprint. It is pretty nasty, but I thought you might be interested.
module CsvShaper
class Encoder
# Monkey Patch to reduce memory usage while generating the csv.
# - CSV::Rows are converted to a string immediately as they have a large footprint
# - single map! rather than nested calls to map (note: this has nasty side effect of making this instance un-reusable)
# This is a quick and nasty solution. A better approach would be to write out the rows as they are generated
# rather than build the whole CSV document in memory.
def to_csv(local_config = nil)
csv_options = options.merge(local_options(local_config))
cols = @header.mapped_columns
@rows.map! do |row|
padded_row = CSV::Row.new(row.cells.keys, row.cells.values).values_at(*@header.columns)
CSV::Row.new(cols, padded_row, false).to_csv
end
cols.join(",") + "\n" + @rows.join
end
end
end
Hi,
What do you think abut using @csv_filename instead of @filename which is very generic and can be confusing ?
Currently getting a lot of noise in PRs from existing code
Unless I'm completely blind, I cannot seem to find a way to render partials from a csv.shaper template. A simple example would have the parent template containing a line like
render "invoices/csv/calls_summary"
with a file in that directory called calls_summary.csv.shaper
. However when I try to render, I get the error message
you must define some headers using csv.headers ...
Am I missing or misunderstanding something?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.