Return Ajax request to R (shiny) about dt

[yihui]

I'm not sure I completely understand your question, but please read Section 2 here if you have not: http://rstudio.github.io/DT/server.html Basically you can rewrite this function https://github.com/rstudio/DT/blob/78c9a30cc30e/R/shiny.R#L137-L226 in whatever language or using any R packages you want. For example, the data argument can be a database connection, and you can perform sorting in the database instead of using sort() in R.

[kismsu]

Got it, this is exactly what I need. Rewrite function datatableJSON with tbl_src input from dplyr package.

[yihui]

Please let me know if you make any progress on it, since I'm interested in alternative implementations (hopefully faster). My current implementation is pretty mundane.

[kismsu]

Hi, I've spent half of the day trying to make DT work for all dplyr::tbl inputs (including database tables) and hit an obstacle. Your filterRow function calculates min and max to send it to slider filter. However, for database table you don't want to calculate ranges for all columns in advance, as for large table it might be slow. I've looked at the code and I have to be able to turn off your nice filters and maybe use the old version (as in shiny package).

[yihui]

Did you time the calculation of ranges? I guess if this step is slow enough, the table will not be very responsive, either, when you sort or search the table.

[kismsu]

I've tested sorting on 3.5 million rows table. It takes about 5-7 sec to response. Tried to get range in SQL, about 2 sec per column. I have around 30 columns in my table, might be too long.

[yihui]

In that case, I guess sorting and searching will also be very slow.

I will think about turning off the sliders and select inputs in column filters, and just use plain text inputs instead. Users will have to type in the text inputs.

[StevenMMortimer]

I have a large datatable (~1M rows) in a Shiny app with filters for the user that update a plot based on those filters just like this example: https://yihui.shinyapps.io/DT-info/

The problem is that the application hangs during filtering, I think because the entire table is on the client-side, so I moved to using server-side processing like this example (http://rstudio.github.io/DT/server.html). It's much faster, but then I can't retrieve the filtered row indices using input$tbl_rows_current because the client side only knows about the records that have been served and not the indices relative to the entire table. Simple example: https://gist.github.com/mortimes3/a0619adbe80fcf5cbacb

In short, I've tried blending the server-side processing example with the Shiny datatables information example and I've got good filtering performance, but can't access the row indices I've filtered.

Is it possible to have something like input$tbl_rows_current available when server-side processing is enabled? Or is something easily possible with drawCallback?

[yihui]

@ReportMort Thanks for the report. I'm aware of this problem, and I have a vague solution in my mind (using the first column, i.e. row names, instead of row indices). I'll see if I can solve the problem next week.