alexanderkrutov / datatables.queryable

.Net library for clever processing of requests from datatables.net jQuery plugin on the server side (ASP.NET, Nancy or any other web server).

DataTables.Queryable

What is it?

It is a .Net library for clever processing of requests from datatables.net jQuery plugin on the server side (ASP.NET, Nancy or any other web server).

The library significantly reduces boilerplate code and helps to avoid writing same logic of parsing requests for different model types.

Installation

PM> Install-Package DataTables.Queryable

How to use it?

// ASP.NET action handler inside a controller:
public JsonResult DataTablesRequestAction()
{
    // make a DataTablesRequest object from the incoming Http query string
    var request = new DataTablesRequest<Person>(Request.QueryString);
    
    using (var ctx = new DatabaseContext())
    {
        // take persons from database, apply the DataTablesRequest filter and paginate
        var persons = ctx.Persons.ToPagedList(request);
     
        // push back a result in JSON form applicable for datatables.net
        return JsonDataTable(persons);
    }
}

Need more info? Welcome to the wiki.

How it works?

1. DataTables.Queryable parses data from an incoming Http request and extracts related parameters (search text, columns ordering info, page number and number of records per page and etc.);
2. Dynamically builds an expression tree from the request parameters and information about model type T using reflection;
3. Filters provided IQueryable<T> with the expression tree.
4. Returns query result as a PagedList<T> instance (collection of items that represents a single page of extracted data).

Take a closer look what happens inside.

Features

• Global search, individual column search, ordering by one or many columns, pagination
• Custom request parameters
• Custom search predicates
• Supports nested properties
• Lazy data loading from provided IQueryable<T> (only filtered records will be extracted)
• Compatible with Entity Framework

License

DataTables.Queryable is licensed under MIT license.

datatables.queryable's Issues

How to use with Razor Pages

Hi. Great library! Can it be used with Core Razor Pages? Any example? Thanks.

How to add (Edit &Delete) buttons in datatable?

JsonDataTable not available?

I searched through the repo and I couldn't find JsonDataTable method. any ideas?

Add custom Data Fields to DataTablesAjaxPostModel

I'm having to use the DataTablesAjaxPostModel in my implementation because there are too many columns and the "GET" query is too long. However, this method does not allow for passing back custom data points in the AJAX call as can be done with the GET function. Would be nice functionality to have those included in some ways if possible.

I think `name` should try to be parsed BEFORE `data`. Here is why:

https://github.com/AlexanderKrutov/DataTables.Queryable/blob/master/DataTables.Queryable/DataTablesRequest.cs#L173-L199

Because of the way DataTables.Queryable (DTQ) works with EF models directly, we usually AutoMapper back these results. This can lead to the return ViewModel being very different than the Entity.

Right now, jQuery DataTables uses columnDefs.data to try and traverse the actual data it gets back from the server. But name is different.

I want to be able to do:

{
    columnDefs:
    [
        {
            name: "Entity.ActualNavigationProperty"
            data: "ReturnViewModel.Property" 
        }
    ]
}

Hopefully this is clear. name is used so when DTQ tries to traverse the actual properties to search and configure custom column searchs and all that and then data is used by jQuery DT to traverse the returned viewmodel.

Ordering on a complex type does not work.

When I try and order on a column with a data definition of a complex type, my server throws the following error:

DevConf [11:59:10 Warning] Microsoft.EntityFrameworkCore.Query
The LINQ expression 'orderby IIF((Convert(Property([dto], "ManagedById"), Object) != null), new ContactViewModel() {Id = [dto.ManagedBy].Id, Name = [dto.ManagedBy].Name, EmailAddress = [dto.ManagedBy].EmailAddress, IsTeam = [dto.ManagedBy].IsTeam}, null) desc' could not be translated and will be evaluated locally.

DevConf [11:59:11 Error] Microsoft.EntityFrameworkCore.Query
An exception occurred in the database while iterating the results of a query for context type 'WuitCMDB.Infrastructure.MyDbContext'.

This seems to make sense to me, the query is evaluated locally because of my complex type, and my complex type does not implement IQueryable.

What I think needs to happen, is a possible overload in the library so we can specify a specific property to order against when a complex type is found. IE: ContactViewModel -> ContactViewModel.Name?

I am not entirely sure the best way to fix this. I really wish this would not evaluate locally though.

Column with no data

I'm trying to render a column with no data coming from Database.
Currently I had to set columDefs data AND name to null, then modified DataTablesRequest.cs at row 174 I skip the exception if propertyName is null and both data and name are empty.

Is there an easier way to skip table association for fields without modifying the code, or you want me to pull request this little change.

Can we use/call SQL store procedure?

Support for Multiple Filter Values on Same Column

DataTables allows a regex search joining values on "|" to filter on a column for multiple values.

Documented here:
https://datatables.net/forums/discussion/6946/multiple-filters-on-same-column

Currently, the ability to join search criteria with a "|" does not produce the expected results.

AutoMapper ProjectTo() blows up.

System.InvalidCastException: Unable to cast object of type 'Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[MyApp.API.ViewModels.InventoryServerViewModel]' to type 'System.Linq.IQueryable`1[MyApp.Entities.MyInventoryServer]'.
   at DataTables.Queryable.DataTablesQueryProvider`1.CreateQuery(Expression expression)
   at AutoMapper.QueryableExtensions.ProjectionExpression.Select(IQueryable source, LambdaExpression lambda) in C:\projects\automapper\src\AutoMapper\QueryableExtensions\ProjectionExpression.cs:line 89
   at System.Linq.Enumerable.Aggregate[TSource,TAccumulate](IEnumerable`1 source, TAccumulate seed, Func`3 func)
   at AutoMapper.QueryableExtensions.ProjectionExpression.To[TResult](Object parameters, Expression`1[] membersToExpand) in C:\projects\automapper\src\AutoMapper\QueryableExtensions\ProjectionExpression.cs:line 62
   at AutoMapper.QueryableExtensions.Extensions.ProjectTo[TDestination](IQueryable source, IConfigurationProvider configuration, Object parameters, Expression`1[] membersToExpand) in C:\projects\automapper\src\AutoMapper\QueryableExtensions\Extensions.cs:line 60
   at AutoMapper.QueryableExtensions.Extensions.ProjectTo[TDestination](IQueryable source, IConfigurationProvider configuration, Expression`1[] membersToExpand) in C:\projects\automapper\src\AutoMapper\QueryableExtensions\Extensions.cs:line 76

http://docs.automapper.org/en/stable/Queryable-Extensions.html

https://github.com/AlexanderKrutov/DataTables.Queryable/blob/master/DataTables.Queryable/DataTablesQueryProvider.cs#L17-L25

My code:

            return queryable
               .Filter(request)
            .ProjectTo<TReturn>(mapperConfig)

I think ProjectTo is invoking CreateQuery because under the hood it runs .Select(). I am not sure, I am a little out of my experience zone. Maybe we need to be doing AsQueryable() instead of casing in the QueryProvider?

NullReferenceException inside GetPropertyByName

https://github.com/AlexanderKrutov/DataTables.Queryable/blob/master/DataTables.Queryable/DataTablesRequest.cs#L254

The bug is here type.GetProperty(parts[0]).PropertyType. GetProperty returns null, so this will explode and is never caught. Obviously this was hard to catch because it only happens when you are accessing a nested property.

.net core 2.0 JsonDataTable ?

returning JsonDataTable doesn't work in .net core any workaround?

"paging" option causes exception

If paging option is enabled for DataTable, servere-side code throws an ArgumentException:

Limit must have a non-negative value.

It happens because of takeCount parameter. Value of the takeCount parameter obtained from pageSize DataTables query parameter which is negative (i.e. -1) when pagination is disabled.

queryable.Skip(skipCount).Take(takeCount).ToList()

How to fix: handle negative values of pageSize query parameter.

Multi column ordering does not work?

When I shift click columns on the samples they do not multi-order. Specifically click Name, and then shift click Position twice so name is ASC and Position is DESC.

Support passing up CancellationTokens to ToPagedListAsync().

ASPNET Core automatically wires up a CancellationToken in your controller actions, so it would be really helpful if we could add ToListAsync() to the paging stuff.

MVC will automatically bind any CancellationToken parameters in an action method to the HttpContext.RequestAborted token, using the CancellationTokenModelBinder. This model binder is registered automatically when you call services.AddMvc() (or services.AddMvcCore()) in Startup.ConfigureServices().
via https://andrewlock.net/using-cancellationtokens-in-asp-net-core-mvc-controllers/

We would put it here and pass it all the way up to the calling code https://github.com/AlexanderKrutov/DataTables.Queryable/blob/master/DataTables.Queryable/PagedList.cs#L78 and this way we could avoid doing Task.Factory stuff like here https://github.com/AlexanderKrutov/DataTables.Queryable/blob/master/DataTables.Queryable/QueryableExtensions.cs#L78

Error while global search. There is not null check

Hi

This is truly a wonderful lib.

I am unable to resolve a big issue that is if the value is null there is a search error,

Before this
Expression.Call(propertyExp, Object_ToString);

there should be an null check

and later instead of this
Expression.Lambda<Func<T, bool>>(containsMethodExp, parameterExp);

Something like this

var andExp = Expression.And(nullChk, containsMethodExp);
return Expression.Lambda<Func<T, bool>>(andExp, parameterExp);

Needed help to do a null chk expression,
Scratching head from 2 days.

ToPagedListAsync Call is ambiguous

dotnet 8 I started getting an error with this:

Any chance we can update the code?

ToNameValueCollection() throws when a column is provided but a search is not.

If you use the DataTablesAjaxPostModel the ToNameValueCollection forces you to provide a search object, even if you just want to order and not search.

Better error handling when the request tries to reflect to the desired type.

https://github.com/AlexanderKrutov/DataTables.Queryable/blob/master/DataTables.Queryable/DataTablesRequest.cs#L214-L226

Just spent a while debugging a simple issue. My viewmodel I wanted to map my request to did not have the correct property.

It would be nice to have a friendly error "Could not find a property called 'Address.Street' on Type Person."

Pulling these straight from the request instead of accepting a DTO?

It would be nice to accept the following maybe in addition to using the request in its "raw" form. The main reason for this is that Swashbuckle cannot tell swagger about the parameter. Also I think this would allow for better overriding and custom logic and stuff.

public class DataTableAjaxPostModel
{
    public int draw { get; set; }
    public int start { get; set; }
    public int length { get; set; }
    public List<Column> columns { get; set; }
    public Search search { get; set; }
    public List<Order> order { get; set; }
}
 
public class Column
{
    public string data { get; set; }
    public string name { get; set; }
    public bool searchable { get; set; }
    public bool orderable { get; set; }
    public Search search { get; set; }
}
 
public class Search
{
    public string value { get; set; }
    public string regex { get; set; }
}
 
public class Order
{
    public int column { get; set; }
    public string dir { get; set; }
}

Taken from: https://datatables.net/forums/discussion/40690/sample-implementation-of-serverside-processing-in-c-mvc-ef-with-paging-sorting-searching

Can we get an `ToPagedListAsync`?

Unused code?

https://github.com/AlexanderKrutov/DataTables.Queryable/blob/master/DataTables.Queryable/DataTablesRequest.cs#L129

That line never looks used to me.

In the sample, if I type "Mar" it finds start dates but "March" does not. Why is this?

VS

Can we get a version that targets the new netstandard2.0?

Complex Models

Hi,
I'm trying to access a complex model with entity framework and in my columnDefs I would have something like this:

{ targets: 0, data: 'event.code' }

Basically my EF model contains relations to other tables which results in a complex model.
How to access properies of associated objects?

NullReferenceException inside BuildStringContainsPredicate

https://github.com/AlexanderKrutov/DataTables.Queryable/blob/master/DataTables.Queryable/QueryableExtensions.cs#L349

If the option caseInsensitive is set for columns, you do a ToLower call on the object string. If the object is null, both expressions which are build up on line 354 and 355 will call ToLower first. So you get a NullPointer before you check for not null.

Allow ToPagedListAsync to take cancellation token

ToPagedListAsync

I really need to know what name/data failed when calling GetPropertyByName.

https://github.com/AlexanderKrutov/DataTables.Queryable/blob/master/DataTables.Queryable/DataTablesRequest.cs#L183

This might be a good argument for a custom exception (GetPropertyByStringException?), or pass the name to ArgumentException.

I am try/catching the instantiation of DataTablesRequest so I can gracefully handle any invalid column configurations instead of my server blowing up with a 500 error. I think other people will want to do this too.

As it stands right now, I am unable to identify and report the "bad" column type.

Usage

// ...
throw new DataTablesGetPropertyByStringException(type, name);


// Later in a catch(DataTablesGetPropertyByStringException ex) ...

return BadRequest($"Sorry, the property {ex.name} did not exist on {ex.type}");

Using the library with in-memory objects

Hi,

I have a model with some calculated fields that must be searched but since they do not exist in the schema naturaly had a System.NotSupportedException: 'The specified type member 'FieldName' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.' thrown at

DataTables.Queryable/DataTables.Queryable/DataTablesQueryable.cs

Line 84 in 80398b1

return sourceQueryable.ToString();

I am aware of the purpose of this library, the intention is relying on data context.

So my question is, what would be the path to use the library with in-memory objects e.g. List<MyMode> ? My dataset is not too big so it won't hurt. Any tips?