vapor-community / pagination

Simple Vapor 3 Pagination

Vapor Pagination

Pagination is based off of the Fluent 2 pagination system.

Getting Started

Add this to your Package.swift file

.package(url: "https://github.com/vapor-community/pagination.git", from: "1.0.0")

Conform your model to Paginatable

extension MyModel: Paginatable { }

Once you have done that, it's as simple as returning your query in paginated format.

func test(_ req: Request) throws -> Future<Paginated<MyModel>> {
    return try MyModel.query(on: req).paginate(for: req)
}

Even return items off of the query builder

func test(_ req: Request) throws -> Future<Paginated<MyModel>> {
    return try MyModel.query(on: req).filter(\MyModel.name == "Test").paginate(for: req)
}

Making a request with the parameters is easy is appending ?page= and/or ?per=

curl "http://localhost:8080/api/v1/models?page=1&per=10"

A response looks like this

{
  "data": [{
    "updatedAt": "2018-03-07T00:00:00Z",
    "createdAt": "2018-03-07T00:00:00Z",
    "name": "My Test Model"
  }],
  "page": {
    "position": {
      "current": 1,
      "max": 1
    },
    "data": {
      "per": 10,
      "total": 2
    }
  }
}

pagination's Issues

Fluent filters are missing

With the current API I can't do something like User.query(on: req).filter(\.email == searchTerm).paginate(for: req) in a searchendpoint because paginate(for: req) call the all() method.

My proposal is that Pagination package handle only the split and page build without querying stuff, more like an Array extension maybe 🤔 , in that way you can access to the full fluent API and do something like this:

func index(_ req: Request) throws -> Future<Paginated<User>> {
    return User.query(on: req).all().paginate(for: req)
}

func search(_ req: Request) throws -> Future<Paginated<User>> {
    guard let searchTerm = req.query[String.self, at: "email"] else {
        throw Abort(.badRequest)
    }
        
    return try User.query(on: req).filter(\.email == searchTerm).paginate(for: req)
}

Update documentation for size use

The size adjustment of the request using ?size= is not working.

Performing tests the correct argument to use is ?per= to actually adjust the size of the returning data array.

Pagination fails when sorting

I'm trying to make a query with pagination, filtering and sorting. My query looks like this:

try Space.query(on: req).filter(\.isFeatured == true).sort(\.updatedAt, .descending)
    .paginate(for: req).flatMap { (paginated) in
        try paginated.data.relationals(req: req)
}

It worked fine before adding the .sort(...) call. But after, I get this message:

column "spaces.createdAt" must appear in the GROUP BY clause or be used in an aggregate function

What do I need to do? (btw I'm using PSQL)

_{perhaps related to #20}

Always return 1 as the max number of pages

When running a request using pagination, it always return 1 as the maximum number of pages.
I have located the issue to originated in line 33 in Paginated.swift, where the number of pages i calculated;

   let count = Int(ceil(Double(self.total) / Double(self.size)))

It always return 1 since self.total returns the number of entries at the given page, not the total number of returned entries. Testing with a page without any entries, it returns 0 as max page number.

I ran a test with 13 entries in total and 5 entries per page. This should return a total of 3 pages. The following the the meta data from page 2:

// testing for page 2 of 3
"page": {
        "position": {
            "current": 2, //"next" is missing, which is related to the issue with counting the maximum number of pages
            "previous": 1,
            "max": 1   //Return 1 even though there are 3 pages
        },
        "data": {
            "per": 5,
            "total": 5
        }
    }
// Testing for page 4, which is empty
{
    "data": [],
    "page": {
        "position": {
            "current": 4,
            "previous": 3,
            "max": 0
        },
        "data": {
            "per": 5,
            "total": 0
        }
    }
}

The test is run with the following versions
Vapor 3.0.2
Fluent 3.0.0-rc.2.4.1
Pagination 1.0.0 and Pagination 1.0.1

Documentation

For the 2.0.0 release, full documentation should be a must

• Basic demo similar to the Readme file
• Custom responses with the new mapping and custom structs

Paginate non model objects

Is it possible to simply paginate an array of non Model objects?
If no, then why is pagination only allowed on database queries?

🙃

Support for `join`/`alsoDecode`

Motivation Behind Feature

At this moment, unless I’m mistaken, Pagination needs to operate on a QueryBuilder that produces Future<[Model]>. That makes it unable to paginate results that also fetch data from a different table, by using the Fluent’s join(_:to:)/alsoDecode(_:) methods, as these change the eventual output of QueryBuilder to Future<[(ModelA, ModelB)]>.

Feature Description

It would be awesome if we could paginate even these slightly more complicated queries. I get a feeling that the eventual implementation would have to also rely on an implementation of #11 that would allow us to specify how should the end result be joined for a valid JSON representation.

Alternatives or Workarounds

None considered or thought of.

Order of paginated results

Paginated results seem to be outputted in descending order.
I have a collection of 100 posts. Page 1 includes 10 posts going from post #100 to post #91.
The function looks like this:

func test(_ req: Request) throws -> Future<Paginated<Post>> {
    return try Post.query(on: req).paginate(for: req)
}

Is this expected behavior?

Transform Closures (Or something like that)

Motivation Behind Feature

Now, convert the results to other structs/public responses is not posible, for example, a common case is the User model which have sensitive data like password, confirmation token, even more, but that fields can't be returned in the response, so we need to create a PublicUser struct to exclude that fields

Feature Description
Maybe a solution is to pass some kind of transform closure which can be used in the pagination process

Different sorting for different endpoint

How to sort with different column for different endpoint in controller?

Incorrect page total when filtering db and adding pagination

Describe the bug

Hello!

I'm experiencing an issue with the pagination.

As example when I have code like this

...
.paginate(on: request, response: APIPaginated.self) { builder in
            if let fromDate = fromDate {
                return builder
                .filter(\.userId == userId)
                .filter(\.closed == false)
                .filter(\.createdAt >= fromDate)
                .sort(\.createdAt)
                .all()
            } else {
                return builder
                .filter(\.userId == userId)
                .filter(\.closed == false)
                .sort(\.createdAt)
                .all()
            }
        }

Things work but the total is the wrong total and not based on the filtered total.

Steps To Reproduce
Steps to reproduce the behavior:

1. make a custom paginated response
2. paginate your data with filters
3. notice the total is wrong

Expected behavior
I expect that when I query and filter the db the total would reflect the total after filtering

Version Information

• OS: macOS
• Vapor Version: 3.3.1
• Fluent Version: 3.2.1

Public vars?

any reason not to make the vars public?

public struct Position: Content {
    var current: Int
    var next: Int?
    var previous: Int?
    var max: Int
}
public struct PageData: Content {
    var per: Int
    var total: Int
}
public struct PageInfo: Content {
    var position: Position
    var data: PageData
}
public struct Paginated<M: Content>: Content {
    var page: PageInfo
    var data: [M]
}

Get all per page

Motivation Behind Feature
Sometimes, a frontend might want to get all items on a single page.

Feature Description
I'd like to pass a special value for the per parameter.
This could be ?per=all, or ?per=-1, although I'd prefer the all version b/c it's more legible.

Alternatives or Workarounds
Something that'd work rn would be to pass ?per=999999, but that doesn't really look so elegant to me 🙂

Support database connection

It should be possible to use pagination with a pooled connection (as this is required on some hosts, e.g. heroku):

func index(_ request: Request) throws -> Future<Paginated<Item>> {
        return request.withPooledConnection(to: .psql) { connection in
            return try Item.query(on: connection).paginate(for: request)
        }
    }

It would be nice to maybe have an optional parameter connection: Database that can be passe to the paginate function.

Is there a way to customize the response format?

Can I customize the response to include the URLs for the next pages and also have a meta section? Similar to this:

{
  "meta": {
    "total-pages": 13
  },
  "data": [
    {
      "type": "articles",
      "id": "3",
      "attributes": {
        "title": "JSON API paints my bikeshed!",
        "body": "The shortest article. Ever.",
        "created": "2015-05-22T14:56:29.000Z",
        "updated": "2015-05-22T14:56:28.000Z"
      }
    }
  ],
  "links": {
    "self": "http://example.com/articles?page[number]=3&page[size]=1",
    "first": "http://example.com/articles?page[number]=1&page[size]=1",
    "prev": "http://example.com/articles?page[number]=2&page[size]=1",
    "next": "http://example.com/articles?page[number]=4&page[size]=1",
    "last": "http://example.com/articles?page[number]=13&page[size]=1"
  }
}

License information

Question Subject

Do you think it would be possible to add license information to the repository? Preferably in the form of a LICENSE file, similar to how all official Vapor repositories do it.

Thank you!

The pagination fails to work with groupBy and sort

Describe the bug

i'm try to using sort inside my queries and use with it Pagination :
func getNews(_ req: Request) throws -> Future<Paginated<Article>> { return try Article.query(on: req).sort(\Article.publishedAt).paginate(for: req) }
But it's fails to return any data and getting this error:
{ "error": true, "reason": "column \"Article.publishedAt\" must appear in the GROUP BY clause or be used in an aggregate function" }