Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Keyset/seek pagination and filter by search term

Using table products(id, title, price, category_id)

How to filter product by title, price, category_id, pieces and retrieve second page using keyset/seek pagination? a page have 10 items

https://blog.jooq.org/2016/08/10/why-most-programmers-get-pagination-wrong/

SQL using offset pagination will be

SELECT * FROM products
WHERE title like '%search_term%' AND price > 100 AND price < 400 AND category_id=11
ORDER BY price DESC
LIMIT 10 OFFSET 10

Results can be in this order

(id=23, title='Some text', price=354, category_id=11)
(id=41, title='Big text', price=333, category_id=11)
(id=43, title='big big text', price=333, category_id=11)
(id=38, title='A text', price=288, category_id=11)
(id=11, title='text', price=200, category_id=11)
like image 737
tewumat Avatar asked May 15 '18 10:05

tewumat


People also ask

How does keyset pagination work?

Keyset pagination (also known as the "seek method") is used to fetch a subset of records from a table quickly. It does this by restricting the set of records returned with a combination of WHERE and LIMIT clauses.

What is Seek pagination?

The seek method is based on filtering out the data from the previous pages. We do this by having the client send the ID of the last record listed. We take that ID and place it in the WHERE clause providing us with only relevant data.

What is pagination in Postgres?

Paging is achieved in PostgreSQL by using the OFFSET, LIMIT and optionally FETCH NEXT operators. LIMIT and OFFSET allow you to retrieve just a portion of the rows that are generated by the rest of the query.

Do not use offset for pagination?

What is wrong with OFFSET and LIMIT? As we briefly explored in the past paragraphs, OFFSET and LIMIT work great for projects with low to no data usage. The issue arises when your database starts gathering more data than your server can store in memory and you still need to paginate performantly through them all.


1 Answers

A pure SQL solution for keyset pagination

The first thing to understand when talking about keyset pagination is that there's no such thing as the "second" page. There are only "next pages" given a "current page". In your case, the current page would be the one that ends on:

(id=11, title='text', price=200, category_id=11)

So, the next page would be the one that has a (price, id) < (200, 11) (current price, id). If this query produces your first page:

SELECT * 
FROM products
-- "Ordinary predicates"
WHERE title LIKE '%search_term%' 
AND price > 100 AND price < 400 
AND category_id = 11
ORDER BY price DESC, id DESC
LIMIT 10

Then, this query will produce your next page

SELECT * 
FROM products
-- "Ordinary predicates"
WHERE title LIKE '%search_term%' 
AND price > 100 AND price < 400 
AND category_id = 11
-- "Keyset pagination predicate"
AND (price, id) < (200, 11)
ORDER BY price DESC, id DESC
LIMIT 10

Alternatively, that predicate could be expanded to this:

-- "Keyset pagination predicates"
AND (price < 200 OR price = 200 AND id < 11)

Or even to this:

-- "Keyset pagination predicate"
AND price <= 200
AND (price < 200 OR price = 200 AND id < 11)

Depending on the database, the three different predicates may perform differently

A jOOQ solution

Since you're referencing the jOOQ blog, here's how you would write the query on the second page using jOOQ:

DSL.using(configuration)
   .selectFrom(PRODUCTS)
   .where(PRODUCTS.TITLE.like("%search_term%")
   .and(PRODUCTS.PRICE.gt(100))
   .and(PRODUCTS.PRICE.lt(400))
   .and(PRODUCTS.CATEGORY_ID.eq(11))
   .orderBy(PRODUCTS.PRICE.desc(), PRODUCTS.ID.desc())
   .seek(200, 11) // Automatic generation of keyset pagination predicates
   .limit(10)
   .fetch();
like image 124
Lukas Eder Avatar answered Oct 22 '22 00:10

Lukas Eder