Paging, sorting, filtering and retrieving specific fields in your RESTful API

Ref. https://medium.com/@guillaume.viguierjust/paging-sorting-filtering-and-retrieving-specific-fields-in-your-restful-api-a0d289bc574a

Fifth article in the series of “ Pragmatic decisions for your RESTful API”, this post talks about paging, sorting, filtering and retrieving specific fields in your RESTful API.

Paging your results: use limit & offset query parameters, return x-total-count header

To page your results, use an offset and limit query parameters. For example, to return between user 50 and 100:

GET /users?offset=50&limit=50

Some other things to think about when paging your results:

1. Return the total number of results via an X-Total-Count header
2. If you want to provide links to previous and next pages, you can use the Link HTTP header
3. In your implementation, set a maximum limit, such as 1000, to avoid server issues

There are other possibilites out there to page results, such as using the Range header. They are not, however, very practical. Think about your end users: will it be easier for them to simply add an offset and limit as query parameters of a GET request, or to define this in a Range header ?

Sorting your results: use sort query parameter

Simply use a sort query parameter, allowing the ascending or descending order to be defined via a minus (-) sign, for example:

GET /users?sort=-created_at

Filtering your results: use query parameters

Use a unique query parameter for each of your fields, for example, to filter users based on their first name:

GET /users?first_name=Guillaume

Searching your results: use a q query parameter

If you would like to add full text search to your API, use a q query parameter, for example:

GET /users?q=Guillaume

Retrieving specific fields: use a fields query parameter

Your API should allow end users to retrieve specific fields, instead of retrieving the full object. To do this, use a query parameter named fields. For example, to retrieve only the first_name and last_name attributes of the users:

GET /users?fields=first_name,last_name

Advanced filtering: use LHS brackets

The filtering method proposed above, using query parameters, works well, but provides the possibility to filter with exact matches only. What if you want to implement the following filters for example:

You have various possibilities.

1. Using LHS brackets with an operator: price[lte]=200 status[ne]=past
2. Using RHS colon with an operator: price=lte:200 status=ne:past
3. Using Lucene syntax or ElasticSearch Simple Query strings directly: q=price:<200 q=-status:past

Let’s look at them in more details:

LHS brackets with an operator

• Easy to use for your users
• Relatively simple to parse server side
• Relatively intuitive

RHS colon with an operator

• Easy to parse server side
• Easy to use for your users
• No possibility to do an OR for combined filters
• Less intuitive than LHS brackets

Lucene or ElasticSearch syntax

• The most flexible of all
• Almost no parsing required server-side (can be handed over directly to ElasticSearch for example)
• Steep learning curve for your users
• Requires URL percent encoding (makes it harder to use from curl or postman)

If you need to provide maximum flexibility for your users, and basically allow them to query your API almost as if they were querying the database directly, my advice is to use the Lucene or ElasticSearch syntax directly, even though this will require more learning from your users.

However, for the vast majority of cases, using the LHS brackets with an operator is the recommended solution IF your API needs such advanced filtering methods.

If you want to read all of the articles of the series “ pragmatic decisions for your RESTful API”, you can start here. The next article will focus on versioning a RESTful API.

If you need it in a handy format, this series of articles is also available as an eBook on Amazon.

---

Originally published at https://www.gvj-web.com on March 26, 2018.