4 + 1 ways for making HTTP requests with Node.js: async/await edition

Ref. https://www.valentinog.com/blog/http-js/

HTTP requests are a means for fetching data from a remote source. It could be an API, a website, or something else: at some point you will need some code to get meaningful data from one of those remote sources.

Starting from the easier one we will explore the "classic way" for doing HTTP requests all the way through libraries which support Promises. I will focus mostly on GET requests in order to keep things simple and understandable.

What you will learn

• How to make HTTP requests in Node.js with various modules
• pros and cons of every module

Requirements

To follow along you should have a basic understanding of JavaScript and ES6. Also, make sure to have one of the latest versions of Node.js. In the following post we'll use async/await, introduced in Node 7.6.0.

Making HTTP requests with Node.js: why?

At this point you might be asking "Why would I ever do an HTTP request?".

The answer is simple: as a JavaScript developer you will interact every day with remote APIs and webservers. Almost everything today is available behind an API: weather forecasts, geolocation services and so on.

Node.js can be used to serve a vast range of purposes: you can build a command line tool, a proxy, a web server, and in its simplest form can be used just for querying a remote API and returning the output to the user.

In the next examples we'll be making HTTP requests with Node.js by calling a convenient "fake" API: the JSON Placeholder API.

Setting up the project

To start, create an empty folder and initialize the project:

mkdir making-http-requests-node-js && cd $_ npm init -y

There are two simple ways for making HTTP requests with Node.js: with a library which follows the classic callback pattern, or even better with a library which supports Promises. Working with Promises means you could also use async/await.

Let's start with callbacks!

Making HTTP requests with Node.js: http.get and https.get

http.get and https.get (for HTTPS requests), are the first choices for making requests in Node.js. If you just need to GET something from an API, stick with them.

PROS:

• native API, there is no need to install third party modules
• the response is a stream

CONS:

• a bit verbose
• the response is a stream
• no support for Promises

To test things out create a new file named https-native.js:

const https = require("https"); const url = "https://jsonplaceholder.typicode.com/posts/1"; https.get(url, res => { res.setEncoding("utf8"); let body = ""; res.on("data", data => { body += data; }); res.on("end", () => { body = JSON.parse(body); console.log(body); }); });

Now if you run this code with:

node https-native.js

you should be able to see the following output:

{ userId: 1, id: 1, title: 'sunt aut facere repellat provident occaecati excepturi optio reprehenderit', body: 'quia et suscipit\nsuscipit recusandae consequuntur expedita' }

https.get expects an url as a first argument and a callback as a second argument. The returned response is an http.ClientRequest object. That means, in order to manipulate the body of the response you have to listen for events: notice res.on() in the example.

Now, in this example I'm just logging the response to the console. In a real program you may want to pass the response to a callback.

The http.ClientRequest object emits events that you can listen to. And that's both good and "bad": good because you will be tempted to dig further into the Node.js internals to learn more and "bad" because you are forced to do a lot of manipulation if you want to extract the JSON response.

In the end, working with http.get could be slightly more verbose compared to other libraries but that shouldn't be considered a drawback.

Making HTTP requests with Node.js: the request module

Note: the request module has been deprecated in February 2020.

request is one of the most popular NPM module for making HTTP requests with Node.js. It supports both HTTP and HTTPS and follows redirects by default.

PROS:

• ease of use

CONS:

• no Promises
• too many dependencies

To install the module run:

npm i request

To test out the example create a new file named request-module.js:

const request = require("request"); const url = "https://jsonplaceholder.typicode.com/posts/1"; request.get(url, (error, response, body) => { let json = JSON.parse(body); console.log(json); });

By running the code with:

node request-module.js

you should be able to see the same output as in the previous example. request.get expects an url as a first argument and a callback as a second argument.

Working with the request module is pleasing. As you can see in the example, it is much more concise than http.get.

There’s a drawback though: request relies on 22 dependencies. Now, I wouldn't consider this a real problem, but if your goal is to make just an HTTP GET request, sticking with http.get will be enough to get the job done.

The request module does not support Promises. It could be promisified with util.promisify or even better you could use request-promise, a request version which returns promises (and has less dependencies).

Making HTTP requests with Node.js: I Promise I'll be async

So far we've seen how to make HTTP requests in the most basic way with callbacks.

But there is a better (sometimes) way to handle async code: using Promises alongside with async/await. In the next examples we'll see how to use a bunch of Node.js modules which support Promises out of the box.

Making HTTP requests with Node.js: the node-fetch module

node-fetch is an implementation of the native Fetch API for Node.js. It's basically the same as window.fetch so if you're accustomed to use the original it won't be difficult to pick the Node.js implementation.

PROS:

• support for Promises
• same API as window.fetch
• few dependencies

CONS:

• same ergonomics as window.fetch

To install the module run:

npm i node-fetch

To test out the example create a new file named node-fetch.js:

const fetch = require("node-fetch"); const url = "https://jsonplaceholder.typicode.com/posts/1"; const getData = async url => { try { const response = await fetch(url); const json = await response.json(); console.log(json); } catch (error) { console.log(error); } }; getData(url);

By running the code with:

node node-fetch.js

you should be able to see the same output again. If you paid attention I've listed "same API as window.fetch" both in Pros and Cons.

That's because not everybody likes the Fetch API. Some developers (me?) can't stand out the fact that to manipulate the response you have to call json() and two times then. But in the end it's a matter of getting the job done: use whichever library you prefer.

Making HTTP requests with Node.js: the r2 module

The request module for Node.js was written by Mikeal Rogers back in 2010. In 2017 he is back with the r2 module. The r2 module uses Promises and is another implementation of the browser's Fetch API. That means r2 depends on node-fetch.

At first it wasn't clear to me why would I consider using r2 over node-fetch. But I thought the module is worth a mention.

PROS:

• support for Promises
• same API as window.fetch
• few dependencies

CONS:

• it depends on node-fetch

To install the module run:

npm i r2

To test things out create a new file named r2-module.js:

const r2 = require("r2"); const url = "https://jsonplaceholder.typicode.com/posts/1"; const getData = async url => { try { const response = await r2(url).json; console.log(response); } catch (error) { console.log(error); } }; getData(url);

Run the code with:

node r2-module.js

and you should be able to see (again) the same output. In all honesty I didn't take the time to look at r2 in details. But I'm sure it has more to offer.

Making HTTP requests with Node.js: the axios module

Axios is another super popular NPM module for making HTTP requests. It supports Promises by default.

Axios can be used both for the front-end and the back-end and one of its core feature is the ability to transform both the request and the response. You don't need to explicitly process the response in order to get JSON as you did with node-fetch.

PROS:

• support for Promises
• ease of use
• only 2 dependencies

CONS:

• ??

Install axios inside your project folder:

npm i axios

To test out the example create a new file named axios-module.js:

const axios = require("axios"); const url = "https://jsonplaceholder.typicode.com/posts/1"; const getData = async url => { try { const response = await axios.get(url); const data = response.data; console.log(data); } catch (error) { console.log(error); } }; getData(url);

Again, by running the above code you should be able to see the same output of the previous examples. I really can't find any drawback in axios. It's super simple to use, highly configurable and intuitive. Last but not least it has only 2 dependencies.

Other libraries

There are also other libraries for HTTP requests in Node.js like bent, a modern alternative to request. Give it a try.

Making HTTP requests with Node.js: conclusions

As with almost everything with JavaScript, sometimes picking one module over another it's a matter of personal preferences.

My rule of thumb is to pick the smallest library with the fewer dependencies possible based on what I want to do. If I were to make a super simple GET request I wouldn't install neither axios nor node-fetch.

In contrast, using a third party module could save you a lot of coding when in need for more complex functions: i.e. request and response manipulation.

Thanks for reading!