Back to Basics: HTTP Requests in Rails Apps

The Hypertext Transfer Protocol specifies how a client machine requests information or actions from servers. This protocol specifies how two machines share information, which is called a request. These requests are composed of several parts which I’ll outline below.

The first line of an HTTP request is called the Request-Line. It contains:

Let’s take a closer look at these four elements.

URI

A URI or Uniform Resource Identifier is how objects are identified. Clients use URIs to tell the server what object to act on for a given request. In more general terms a URI is nothing more than a web address.

Request Header Fields

A client can communicate additional information to the server via a request’s headers. In addition to just communicating information about the client, ie. what type of browser the request originated from, these values can modify how the server responds to the request. For example, the Content-Type header value is used by the Rails framework to decode a request’s message body.

Message Body

The message body is used to send information from the client to the server about the entity the client wants to modify or create. The message can be communicated using several different encoding mechanisms, some of which I’ll discuss below. It is important to note that not all of the requests I discuss below are allowed to have message bodies.

Method

The method, sometimes called “verb” or “action”, tells the server what the client wants it to do. There are many different methods available, but we’re going to limit this blog to the four that are most relevant to Rails developers.

  • GET - how a client machine tells a server that it wants information about the item identified by the URI. Because GET requests are all about asking for information, they are not permitted to have request bodies. You still have the URI query string available to you if you need to send data from the client to the server on a GET request.
  • POST - how a client tells a server to add an entity as a child of the object identified by the URI. The entity that the client expects the server to add is transmitted in the request body.
  • PATCH - how a client tells a server it wants to modify an object identified by the URI the request is sent to.
  • DELETE - as you might guess, how a client tells a server to remove an object identified by the URI the request is sent to.

Let’s make some requests

cURL

cURL is a utility that makes it possible to send requests from the command line. We’ll use cURL to make some requests to a test Rails app which responds with strings.

Routes:

BackToBasics::Application.routes.draw do
  match '/curl_example' => 'request_example#curl_get_example', via: :get
  match '/curl_example' => 'request_example#curl_post_example', via: :post
end

Controller:

class RequestExampleController < ActionController::Base
  def curl_get_example
    render text: 'Thanks for sending a GET request with cURL!'
  end

  def curl_post_example
    render text: "Thanks for sending a POST request with cURL! Payload: #{request.body.read}"
  end
end

First we’ll make a GET request. We tell cURL to do a GET request (which is actually the default) with the “X” option.

% curl -X GET http://localhost:3000/curl_example
Thanks for sending a GET request with cURL!

Rails server log:

Started GET "/curl_example" for 127.0.0.1 at 2013-06-21 14:38:22 -0700
Processing by RequestExampleController#curl_get_example as */*
  Rendered text template (0.0ms)
Completed 200 OK in 1ms (Views: 0.3ms | ActiveRecord: 0.0ms)

As you can see, our Rails app receives the GET request we sent from the terminal and then responds with the string we provided in the controller. The app uses the request’s URI and Method to figure out which controller and action to call.

Next we’ll make a POST request with a data payload. Again, we use “X” to specify the method. We also use “d” to specify the data to send in the payload.

% curl -X POST -d "backToBasics=for the win" http://localhost:3000/curl_example
Thanks for sending a POST request with cURL! Payload: backToBasics=for the win

Rails server log:

Started POST "/curl_example" for 127.0.0.1 at 2013-06-21 14:47:37 -0700
Processing by RequestExampleController#curl_post_example as */*
  Parameters: {"backToBasics"=>"for the win"}
  Rendered text template (0.0ms)
Completed 200 OK in 0ms (Views: 0.3ms | ActiveRecord: 0.0ms)

The Rails app receives our request. This time, in addition to the URL, it logs the data payload as hash of parameters.

Web Browsers

We’re all familiar with surfing the web. I’m sure it comes as no surprise that this experience is made up of a series of requests and responses. Let’s take a look at what’s happening when we type a URL into our browser’s address bar and hit enter. We’ll also look at what happens when we enter data into a form and submit it.

Below is the demo controller we’ll be sending requests to for this example.

Routes:

BackToBasics::Application.routes.draw do
  root to: 'request_example#index'
  match '/request' => 'request_example#create', via: :post
  match '/request' => 'request_example#create', via: :get
end

Controller:

class RequestExampleController < ActionController::Base
  def index
  end

  def create
    render json: params
  end
end

Address Bar

Typing a URL into the address bar of a web browser sends a GET request to the URL specified. Sending a GET request in this fashion looks identical to sending a GET request through the terminal with cURL.

If we set the root path of our demo app to the index action of our dummy controller and navigate to localhost:3000 the browser will send the GET request. If we take a look at the Rails console we’ll notice the output is almost identical to what we saw with cURL.

Started GET "/" for 127.0.0.1 at 2014-01-31 15:09:53 -0800
Processing by RequestExampleController#index as HTML
  Rendered request_example/index.html.erb (1.2ms)
Completed 200 OK in 26ms (Views: 25.5ms | ActiveRecord: 0.0ms)

Forms

A form is made up of several key parts (we’ll look at several simple examples a bit later). In the opening form tag we have the action attribute. This attribute tells the form where to send the request. In addition to the action attribute we have the method attribute. This tells the form what type of request to send to the URI specified in the action attribute.

Request bodies are defined by a form’s markup. In the form tag there is an attribute called enctype, this attribute tells the browser how to encode the form data. There are several different values this attribute can have. The default is application/x-www-form-urlencoded, which tells the browser to encode all of the values. If a form includes a file upload an enctype of multipart/form-data should be used. This encodes none of the values. Finally, you can set the enctype to text/plain this converts spaces, but leaves all other characters unencoded. Inside the inside the form element we have input elements. These elements will render as assorted input types in our website. Each input element in the form should have a name attribute. This name attribute tells the browser what to name the data specified in that input in the message body. The type attribute tells the browser in what format to communicate the data in the message body. There is more to this as outlined here.

GET

Let’s take a look at how we could send a GET request with a form.

The simple form below is made up of one text field and that text field will have the name my_data. The final input is the submit which tells the form we actually want to send the request to the URI specified in the action attribute.

Let’s send a request. Assume a user has navigated to a page and the following form has been rendered. In the text box named my_data a user has entered the string “back to basics" and clicked the submit button.

<form action="/request" method="GET">
  <input type="text" name="my_data">
  <input type=submit>
</form>

Rails server log:

Started GET "/request?my_data=back+to+basics" for 127.0.0.1 at 2013-06-21 14:44:25 -0700
Processing by RequestExampleController#create as HTML
  Parameters: {"my_data"=>"back to basics"}
Completed 200 OK in 0ms (Views: 0.2ms | ActiveRecord: 0.0ms)

There are several interesting things about this request. You’ll notice that our Rails app received the request, but the URL includes a query parameter called my_data. This is the result of our decision to use the GET method for this request. Because GET requests have no payloads the data we collected with our form is added to the URI. In addition, you’ll notice that our text input ends up with a name of my_data in the payload, or the value we specified with the name attribute of our text input.

We can open the Network tab in our developer tools (Firefox or Chrome) and see that the data was added to the query string. This is because the action on the form is GET.

POST

The POST action works almost identically to the GET request with the exception of the payload. Let’s submit our form with the same text input and see what happens this time.

<form action="/request" method="POST">
  <input type="text" name="my_data">
  <input type=submit>
</form>

Rails server log:

Started POST "/request" for 127.0.0.1 at 2013-06-21 14:49:11 -0700
Processing by RequestExampleController#create as HTML
  Parameters: {"my_data"=>"back to basics"}
Completed 200 OK in 0ms (Views: 0.2ms | ActiveRecord: 0.0ms)

The first thing to notice is that our URL no longer contains the query parameter. It’s also important to note that our parameters hash looks identical. Let’s take a look at our network tab and see if we can learn anything about the request.

Examining the request we see that our payload includes what’s called form data. Our input elements are converted to a request payload and sent to the server. Again the name of our text input element is used as the name associated with the user’s text input.

XMLHttpRequest

It’s also possible to send requests via JavaScript. There is nothing special about these request from a mechanical perspective. They’re just requests like the ones we’ve sent above.

Because these requests are sent with JavaScript, in order to see what happens we’ll have to provide a function that deals with the response. We’ll use a simple function that will write whatever response we get to the JavaScript console.

function callback () {
  console.log(this.responseText);
};

Ajax Form Data

When sending our Ajax requests we have several different options as to how we want to send the data. As we saw above there is a concept of form data. We can easily create a form data payload using only JavaScript.

var request = new XMLHttpRequest();
request.onload = callback;
request.open("post", "http://localhost:3000/request");
var formData = new FormData();
formData.append('my_data', 'back to basics')
request.send(formData);

Rails server log:

Started POST "/request" for 127.0.0.1 at 2013-06-21 14:53:03 -0700
Processing by RequestExampleController#create as */*
  Parameters: {"my_data"=>"back to basics"}
Completed 200 OK in 0ms (Views: 0.1ms | ActiveRecord: 0.0ms)

As is obvious from our log this request was handled no different than a “normal" form submission by our Rails application. There is no special magic about an Ajax request as far as our server is concerned.

Ajax JSON Data

Another option we have is to use JSON. In order to do this we need to slightly modify our request headers and tell our server that it needs to do something slightly different to parse our payload.

var request = new XMLHttpRequest();
request.onload = callback;
request.open("post", "http://localhost:3000/request");
request.setRequestHeader("Content-Type", "application/json");
request.send('{"my_data":"back to basics"}');

Rails server log:

Started POST "/request" for 127.0.0.1 at 2013-06-21 14:55:55 -0700
Processing by RequestExampleController#create as */*
  Parameters: {"my_data"=>"back to basics", "request_example"=>{"my_data"=>"back to basics"}}
Completed 200 OK in 0ms (Views: 0.2ms | ActiveRecord: 0.0ms)

As you can see by simply modifying our request headers our Rails app is able to appropriately parse our payload and we end up with our my_data value available to our application.

Requests are one of the foundational elements of the internet as we know it. Understanding the individual elements of a request can make it much easier to debug issues with our Rails apps.