HTTP Servers

This document describes how to implement an HTTP server, from an incoming client request to closing the response.

Intermediate nodes (gateways and proxies) must also follow the behavior for clients, since they are making (forwarding) requests, as well as accepting them.

Not every section will be applicable to each server, read carefully.

Listening for Requests

There are multiple ways to listen for HTTP requests, and you may wish to implement more than one of them.

HTTP is primarily defined in the concrete message syntax, shared by HTTP/1.x and is similar to MIME, used for email. HTTP servers should typically always listen on HTTP/1.x (HTTP/1.x refers to both HTTP/1.1 and HTTP/1.0).

HTTP/1.x by default listens on TCP port 80, and HTTP over TLS on port 443. HTTP also provides alternate ports 8008 and 8080, which is useful for origin servers behind a gateway/firewall, if a unix socket cannot be used.

Servers are expected to be robust against denial-of-service attacks.

An incoming connection might not necessarily be the one making the requests, and a connection might send requests from many different user agents.

Status codes and emitting errors

Some operations below speak of emitting an error, or emitting a status code. This means the response to that request will be a status code (4xx or 5xx if an error), generating a document describing the error (if allowed), and further handling of the request will end (except as necessary to unlock and clean up resources, etc).

Retry-After

In certain cases, especially due to server load or unrecognized parameters from the client, the server may want to advertise how long the client should wait to retry a request. Use the Retry-After header to specify this.

Handle rate limiting and overload condition

In some cases, especially for closed applications, servers may want to signal they are overloaded and the request should be retried at a later date. In this case, the server may respond with 503 (Service Unavailable). The server should accompany this with a Retry-After header.

Parse incoming request headers (HTTP/1.x)

Except for the simplest of embedded routers, most implementations will need to parse all of the request headers and buffer them in memory, since headers may be specified in any order.

Header length limits

While HTTP does not specify any specific limits on headers, servers may emit an error if a request has exceeded resource limits:

414 (URI Too Long): When the request-URI is very long (this length is 8190 in Apache HTTPD)
431 (Request Header Fields Too Large): When there's a header that's too long, or too many headers. Send this when a header name is very long (the longest known header plus some amount to accommodate new headers without interrupting client requests, about 100 bytes), when the value is very long (8190 bytes in Apache HTTPD), or when there's too many headers (100 count in Apache HTTPD)

If the server statically knows all of the resources it needs to process, the request URI limit should be at least as large as the largest URI.

Resource limits for the request-body are treated below.

HTTP version

The HTTP version is specified on the first line, the request-line, for example:

GET /where?q=now HTTP/1.1

If the HTTP-version is not HTTP/1.x, where x is some digit, then return 505 (HTTP Version Not Supported).

Consume Connection headers

Several headers are hop-by-hop headers and must be modified by, or set by, every node along the request pipeline.

These headers are typically managed by HTTP request libraries. And request libraries should not allow programs to set these headers (except for debugging purposes).

Connection header

In HTTP/1, Connection header lists header names and special flags that apply to the current, hop-by-hop connection (instead of the HTTP message itself).

Do not forward the Connection header, nor any headers named within it.

Connection may also list close as a flag indicating the connection will be closed at the end of this request. See "Clean up connection" below for how to handle this flag.

Verify connection requirements

Validate Expect header

The Expect header enumerates the features the client expects the endpoint to implement in order to be able to correctly parse a request or response.

If the header is set, and is something besides 100-continue (case-insensitive), emit 417 (Expectation Failed).

If the value is "100-continue", the server will need to emit a 100 (Continue) header before the client begins uploading its request-body (implemented below)

Validate Host header

HTTP/1.1 clients MUST send a Host header, even when using the absolute-form or with a request URI with no authority (in which case the header will be empty).

If the request lacks a Host header, or has more than one Host header, return status 400 (Bad Request).

Compute effective request URI

In HTTP/1, two lines are used to form the effective Request URI: the request-line (the first line in the request), and the Host header.

In HTTP/2, multiple pseudo-headers form the effective request URI; the URI is broken apart to take advantage of HTTP header compression.

If the asterisk-form is used with a method besides OPTIONS, emit 405 (Method Not Allowed).

If the authority-form is used with a method besides CONNECT, emit 405 (Method Not Allowed).

Route Request-URI

HTTP servers may act as intermediaries and forward the HTTP message to another server; if and where the message is forwarded to may vary by the scheme, authority, or path component of the URI.

If the server is forwarding the message to another node instead of handling it as an origin, it must implement the Via and Max-Forwards headers. If Max-Forwards: 0 is specified for a TRACE or OPTIONS request, the server must not forward the message, but must act as the origin server.

Recall that connection headers (including the Connection header itself, described above) cannot be forwarded since they convey information about the particular connection between nodes.

Via header

The Via header specifies which intermediate nodes also processed the request, and will process the response. The server may use this for logging, debugging, or other informational purposes.

If this message is going to be forwarded to another server, then append the current server information to the Via header.

Forwarded header

The Forwarded header specifies which intermediate nodes also processed the request and associated information about the hop. The server may use this for logging, debugging, or other informational purposes.

This varies from the "Via" header in that the Forwarded header is extensible, and can contain information such as the client IP address, which would otherwise be lost by a proxy or gateway.

If this message is going to be forwarded to another server, then append the current server information to the Forwarded header.

Max-Forwards header

For nodes that would normally forward the request, the Max-Forwards header indicates if the node may forward the request. If the received value is 0, then the node cannot forward the message, but must act as the target recipient; otherwise it must forward the message after decrementing the value.

Origin servers may generally ignore this header.

Authenticate credentials

Users may send an Authorization header (or Proxy-Authorization header for intermediaries) that carries credentials authorizing them to perform the request.

Some user agents might identify themselves with a TLS client certificate. In these cases, that connection is authenticated, but not necessarily any individual HTTP message, which might be forwarded from a user agent, or be generated by a third party script; the server should still require authorization headers in addition to the client certificate to prevent ambient authority attacks. (Subrequests from untrusted sources should be careful to make the request over an unauthenticated connection, to prevent an ambient authority/confused deputy attack.)

The operations performed for the rest of request processing assume the server will verify these credentials before performing the operation. If an operation requires authority, but the client has not specified any credentials, or the credentials are insufficient for the requested operation, emit 401 (Unauthorized) with a WWW-Authenticate header.

Rate limiting users

If the server is able to keep track of requests per user, it may signal a request from that specific user has exceeded limits with 429 (Too Many Requests). The server should use Retry-After to specify the next time a client will be able to make successful requests.

The server can make this determination based on routing information (like IP address), or valid credentials, whichever is favorable to the user.

Request metadata

The following headers contain information about the request or the user making the request:

From: An email address that can be contacted if there's any questions about the user-agent or user making the request. We suggest using this for automated scripts and bots, in the case of misbehavior.
Referer: Specifies where the current request URI was obtained from.
User-Agent: Names/describes the user agent making the request.

Consider logging this information for debugging purposes.

Handle method

Process subroutines that vary based on incoming method.

GET, HEAD

Dereference the resource according to the section below.

OPTIONS

OPTIONS is specified to permit requesting metadata about a resource as a document, such as details of Content-Type negotiation, though at present time there is no standard for how to represent such information.

PUT

The PUT method updates a resource with an uploaded document, creating it if it doesn't exist.

The server should look up the resource being replaced (if any) and perform the necessary post-replace hooks, but this is not of concern to the client.

Dereference resource

Most methods will need to dereference the resource: look up the effective request URI from the routing table and/or database, to determine if it exists.

If the resource is expected to exist, but does not exist, return 404 (Not Found).

If a resource is marked as deleted, return 410 (Gone).

If the resource is supposed to be created, but the server cannot create the resource (e.g. the selected URI is read-only), return 403 (Forbidden).

Applications should route based on the full URI and be prepared to route requests for multiple different hosts.

Redirects

The server can mark some resources as having moved to a different URI, for various reasons.

Servers should avoid renaming resources if there's not a pressing problem with keeping a URI.

In service of redirects, select an appropriate 3xx redirect, and use the Location header to specify the resource to direct the user-agent to.

Resources and other operations can send a redirect (e.g. a service that points a user to a random webpage), these uses will be explored later.

Handle Upgrade

The Upgrade request header asks the server to transition the protocol from HTTP/1.1 to some other protocol. Multiple protocols may be specified, which the server may choose from. The request is usually sent as a GET request.

If the server intends to honor the request, then respond with 101 (Switching Protocols), a Connection: upgrade header, and an Upgrade header specifying the protocol. The server may ignore the header if it's content with the protocol as it is.

Validate request body headers

Verify a request-body is acceptable

The presence of a message body in a request is signaled by a Content-Length or Transfer-Encoding header field. Verify that the current resource supports a method that accepts a content body.

Identify request body URI

It is possible for the user-agent to specify a URI for the request body with the Content-Location header.

If it is present, its meaning is left largely up to the server do decide.

Transfer-Encoding

Verify that all applied transfer codings are known to the server, otherwise return 415 (Unsupported Media Type).

Content-Type

The Content-Type header determines which media type will be used to decode the entity-body. Media types are listed in a non-trivial format that accepts case, whitespace, parameters, and quoted-strings, so use a library to parse and normalize the Content-Type before reading it.

Generally for every media type, there is a single specification that specifies how to parse and use the document. Parse the Content-Type (using a library that parses media types) and resolve which parser the server should use to parse the body, if any.

If the server has no way to consume the media type, return 415 (Unsupported Media Type).

Content-Type charset

Most media types, especially text/* types, implement a character set, a method of decoding a byte to a particular character. If the media type uses a charset decoding, ensure the server is capable of decoding it. The server should support ASCII and UTF-8.

Content-Length

Verify that the Content-Length is reasonable. If it is above the maximum length the server is willing to handle, return 413 (Payload Too Large). The server may wish to vary this maximum based on the request-URI, Content-Type, method, and user authorization.

Content-Encoding

Verify that all applied Content-Encodings are known to the server, otherwise return 415 (Unsupported Media Type).

Content-Language

HTTP allows the sender to identify the primary language of the entity body (request body or response body) using the Content-Language header.

Handle 100-continue

If the client set the 100-continue flag, that means the client has paused uploading the request-body, and is waiting on verification from the server that the proposed upload looks acceptable and fits the resource limits.

Since at this point all the headers look acceptable, emit a 100 (Continue) response: an intermediate response that will be replaced with the actual status code later on.

Alternatively, if the client did not ask for 100-continue and there's a serious problem with accepting a large upload, the server can close the entire TCP connection.

Consume message-body

If nothing looks wrong with the request metadata, the server can begin accepting the payload body.

If the first transfer coding to decode is "chunked", then read the chunked stream until it indicates the request-body is EOF.
Otherwise if there is a Content-Length header, read that many bytes, then the request ends.

Decode transfer-codings

A transfer coding is a transform applied to the requested resource for transporting the representation over the network, usually to compress the representation, and to indicate when the body has ended.

HTTP servers MUST support the "chunked" encoding. Servers should support the following transfer-codings:

chunked
compress
deflate
gzip

The chunked transfer coding

The chunked Transfer-Encoding is used to specify more detailed information about the status of the transfer, especially indicating when the transfer has ended.

Decode content-encoding

The values for Content-Encoding are:

identity
compress
deflate
gzip

Handle request body

Procedure for handling the request body varies with the method:

GET: Ignore the request body, or return 413 (Payload Too Large). GET with a request body is technically legal, and could be used as a kind of safe POST; in this case the server must indicate the response is not cachable.
HEAD: Same as GET.
POST: Semantics are defined by the resource; the server should have a subroutine associated with the resource that accepts and processes the POST request (including request body).
PUT: Store a resource on the server at a specific URI. The subroutine that stores the data may vary based on the request URI (e.g. authority or path) and Content-Type.
DELETE: Behavior is undefined, discard or return 413 (Payload Too Large).
CONNECT: Should have been handled in previous steps, generate 500 (Internal Server Error).
OPTIONS: Behavior is undefined, discard or return 413 (Payload Too Large).
TRACE: Clients are prohibited from providing a request body for TRACE; enforce this by emitting 413 (Payload Too Large).

Test conditional headers

Before any changes to the server state have been made, the server should test any conditional headers to see if the server should skip this request. See conditional requests for handling. Depending on the requested action, this step may return 304 (Not Modified) or 412 (Precondition Failed).

Report changes to server state

Created resources

If the request was one to create a resource (either a PUT request to a resource that didn't previously exist, or a POST request that created a new resource), then:

Set the status code to 201 (Created)
Set a Location header to point to the resource created. If multiple resources were created, point to a "primary" resource with sufficient links so that the other created resources may be discovered.
If the request is known to have come from a Web browser (e.g. by using a form field that says prefer=redirect), then instead use status code 303 to have the Web browser direct the user agent to the newly created resource.

Modified resources

If the request was one to modify a resource (usually a PUT request to a resource that already existed), then set the status code to 200 (OK) or 204 (No Content).

Ongoing process

If the request will take a long time to fulfill (more than several seconds), and wishes to inform the client of this, it can send a 202 (Accepted) response.

The result should describe how to request the result of the action when it finishes. Currently, there is no standardized means to do this.

Response line and connection details

HTTP-version

The server must reply with an HTTP version to which both the server and client are conformant. If the server is responding to a client in an HTTP version it doesn't understand, send the latest version the server understands in the response-line (e.g. HTTP/1.1).

Date

The Date header is required for responses, unless the server doesn't have an accurate time to send. (Clients may also send this header, but it's optional for them.)

For implementation, see HTTP Date header.

Connection

If the server is going to close the connection at the end of the response, it must send Connection: close to the client.

Select representation

Content-Type negotiation

If the request-URI is allowed to vary among a number of dimensions, use the various Accept headers to determine which representation to pick:

Accept
Accept-Charset
Accept-Language

See the page on Content-Type Negotiation for more information.

Set Content-Location

With the different dimensions of variance negotiated, compute the URI that uniquely identifies this selection. Send it in the Content-Location header. Preferably compute and send a relative URI, to minimize the chances that an intermediate node introduces problems if it rewrites the request-URI (e.g. the host header, or path).

Set Vary

If the server selected a representation from among multiple representation, the dimensions that it selected among must be listed in the Vary header.

Prefer header

The Prefer header specifies what sort of response a client may prefer from the server.

The server may indicate which of the preferences it honored using the Preference-Applied response header.

Transfer codings

The TE header specifies which transfer codings a client is capable of decoding.

The chunked transfer coding (without trailers) is assumed for HTTP/1.1 user agents.

If the client specifies the special value trailers, the user agent supports trailers in the chunked transfer coding. If the server wishes to use trailers, it must send the Trailer header.

Freshness and caching

Write the several headers that can be used for caching.

Caching control

Age
Cache-Control
Expires

Metadata about selected representation

ETag
Last-Modified

Write response body

Encode content coding

A content coding is a transform applied to the representation intended to be decoded by the final recipient. It is not typically decoded by intermediaries, except transforming proxies.

The applied content codings are specified in the Content-Encoding header.

Content codings are usually cached by the server.

Compute length

If the server knows the length of the response, it should send the Content-Length header. The client will read that many bytes after the end of headers as the response-body, and then begin reading the next response in the pipeline.

If the server does not know the length of the response by this point, different HTTP protocols handle the response differently:

HTTP/1.0: Terminate the connection to end the response. Ensure that Connection: close was sent earlier.
HTTP/1.1: Use the "chunked" Transfer-Encoding.

A server MUST NOT send Content-Length with a 1xx status, a (204 No Content) status, or in response to a CONNECT request.

Encode transfer coding

A transfer coding is a transform applied to the message in order to send the message over the network. Transfer codings are semantically transparent, and frequently re-encoded by intermediaries.

It is only available on HTTP/1.1, used as a method of specifying the end of an indeterminate response.

Applying a transfer coding means the server cannot send a Content-Length header, so its use should be avoided in favor of using Content-Length.

Write data

The data to respond with varies by method:

GET: the target resource
HEAD: no response body, but has the same headers as GET
POST: the output of the subroutine associated with the requested resource
PUT, DELETE: the result of operations performed
OPTIONS: list of communications options
TRACE: the request message as received by the end server

Clean up connection

Handle Connection: close

If the client specified the close token in the Connection header, or if the connection is HTTP/1.0, the server must close the TCP channel here (even if the client sent additional requests).

Recall that the server sends Connection: close to the client before it does this.