REST API Design

Theory

Ivan Novakov

ivan.novakov@merck.com

What are the typical features of a RESTful API?

What is REST?

REpresentational State Transfer 

A PhD dissertation by Roy Fielding

A core set of principles, properties and constraints describing the architectural style of the World Wide Web.

Focus on component roles and interactions rather than implementation details.

200 OK
Content-Type: text/html

<html>
    <body>
        <h1>Title</h1>
        <p>John Smith</p>
        <!-- ... -->
    </body>
</html>

200 OK
Content-Type: application/json

{
    "id": 123,
    "title": "Title",
    "author": "John Smith",
    ...
}

REST != API

REST is the foundation of the World Wide Web.

GET /posts/123 HTTP/1.1
Accept: text/html

Request

GET /posts/123 HTTP/1.1
Accept: application/json

Response

REST principles

Richardson Maturity Model:

  • Level 0: HTTP as a transport layer
  • Level 1: Resources
  • Level 2: HTTP verbs (and codes, headers, ...)
  • Level 3: Hypermedia

REST and HTTP

HTTP as a transport layer.

HTTP as a tunneling mechanism for remote interaction.

SOAP, XML-RPC, ... they (may) also use HTTP.

Resources

Everything is a resource!

Resource is an object, entity, ... anything, which can be described using a noun.

...or a collection of resources.

Identifiers

Every resource has a unique identifier (URI/URL).

Resource URLs usually resemble directory-like hierarchical structures.

https://api.news.example.com/v1/articles/123
https://api.finance.example.com/v2/currencies/USD
https://api.directory.example.com/v1/users
https://api.directory.example.com/v1/users?location=Prague
/articles
/articles/123
/articles/123/tags
/articles/123/tags/elections

Sub-resources

Resources dependent on the main resource:

/articles/123/paragraphs
/articles/123/paragraphs/1
/articles/123/tags
/articles/123/tags/elections
/users/jsmith/certificates
/users/jsmith/roomNumber

Resources related to the main resource:

Much better than:

/paragraphs?article=123

Resources and identifiers

/articles/123
/articles/123/author
/authors
/authors/456
/authors/456/publications
/categories
/categories/789
/categories/789/articles

/server/restart
/rpc/getStatus
/rpc/updateStatus
/rpc/getUsersByName
/article/123/publish
/article/123/delete

Resource:

Not a resource:

Verbs

Resources are manipulated through verbs - GET, POST, PUT, PATCH, DELETE, ...

The usage of these verbs should follow closely the HTTP specification.

GET

  • Used to "fetch" a resource.
  • A "safe" operation, it doesn't modify data on the server.
  • Never use GET for modifying content.
GET /articles/123 HTTP/1.1
Accept: application/json

POST

  • A "write" operation.
  • Used to create new resources.
  • Non-idempotent.
POST /articles HTTP/1.1
Content-Type: application/json

{
  "title": "A new title",
  "author": "John Smith",
  ...
}

PUT

  • A "write" operation.
  • Used to update (replace) resources as a whole.
  • Idempotent.
PUT /articles/123 HTTP/1.1
Content-Type: application/json

{
  "title": "A new title",
  "author": "John Smith",
  ...
}

PATCH

  • A "write" operation.
  • Used for partial updates.
  • Non-idempotent.
PATCH /articles HTTP/1.1
Content-Type: application/json

{
  "title": "A new title"
}

DELETE

  • A "write" operation.
  • Used to delete (deactivate, remove, ...) resources.
  • Idempotent.
DELETE /articles/123 HTTP/1.1

Verbs

Verb /articles /articles/123
GET Fetch a collection Fetch a resource
POST Create a new resource
PUT Update a resource
PATCH Partially update a resource
DELETE Delete a resource

HTTP status codes

A RESTful service uses the HTTP status code as a primary indicator about the success of a request.

  • 2xx: Successful
  • 3xx: Client should perform an additonal action
  • 4xx: Error caused by the client
  • 5xx: Error caused by the server

2xx

  • 200 OK
  • 201 Created
  • 204 No Content
  • 206 Partial Content

Success

3xx

  • 301 Moved Permanently
  • 302 Found
  • 304 Not Modified

The client should perform an additonal action

4xx

  • 400 Bad Request
  • 401 Unauthorized
  • 403 Forbidden
  • 404 Not Found
  • 405 Method Not Allowed
  • 415 Unsupported Media Type

Error caused by the client

5xx

  • 500 Internal Server Error
  • 501 Not Implemented
  • 503 Service Unavailable

Error caused by the server

Error reporting

400 Bad Request
Content-Type: application/json

{
    "error": "invalid input",
    "message": "The username should be more than 3 characters long.",
}
503 Service Unavailable
Content-Type: application/json

{
    "error": "service down",
    "message": "Cannot connect to the database",
    "details": {
        "db": "ORA-12150: TNS:unable to send data"
    }
}

Headers

Use HTTP headers for any metadata you need to transfer between the client and the server.

Metadata - any data, which are not part of the resource, but are somehow part of the interaction:

  • describe the data
  • describe how to handle the data
  • transport related

Content negotiation

GET /users/123 HTTP/1.1
Accept: application/json
200 OK
Content-Type: application/json;charset=utf-8
Content-Length: 180
 
{ 
    "username": "jsmith",
    ...
}

Authentication and authorization

GET /users/123
Accept: application/json
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=
401 Unauthorized
WWW-Authenticate: Basic realm="My Service"
GET /users/123
Accept: application/json

Request

Second request

Response

Cache control

GET /users/123
Accept: application/json
200 OK
Content-Type: application/json
Last-Modified: Fri, 03 Jun 2016 14:30:30 GMT

{ ... }
GET /users/123
Accept: application/json
If-Modified-Since: Fri, 03 Jun 2016 14:30:30 GMT
304 Not Modified
Content-Length: 0

Request

Request

Response

Response

Custom headers

X-Merck-<name>: <value>

X-Merck-APIKey: 1234567890
X-Merck-Some-Other-Header: foo

Current naming recommendation:

But the usage of "X-" prefixed headers has been deprecated!

Not quite relevant for us. Anyway, use some unique prefix, when defining custom headers.

Query parameters

/users?location=Prague
/users?location=Prague&firstName=Ivan

Filtering

Paging

/users?page=12&pageSize=10
/users?limit=12&offset=110

Sorting

/users?sortBy=firstName&sortDest=ASC
/users?sortBy=employeeNumber&sortDest=DESC

Combined

/users?location=Prague&firstName=Ivan&sortBy=employeeNumber&page=12

Hypermedia (HATEOAS)

  • Resources are provided with links pointing to related resources.
  • The client is able to traverse the resources using the links starting from the "entry point".
  • No other knowledge is required.

Hypermedia As The Engine Of Application State

Hypermedia (HATEOAS)

200 OK
Content-Type: application/json

{
    _links: {
        self: {
            href: "/users/123"
        },
        manager: {
            href: "/users/456"
        }
    }
    username: "jsmith",
    ...
}

200 OK
Content-Type: application/json
Link: </users/123>;rel=self,</users/456>;rel=manager

{
    "username": "jsmith",
    ...
}

Hypermedia (HATEOAS)

200 OK
Content-Type: application/json

[
    {
        "_links": { "self": { "href": "/users/123" } },
        "username": "jsmith"
    },
    {
        "_links": { "self": { "href": "/users/456" } },
        "username": "foobar"
    }
    ...
]
GET /users
Accept: application/json

Representation

The JSON (XML, YAML, CSV, ...) you get from a RESTful endpoint is not the resource, but its representation.

With content negotiation you can be quite flexible when choosing the right representation for your API.

Choosing representation

  • Be consistent
  • Use content negotiation
  • Adopt hypermedia
  • Prefer standards

HAL

Hypertext Application Language

  • JSON based (application/hal+json)
  • Hyperlinks in every resource with the _links meta field.
  • Sub-resources with the _embedded meta field.

HAL

Hypertext Application Language

{
  "_links": {
    "self": { "href": "/employees/123456" },
    "manager": { "href": "/employees/789012" }
  },
  "employeeId": 123456,
  "firstName": "Ivan",
  "lastName": "Novakov",
  "email": "ivan.novakov@company.com"
}

Single resource

HAL

Hypertext Application Language

{
  "_links": {
    "self": { "href": "/employees/123456" }
  },
  "employeeId": 123456,
  "firstName": "Ivan",
  "lastName": "Novakov",
  "email": "ivan.novakov@company.com",
  "_embedded": {
    "manager": {
      "_links": {
        "self": { "href": "/employees/789012" }
      },
      "employeeId": 789012,
      "firstName": "John",
      "lastName": "Smith"
    }
  }
}

Single resource with embedded subresource

{
  "_links": {
    "self": { "href": "/employees/123456" }
  },
  "employeeId": 123456,
  "firstName": "Ivan",
  "lastName": "Novakov",
  "email": "ivan.novakov@company.com",
  "_embedded": {
    "department": {
      "_links": {
        "self": { "href": "/departments/123" }
      },
      "id": 123,
      "name": "IT Department",
      "description": "The department running all the computers"
    },
    "manager": {
      "_links": {
        "self": { "href": "/employees/789012" }
      },
      "employeeId": 789012,
      "firstName": "John",
      "lastName": "Smith"
    }
  }
}

Single resource with embedded subresource

{
  "_links": {
    "self": { "href": "/employees?page=4" },
    "first": { "href": "/employees" },
    "next": { "href": "/employees?page=5" },
    "previous": { "href": "/employees?page=3" },
    "last": { "href": "/employees?page=120" }
  },
  "count": 10,
  "total": 1250,
  "_embedded": {
    "employees": [
      {
        "_links": { "self": { "href": "/employees/41" } },
        "employeeId": 41
      },
      {
        "_links": { "self": { "href": "/employees/42" } },
        "employeeId": 42
      },
      {
        "_links": { "self": { "href": "/employees/43" } },
        "employeeId": 43
      },
      ...
    ]
  }
}

Collection

Some representation formats

Consistency

Resources should be consistent in the way they return data.

An API request should (always) return one of the following:

  • a single resource
  • a collection of resources
  • an error message
  • no content (redirect, delete)

Consistency

GET /articles --> collection
GET /articles/123 --> resource
GET /articles/123/author --> resource
GET /articles/123/tags --> collection
GET /articles/123/tags/foo --> resource

POST /articles --> returns the newly created resource or
               --> 201 Created + redirect to the new resource

PUT or PATCH /articles/123 --> returns the modified resource

DELETE /articles/123 --> 204 No Content

Versioning

Why should your API support different versions?

Backward compatibility!

Versioning

URL:

https://iapi.merck.com/comet-otc/v1/orders
https://iapi.merck.com/comet-otc/v2/orders

Custom request header:

GET /comet-otc/orders
X-Merck-API-Version: 2

Content-Type / Accept header:

GET /comet-otc/orders
Accept: application/vnd.merck.api.comet-otc.v2+json

200 OK
Content-Type: application/vnd.merck.api.comet-otc.v2+json

{ ... }

Versioning

  • Distinguish major versions only.
  • A major version is a version introducing backward incompatibility.
  • When introducing a new version, run it in parallel with the old one for enough time for the clients to switch.
  • Do not run too many versions in parallel - ideally run only two at a time.

Design

Do you really need a REST API?

When you don't need an API:

  • Synchronization.
  • Batch data transfer.
  • Asynchronous messaging.
  • ...

But you can have an API to manage the process.

API vs. DB

API DB
GET /resource/123 SELECT * from table WHERE id = 123
POST /resource INSERT INTO table VALUES (...)
PATCH /resource/123 UPDATE table SET field = value, ...
WHERE id = 123
DELETE /resource/123 DELETE FROM table WHERE id = 123

API vs. DB

When the API is just a thin layer above the DB:

  • implementation details are exposed
  • internal complexity is revealed
  • the API changes every time the DB changes

API vs. DB

APIs should provide a certain level of abstraction

Discussion

Backup

AuthN / AuthZ

Client authenticaton - API key (client ID, client secret, ...)

User authentication - OAuth v2, Open ID Connect, JSON Web Token, ...

GET /comet-otc/orders
X-Merck-APIKey: 4ef87329b14a314a9549fa26a5e3931f

GET /comet-otc/v2/orders
X-Merck-APIKey: 4ef87329b14a314a9549fa26a5e3931f
Authorization: Bearer aa355d09e6a9d0f9e791274f163870be

Security

Confidential vs. public client applications

Mobile app --> public

Browser based HTML5 app --> public

Web app (on a remote server) --> private

Security

Always use HTTPS!

Sensitive values should not be sent as a part of the URL.

GET /comet-otc/v2/orders?apikey=aa355d09e6a9d0f9e791274f163870be

Avoid Basic Auth if possible and use OAuth2 instead. Use standard AuthN/AuthZ mechanisms.

Use rate limiting both on for the client side and the backend side.

Validate all input!

REST API Design - theory

By Ivan Novakov

Made with Slides.com

REST API Design - theory

  • 1,748

More from Ivan Novakov