Making APIs fast
learning from 40 years of SQL
@joakimlundborg
github.com/cortex
Wrapp
SQL
in 2017?
at an API Conference?
Is it Web Scale?
http://www.mongodb-is-web-scale.com/
More control to apps
What is 'fast'?
Never let the
human wait
Latency > throughput
How do we make things fast
1 ns cpu cache reference
100 ns main memory reference
10 000 ns 1kB over GB network
1 000 000 ns 1MB from SSD
150 000 000 ns roundtrip to netherlands
1 m = coffe on your desk
100 m = coffee around the corner
10 km = subway to Solna
1 000 km = distance to Amsterdam
150 000 km = half way to the moon
1 s = 1s
100 s = 1.66 minutes
10 000 s = 2.7 hours
1 000 000 s = 11 .57 days
150 000 000 s = 4.75 years
YOU ARE HERE
A brief history of *SQL
- 1970s SEQUEL
- 1987 ANSI SQL
- 1995 The third manifesto
- 1995 MySQL
- 1996 PostgresQL
- 2003 Memcached
- ~2005 NoSQL
- 2005 CouchDB
- 2008 Cassandra
- 2009 Redis, MongoDB
Data warehousing
Star schemas
denormalization
The NoSQL movement
Key/value stores
Lesson learnt
model data after your access patterns
Lesson learnt
KISS
Lesson learnt
Consistency > minimum speed
JOINs are evil
SELECT thing FROM huge_list
JOIN all_the_related_things
const things = await fetch_huge_list();
const result = things.map(thing => {
fetch_all_the_related_things(thing)
})
The N+1 problem
The N+1 ORM problem
The N+1 ORM bad abstraction problem
Query planning is not a backend problem anymore
EXPLAIN GET /items/1
Summary
how to make things fast
Give clients control of what they fetch
Help client developers understand what is slow
Know your access patterns
Know your data
Design your user API experience for low latency
Design for cacheability
Design for prefetching
It is not about the tools
it is how you use them
Text
Thanks!
Making APIs fast
By cortex
Making APIs fast
- 1,320