Snapshot Testing the Hard Way

Gleb Bahmutov aka "the sub"

glebbahmutov.com github.com/bahmutov

glebbahmutov.com/blog/tags/testing/

@bahmutov

KENSHO

Technology that brings transparency to complex systems

Harvard Sq, WTC NYC

https://jobs.lever.co/kensho

https://github.com/kensho

@bahmutov

Tests!!!

@bahmutov

source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-

Tests are scaffolding

source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-

Customer does not pay for tests 😡

source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-

Once building is finished and sold...

source: http://www.citiconstruction.net/apps/blog/show/43096160-citi-scaffolding-pte-ltd-

You keep maintaining the tests! 😡😡😡

Test types

unit tests, unit tests, unit tests, unit tests, unit tests, unit tests

integration tests, integration tests

end to end tests

@bahmutov

https://glebbahmutov.com/blog/5-times-when-I-write-tests/

Test types: practice

linter, unit tests, unit tests, unit tests, crash reporting

~~integration tests, integration tests~~

end to end tests

@bahmutov

Test types: tools

linter, unit tests, unit tests, unit tests, crash reporting

~~integration tests, integration tests~~

end to end tests

standard

mocha, ava, jest

sentry

API / browser tests

@bahmutov

A good unit test

describe('something', () => {
  it('does this', () => {




  })
})

https://glebbahmutov.com/blog/unit-test-node-code-in-10-seconds/

    const actual = ...
    assert.equals(actual, expected)

A good API test

describe('API feature', () => {
  it('returns items', () => {
    return get(url)
      .then(list => {
        assert.equals(list, expected)
      })
  })
})

https://glebbahmutov.com/blog/rest-api-testing-made-easy/

where does the expected value come from?

Save expected value

describe('API feature', () => {
  it('returns items', () => {
    return get(url)
      .then(list => {








        assert.equals(list, expected)
      })
  })
})

boilerplate!

        let expected
        if (firstTime) {
          save('returns-items.json', list)
          expected = list
        } else {
          expected = load('returns-items.json')
        }

Jest snapshots

describe('API feature', () => {
  test('returns items', () => {
    return get(url)
      .then(list => {
        expect(list).toMatchSnapshot()
      })
  })
})

https://facebook.github.io/jest/blog/2016/07/27/jest-14.html

Jest snapshots

// __snapshots__/test.snapshot.js
exports[`returns items 1`] = `
['foo', 'bar', 'baz']
`

https://facebook.github.io/jest/blog/2016/07/27/jest-14.html

Ava has added snapshot testing

https://github.com/avajs/ava/releases/tag/v0.18.0

Separate jest-snapshot

https://github.com/facebook/jest/issues/2497

@bahmutov

My favorite is Mocha 😞

https://www.javascripting.com/view/mocha

https://glebbahmutov.com/blog/picking-javascript-testing-framework/

@bahmutov

snap-shot

https://github.com/bahmutov/snap-shot

const snapshot = require('snap-shot')
it('is 42', () => {
  snapshot(42)
})

Any testing framework, async, transpiled code, multiple snapshots per test, anonymous tests

@bahmutov

Object snapshots

it('compares objects', () => {
  const o = {
    inner: {
      a: 10,
      b: 20
    },
    foo: 'foo (changed)',
    newProperty: 'here'
  }
  snapshot(o)
})

changed value

new value

https://github.com/taylorhakes/variable-diff

@bahmutov

Text snapshots

it('compares multi line strings', () => {
  snapshot(`
    this is a line
    and a second line (changed)
    with number 42
  `)
})

changed line

https://github.com/millermedeiros/disparity

@bahmutov

Implementation details

const snapshot = require('snap-shot')
it('is 42', () => {
  snapshot(42)
})

// snap-shot/index.js
function snapshot(what) {
  // hmm, which test called me?
}

@bahmutov

Jest / Ava / ...

const snapshot = require('snap-shot')
it('is 42', () => {
  snapshot(42)
})

// other frameworks
function snapshot(what) {
  const testName = this.currentText.name
}

@bahmutov

snap-shot

const snapshot = require('snap-shot')
it('is 42', () => {
  snapshot(42)
})

// snap-shot/index.js
function snapshot(what) {
  const stack = (new Error('')).stack
  // snapshot (node_modules/snap-shot/index.js:20:10)
  // anonymous (my-spec.js:3:2)
  // ...
}

https://glebbahmutov.com/blog/accurate-call-sites/

@bahmutov

use any name!

Abstract Syntax Tree

@bahmutov

https://astexplorer.net/#/gist/18c2791181c5d49d274b92e379730d41/afac95446cb7fe876faed4b2e8d0aa4f571cf71c

location around

call to "snapshot"

const falafel = require('falafel')
const callLocation = {line: 3}
falafel(source, options, node => {
  if (node.type === 'CallExpression') {
    // if node is around callLocation
    // found it!
  }
})

Different uses of Abstract Syntax Tree in "real world" https://glebbahmutov.com/blog/tags/ast/

Parsing and traversing AST is simple with https://github.com/substack/node-falafel

http://slides.com/kentcdodds/a-beginners-guide-to-asts#/

Once we know the test name

Saving / loading /diffing values is simple

https://github.com/bahmutov/snap-shot-core

const snapShot = require('snap-shot-core')
const what // my object / value
const out = snapShot({
  what,
  file: __filename,
  specName: 'my test',
  store,
  compare: compareFn,
  ext: '.test'
})

Dynamic data?

const snapshot = require('snap-shot')
it('returns most popular item', () => {
  const top = api.getMostPopularItem()
  snapshot(top)
})

does not work 😩

Dynamic data!

const snapshot = require('schema-shot')
it('returns most popular item', () => {
  const top = api.getMostPopularItem()
  snapshot(top)
})

works 🤗 !

Schema-shot - snapshot testing for dynamic data

https://glebbahmutov.com/blog/schema-shot/

Stores schema

schemaShot({id: 'd13ef'})

exports['returns most popular item 1'] = {
  "$schema": "http://json-schema.org/draft-04/schema#",
  "type": "object",
  "properties": {
    "id": {
      "type": "string",
      "required": true
    }
  },
  "additionalProperties": false
}

Schema-shot - snapshot testing for dynamic data

https://glebbahmutov.com/blog/schema-shot/

Validates object against schema

schemaShot({id: '7fe101'})

Schema-shot - snapshot testing for dynamic data

https://glebbahmutov.com/blog/schema-shot/

Error: schema difference
  data has additional properties
  data.id: is required

schemaShot({uuid: '66635'})

✅

Future plans

file-system-shot
network-proxy-shot

Help wanted ...