Continuous Integration with Chef

at

Sam Brown
sam.brown@blackboard.com
@SamuelBrownIV


Engineering services - Operations



What do we do?



EVERYTHING!





Infrastructure as code

to the rescue!


"Software Server Craftsmanship" - Mike McGarr, DCCD

Let's talk about 





Continuous Integration

FOR SERVERS?

Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.




Chef CI Pipeline


Continuous Integration Server



https://wiki.jenkins-ci.org/download/attachments/2916393/logo-title.png?version=1&modificationDate=1302753947000

Our Cookbooks



Version control


  • Yes, please.
  • Store ALL source changes
  • Nothing should be on your server that isn't in VC!
  • Trigger CI job via polling or commit-hooks


http://git-scm.com/images/logos/downloads/Git-Logo-1788C.png


Dependency management


Berkshelf


Like Maven for Chef!


Berksfile

site :opscode

metadata
cookbook 'zabbix', git: 'https://github.com/laradji/zabbix.git'

Metadata.rb

depends "zabbix", ">= 0.0.42"
depends "postgresql", "3.1.0"
depends 'simple_iptables', '0.3.0'

Gemfile

source 'https://rubygems.org'

gem 'berkshelf', '2.0.10'
gem 'foodcritic', '3.0.3'
gem 'strainer', '3.3.0'
gem 'chefspec', '3.0.1'
gem 'test-kitchen','1.0.0.beta.3', :group => :integration
gem 'kitchen-vagrant', :group => :integration


Static Analysis


knife test - syntax and validation



foodcritic - Linting and code standards

knife test

knife test           | bundle exec knife cookbook test bb-crowd-postgres
knife test           | checking bb-crowd-postgres
knife test           | Running syntax check on bb-crowd-postgres
knife test           | Validating ruby files
knife test           | Validating templates


foodcritic

FC001: Use strings in preference to symbols to access node attributes
FC002: Avoid string interpolation where not required
FC003: Check whether you are running with chef server before using server-specific features
FC004: Use a service resource to start and stop services
FC005: Avoid repetition of resource declarations
FC006: Mode should be quoted or fully specified when setting file permissions
FC007: Ensure recipe dependencies are reflected in cookbook metadata
FC008: Generated cookbook metadata needs updating


Unit testing


Test individual recipes and cookbook convergence


we're using Chefspec

(works with Berkshelf too!)

chefspec example


require 'chefspec'
require 'chefspec/berkshelf'
describe 'git::default' do   let(:chef_run) {  ChefSpec::Runner.new(platform: 'rhel', version: '6.3') chef_run.node.set['git']['version'] = '1.8.7' chef_run.converge(described_recipe)  } it 'installs git' do expect(chef_run).to install_package('git') end end


Integration testing


Test that a cookbook run succeeds

Run tests against a fully configured node


we're using test-kitchen & serverspec

.kitchen.yml

driver_plugin: vagrant
driver_config:
  require_chef_omnibus: true

platforms:
- name: rhel-6.3
  driver_config:
    box: rhel63-latest
    box_url: http://fileserver.local/rhel63-x86_64-latest.box
    network:
      - ["forwarded_port", {guest: 80, host: 8080}]
    customize:
        memory: 2048

suites: - name: default run_list: ["recipe[bb_crowd]"] attributes: {}

serverspec example

require 'spec_helper'describe 'Blackboard Customized Atlassian Crowd' do    context "crowd installed and ready" do
    describe package('atlassian-crowd') do
      it { should be_installed }
    end
    describe service('crowd') do
      it { should be_enabled }
      it { should be_running }
    end
  end  context "Java installed" do
    describe command('java -version') do
      it { should return_stdout /java version "1.7.[0-9]_[0-9]{2}"/ }
    end
  end  context "iptables configured" do
    describe iptables do
      it { should have_rule('-A PREROUTING -j   crowd').with_table('nat').with_chain('PREROUTING') }
end


publish cookbook


We use 'berks upload' for two reasons

  1. Automatic upload of cookbook dependencies
  2. Automatic version freezing to prevent developers overwriting

Publishing from Jenkins ensures server matches version control
(no rogue workstation pushes)


What's next?


VM Image Pipeline with Packer

Testing against Linux Containers (Docker or LXC's)

Pipeline for Roles

Pipeline for Environments


Thank you!





Continuous Integration with Chef

By samueltbrown

Continuous Integration with Chef

  • 4,820