My CloudConnect 2013 Talk: Infrastructure Automation in the Cloud

1. Innovaon → Execuon → Soluon → Delivered Infrastructure Automationfor the CloudChirag Jog, CTO [email protected] Confidential

2. http://www.clogeny.com© 2013 Clogeny TechnologiesAbout Me•  Strong Experience in DevOps, InfrastructureAutomation•  Built cutting edge products:•  Hybrid Cloud Migration Appliances•  Enterprise Java PaaS•  Multi-cloud single pane of glass Dashboard•  Experience building highly scalable automatedcloud applications•  Leading engineering teamsCo-founder/CTO at Clogeny

7. http://www.clogeny.com© 2013 Clogeny TechnologiesProblems in Managing Infrastructure"   Configuration Drift"   Changes causes Outages"   Wall of Confusion – Dev and Ops"   Non Functional Requirements:•  Backups, security, monitoring, synchronization,performance, capacity planning" Dev,Test and Prod – All Different Environments"   Mystery Machine: On But don’t know why?"   Code Complete != Production Ready

11. http://www.clogeny.com© 2013 Clogeny TechnologiesOld Problem: Infrastructure is Hard"   Solve very little of the problem"   Enterprise Focused"   Large Professional Services Bills"   On-premise"   Acquired by companies with large consultingorganizations"   Google, Amazon, Microsoft built their own toolsbut its secret sauce"   Proprietary Solutions

14. http://www.clogeny.com© 2013 Clogeny TechnologiesNew Problem: Infrastructure isChanging"   Easier to get .. but harder to manage"   Demand is dynamic"   Developers are crucial to Operations"   Web/Cloud services are proliferating"   Manual Configuration no longer a crutch"   Few tools solve all the problems"   No Shrink Wrap software

17. http://www.clogeny.com© 2013 Clogeny TechnologiesAgile Infrastructure"   Infrastructure is Code: Apply the lessons learntfrom Software Development"   Build more Flexible systems and fasterfeedback"   Beyond Continuous Integration: ContinuousDelivery"   10+ deploys per Day – Flickr, 2009

18. http://www.clogeny.com© 2013 Clogeny TechnologiesAmazon.com"   Mean time between deployments: 11.6s"   Max # of deploys per hour : 1079"   Mean # of host simultaneously receiving adeployment: 10,000"   Max # of hosts receiving a deployment: 30,000

20. http://www.clogeny.com© 2013 Clogeny TechnologiesAgile Infrastructure Techniquesu Configuration Managementu Version Control Everythingu Build from sourceu One step deployu Monitoringu Continuous Deliveryu Work Together – Dev and Ops

21. http://www.clogeny.com© 2013 Clogeny TechnologiesWhat is Infrastructure (again) ?• Nodes• Networking• Files• Directories• Symlinks• Mounts• DB SchemaInfrastructure is a collection of resources:•  Routes•  Users•  Groups•  Tasks•  Packages•  Software•  Services•  Configurations•  Stuff

22. http://www.clogeny.com© 2013 Clogeny TechnologiesInfrastructure as Code (really?)package "ntp" do!action :install!end!service "ntpd" do!action [:enable,:start]!end!template "/etc/ntpd.conf" do!source "ntpd.conf.erb"!owner "root"!group "root"!mode 0644!action :create!variables(:time_server =>“time.example.com”)!notifies :restart, “service[ntpd]”!end!

23. http://www.clogeny.com© 2013 Clogeny TechnologiesConfiguration Management"   Infrastructure as Code"   Put systems into a known state"   Audit and enforce consistency"   Manage server lifecycle"   Reason about services, instead of systems"   Apply dev-test-prod cycle to infrastructure

25. http://www.clogeny.com© 2013 Clogeny TechnologiesVersion Control Everything"   Network Configurations"   System Configurations"   Application Configurations"   Application Code"   Database Schema"   Documentation"   Anything else that matters!

26. http://www.clogeny.com© 2013 Clogeny TechnologiesBuild from Source"   Build Application and Infrastructure fromsource" Dev,Test and Prod in sync"   Test from a known state"   Scaling"   Disaster Recovery

28. http://www.clogeny.com© 2013 Clogeny TechnologiesMonitoring"   Define ‘Normal’ Conditions"   Monitor Baselines, Charts, trends"   Monitor, Monitor and Monitor !Ref: http://consultingblogs.emc.com/blogs/gracemollison/image_thumb_123A5775.png

29. http://www.clogeny.com© 2013 Clogeny TechnologiesContinuous Integration"   Check in regularly"   Create an automated and comprehensive testsuite."   Test New builds"   Assert Services are running"   Deploy early and often"   No manual steps!"   Never go home on a broken build"   Build a pipeline!

30. Poll MasterandCompileRun UnitTestsPackageDeployRunSmoke/IntegrationTestsRelease

32. http://www.clogeny.com© 2013 Clogeny TechnologiesWork Together – Dev and Ops" Dev and Ops see the same thing, in the sameplace"   Share Metrics"   Share the Repository•  Configs in sync with Application Code"   Minimize Surprise"   Always ship trunk/master

34. http://www.clogeny.com© 2013 Clogeny TechnologiesWork Together – Dev and Ops" Devs need to understand the infrastructure"   Ops needs to understand the Application"   Manage Flows – Avoid Fires"   Build a culture – Avoid the blame game"   Break the silos and work together!"   Collaboration: not only when things go wrong"   No Rock star mentality

37. http://www.clogeny.com© 2013 Clogeny TechnologiesDevOps"   Processes and Tools•  Automated infrastructure•  Shared version control•  One step build and deploy•  Shared metrics"   Culture•  Respect•  Trust•  Healthy attitude about failure•  Avoiding Blame•  Superhero Culture

40. http://www.clogeny.com© 2013 Clogeny TechnologiesReferences"   Dev2Ops - http://dev2ops.org"   Flickr’s 10 deploys per day -http://www.slideshare.net/jallspaw/10-deploys-per-day-dev-and-ops-cooperation-at-flickr"   Agile Infrastructure -http://www.slideshare.net/littleidea/agile-infrastructure-agile-2009" DevOps – The war is over -http://www.slideshare.net/jedi4ever/devops-the-war-is-over-if-you-want-it" DevOps, the future is here –http://www.slideshare.net/KrisBuytaert/devops-the-future-is-here-its-just-not-evenly-distributed-yet"   Jesse Robbins’s DevOps talk - http://www.slideshare.net/jesserobbins/devops-interop-las-vegas-jesse-robbins-opscode

My CloudConnect 2013 Talk: Infrastructure Automation in the Cloud

Clogeny Technologies

Transcript