AWS Sydney Summit 2013 - Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS

AWS Sydney Summit 2013 - Continuous Deployment Practices, with Production, Test and Development Environments Running on AWS

1. 1. Simon Elisha | Principal Solutions ArchitectCONTINOUS DEPLOYMENTWITH AMAZON WEB SERVICES
2. 2. CONTINUOUSINTEGRATION
3. 3. DEVELOPER
4. 4. SOURCE CODEREPOSITORY
5. 5. SOURCE CODEREPOSITORYPROJECT MANAGEMENTSERVER
6. 6. SOURCE CODEREPOSITORYPROJECT MANAGEMENTSERVERCONTINUOUSINTEGRATION SERVER
7. 7. SOURCE CODEREPOSITORYPROJECT MANAGEMENTSERVERCONTINUOUSINTEGRATION SERVERPICKTASKS
8. 8. SOURCE CODEREPOSITORYPROJECT MANAGEMENTSERVERCONTINUOUSINTEGRATION SERVERSUBMITCODE
9. 9. SOURCE CODEREPOSITORYPROJECT MANAGEMENTSERVERCONTINUOUSINTEGRATION SERVERSCHEDULEBUILD
10. 10. SOURCE CODEREPOSITORYPROJECT MANAGEMENTSERVERCONTINUOUSINTEGRATION SERVERRECURENTBUILDS
11. 11. SOURCE CODEREPOSITORYPROJECT MANAGEMENTSERVERCONTINUOUSINTEGRATION SERVERCODEFETCH
12. 12. SOURCE CODEREPOSITORYPROJECT MANAGEMENTSERVERCONTINUOUSINTEGRATION SERVERCODE QUALITYTESTSTESTRESULTS
13. 13. SOURCE CODEREPOSITORYPROJECT MANAGEMENTSERVERCONTINUOUSINTEGRATION SERVERBUILD OUTPUT
14. 14. SOURCE CODEREPOSITORYPROJECT MANAGEMENTSERVERCONTINUOUSINTEGRATION SERVERDOCSBINARIES /PACKAGES
15. 15. SOURCE CODEREPOSITORYDNSCONTINUOUSINTEGRATION SERVERPROJECTMANAGEMENT SERVERBUILDS
16. 16. KEY = ITERATION
17. 17. ITERATIVELY MODIFY THESYSTEM TO BETTER MEETTHE EXPECTATIONS OFYOUR USERS
18. 18. ON-DEMANDPAY AS YOU GOELASTIC
19. 19. =PROGRAMMABLEPLATFORM
20. 20. AWSCLOUDFORMATIONSTACK-BASED DEPLOYMENTSERVICE
21. 21. CLOUDFORMATIONTEMPLATE
22. 22. {"Description" : "Create RDS with username and password","Resources" : {"MyDB" : {"Type" : "AWS::RDS::DBInstance","Properties" : {"AllocatedStorage" : "500","DBInstanceClass" : "db.m1.small","Engine" : "MySQL","EngineVersion" : "5.5","MasterUsername" : "MyName","MasterUserPassword" : "MyPassword"}}}}
23. 23. "AWS::CloudFormation::Init" : {"config" : {"packages" : {"yum" : {"mysql" : [],"mysql-server" : [],"httpd" : [],"php" : [],"php-mysql" : []}},"sources" : {"/var/www/html" :"https://s3.amazonaws.com/my-builds/build-v4.zip"}}}
24. 24. {"Parameters" : {"KeyName" : {"Description" : "Name of an existing EC2KeyPair to enable SSHaccess to the instance","Type" : "String"}},}
25. 25. CLOUDFORMATIONTEMPLATEPROCEDURALDEFINITIONCreate it programmaticallyKNOWNCONFIGURATIONStore stack configuration insource controlPARAMETERDRIVENDynamic and user-driventemplatesCOLLABORATIONShare templates with easeas just files
26. 26. Template ELBs to frontsecondary cache~100 Nginxsecondary cacheservers2-3 Nginx mid-tier cacheserversStackCLOUDFORMATIONTEMPLATEVIDEO CACHINGINFRASTRUCTURE
27. 27. APPLICATIONVERSIONS&INFRASTRUCTUREVERSIONS
28. 28. CLOUDFORMATIONTEMPLATE
29. 29. TEST ENVIRONMENTS
30. 30. 30,000 REQUESTS / SECOND1 TB TRAFFIC / DAY
31. 31. “…AWS seemed to be the best solutionavailable to allow a small, independentcompany to rapidly develop and test acompletely new infrastructure, and host it.We also loved the flexibility that AWSallowed us, when spinning up smaller testenvironments, for beta trials, QA,localization, and during development. Thelow initial cost was also crucial.”Alex Evans, CTO
32. 32. CONTINUOUSDEPLOYMENTSMALL, FREQUENT CHANGESCONSTANTLY INTEGRATING INTOPRODUCTION.
33. 33. SOFTWARE DEPLOY≠PRODUCT LAUNCH
34. 34. 1.5 BILLION PAGE VIEWS$83 MILLION IN TRANSACTIONS4.2 MILLION ITEMS SOLDOCTOBER 2012
35. 35. 30 DEPLOYS PER DAY= 1 DEPLOY EVERY 20 MINUTES
36. 36. "Production is truly the only place youcan validate your code."
37. 37. HTTP://CODEASCRAFT.ETSY.COM/HTTPS://GITHUB.COM/ETSY/DEPLOYINATOR
38. 38. HTTP://SORCERY>SMUGMUG.COM/
39. 39. AWS OPSWORKSINTEGRATED APPLICATIONMANAGEMENT
40. 40. MODEL, CONTROL AND AUTOMATEAT ANY SCALE AND COMPLEXITY
41. 41. A stack representsyour application. Onestack might be usedfor staging andanother for production.Tell OpsWorks whereit can find your codeand define anyadditional deploymenttasks. OpsWorks willtake care of deployingyour app.Scale your stackbased on time orload. Clone yourproduction stack to adifferent region.Automate workflowsfor common tasks.A layer defines how tosetup and configure aset of instances andrelated resources suchas volumesand software.
42. 42. A stack representsyour application. Onestack might be usedfor staging andanother for production.Tell OpsWorks whereit can find your codeand define anyadditional deploymenttasks. OpsWorks willtake care of deployingyour app.Scale your stackbased on time orload. Clone yourproduction stack to adifferent region.Automate workflowsfor common tasks.A layer defines how tosetup and configure aset of instances andrelated resources suchas volumesand software.
43. 43. A stack representsyour application. Onestack might be usedfor staging andanother for production.Tell OpsWorks whereit can find your codeand define anyadditional deploymenttasks. OpsWorks willtake care of deployingyour app.Scale your stackbased on time orload. Clone yourproduction stack to adifferent region.Automate workflowsfor common tasks.A layer defines how tosetup and configure aset of instances andrelated resources suchas volumesand software.
44. 44. A stack representsyour application. Onestack might be usedfor staging andanother for production.A layer defines how tosetup and configure aset of instances andrelated resources suchas volumesand software.Tell OpsWorks whereit can find your codeand define anyadditional deploymenttasks. OpsWorks willtake care of deployingyour app.Scale your stackbased on time orload. Clone yourproduction stack to adifferent region.Automate workflowsfor common tasks.
45. 45. FREE OF CHARGE
46. 46. 14 BILLION REQUESTS/MONTH50 000 DATABASE UPDATES/SECNO CACHE
47. 47. “AWS OpsWorks gives us the tools weneed to automate operations.We can scale Monster World, one of thelargest Facebook games, to millions ofusers without ever needing more than twobackend developers”Jesper Richter-ReichhelmHead of engineering – Wooga
48. 48. A/B TESTING
49. 49. 24%
50. 50. AWS ElasticBeanstalkAWSOpsWorksAWSCloudFormationAmazon EC2
51. 51. AWS ElasticBeanstalkAWSOpsWorksAWSCloudFormationAmazon EC2CONVENIENCE CONTROL
52. 52. AWS ElasticBeanstalkAWSOpsWorksAWSCloudFormationAmazon EC2CONVENIENCE CONTROLHIGH-LEVEL DO IT YOURSELF
53. 53. LOAD TESTING
54. 54. USING AMAZON EC2 TO SIMULATE2.4 MILLION PLAYERS
55. 55. 11.6sMean time betweendeployments(weekday)1,079Max number ofdeployments in asingle hour10,000Mean number ofhostssimultaneouslyreceiving adeployment30,000Max number ofhostssimultaneouslyreceiving adeploymentDEPLOYMENTS ATAMAZON.COM
56. 56. CONTINUOUSINTEGRATIONCONTINUOUSDEPLOYMENT
57. 57. CONTINUOUSDEPLOYMENT=CONTINUOUSEXPERIMENTATION
58. 58. CONTINUOUSDEPLOYMENT=CONTINUOUSIMPROVEMENT
59. 59. INNOVATE
60. 60. « Want to increase innovation?Lower the cost of failure »Joi Ito
61. 61. SPEED AND AGILITYExperimentOftenFail quickly ata low costMore InnovationExperimentInfrequentlyFailure isexpensiveLess Innovation“ON-PREMISE”
62. 62. 1x QA1x DEVShared across all teams (100+ people)Previous Environment
63. 63. Allow anyone to deploye2eenv for dev & testSupport ContinuousDeliveryObjectives
64. 64. 400-500 instances per day20-30 environments usedfor dev, QA & CIMultiple teams doing CDCurrent State
65. 65. A Tale of Scarcity
66. 66. Limitsare nothelpfulDoes not encourage the rightbehaviour
67. 67. Efficient Usageprovidetoolsassignthe costno hardlimits
68. 68. metrics
69. 69. toolsenable users to bemore efficient
70. 70. One more thing…
71. 71. ARCHITECTURE CENTERhttp://aws.amazon.com/architectureTECHNICAL ARTICLEShttp://aws.amazon.com/articlesPODCASThttp://aws.amazon.com/podcastBLOGhttp://aws.typepad.com
72. 72. CONTINOUS DEPLOYMENTWITH AMAZON WEB SERVICES