In the past couple of years, we've invested heavily into using cloud computing at Stelligent and for our customers. Since we create software delivery systems for our customers, it's been our goal to automate all of these processes and literally click a button to generate software that is ready to go to production.
While we were able to automate many build, database assembly, deployment, testing, static analysis and other processes, the assumption was often that an instance of some sort (bare or virtual metal) had been setup. Even those that refer to the benefits of Continuous Deployment often make an assumption that the instance is available and configured. At Stelligent, we no longer make this assumption. We're automating the procurement and provisioning of the instance - including installing and configuring servers - using cloud computing.
Having spoken with several colleagues and at conferences about what we're doing, they thought it might be helpful for others to learn how we're using cloud computing resources on our internal projects and with some of our customers. While we've used various cloud providers, I will focus on Amazon Web Services (AWS) for the purpose of this blog entry.
AWS - Elastic Compute Cloud (EC2)
EC2 provides on-demand virtual hardware instances using a pay-as-you-go model: you only pay for what you use. This makes it perfect for development and testing. We developed a web-based application that uses EC2 to provision a fully installed tech stack for Continuous Integration and target environments to deploy to our customers' non-production target environments. We are generalizing the functionality and releasing it as our CI as a Service Platform as a Service product in the coming weeks. So, using EC2, developers can use a web application to click a button and provision CI and target environments on demand. Developers are able to create clean target environments on demand when wanting to test a new feature in isolation against the project's technical stack. The application uses the Typica framework to access EC2's API.
AWS - Simple Storage Service (S3)
We're using S3 to store binary files in which we don't need to track multiple versions or for storing software distributions under development that can be recreated from source in other ways. S3 provides a great platform for storing transient files. Moreover, we're excited about utilizing AWS' recently announced S3 Reduce Redundancy Storage to lower our costs for truly transient files. We use tools such as Elasticfox for accessing files in S3. We're also beginning to use CloudFront for managing distributions for users who don't have access to our AWS account.
AWS - Elastic Block Storage (EBS)
EBS is used to attach data volumes that might need to persist after an EC2 instance is terminated. A typical use case goes something like this:
- Test team uses CI as a Service to provision a target environment
- Test team deploys/installs working application onto this provisioned target environment
- Test team stores test data that want to use for future testing
- Test team terminates the instance using CI as a Service (terminates the EC2 instance)
- Test team provisions new instance using CI as a Service, installs software and reattaches EBS
Stelligent CI as a Service
Our CI as a Service product started to "scratch an itch" as we wanted to install and configure a complete target environment using a cloud infrastructure. We noticed we were doing basically the same thing each time: procuring the instance, selecting instance specs, selecting the operating system, configuring the OS, installing and configuring servers and tools such as Java, Ant, MySQL, Tomcat, (Hudson in some cases), etc. So, we wrote a web application to perform the installation and configuration of this tech stack so that we could get a complete deployment environment at the click of a button. Development teams use this service on projects to setup (and terminate) these environments at the click of a button.
In Summary...
This is meant to be an introduction to some of the cloud computing tools we are using on development projects for ourselves and our customers. It gets really interesting when considering process, usage and other factors when using a cloud infrastructure. Stay tuned for more...
