What is Hudson?
Hudson is a powerful and widely used open source continuous integration server providing development teams with a reliable way to monitor changes in source control and trigger a variety of builds. Hudson excels at integrating with almost every tool you can think of. Use Apache Maven, Apache Ant or Gradle or anything you can start with a command line script for builds and send messages via email, SMS, IRC and Skype for notifications.
In addition to providing a platform for continuous integration builds, Hudson can also be extended to support software releases, documentation, monitoring, and a number of use cases secondary to continuous integration. In short, if you can think it, Hudson can do it. From automating system administration tasks with Puppet and verifying infrastructure setup with Cucumber, to building and testing PHP code, to simply building Enterprise Java applications - Hudson stands ready to help.
Martin Fowler and Kent Beck are largely credited with the first use of the term "Continuous Integration" as applied to software development with Beck’s seminal 1999 book Extreme Programming Explained being the first published work touching upon the idea of creating systems to continuously build and test systems in response to changes in source control. In the decade since this concept was introduced, continuous integration is now an established, standard practice used across an entire industry.
Beck, Kent (1999). Extreme Programming Explained. ISBN 0-201-61641-6.
The idea, put simply, is that software development efforts are much easier to manage when test failures and other bugs can be identified closer to the time they were introduced into a complex system. Let’s examine the differences between a development effort using a Continuous Integration server, and a development effort not using Continuous Integration.
Consider a hypothetical group of 20-30 developers working on a large enterprise application.
This development team consists of 20-30 developers divided into groups of 5-12 developers working on focused features and components in a larger system. One group focuses on the database and a collection of APIs shared by all other groups, one group focuses on a complex front-end web application, and another group focuses on back-end systems such as billing and inventory which interface with a large Enterprise Service Bus (ESB).
In this environment the business drives a number of overlapping development schedules. The web application team tends to work on two week development schedules, and the back-end group performs releases in response to changes in the ESB. Any given month resemble a highly choreographed sequence of code freezes, development sprints, and products releases with little room for error and inefficiency.
In summary, these 20-30 developers are under constant pressure to deliver, and the technical managers are tasked with orchestrating the efforts of these various teams to deploy code to production on a regular schedule. This is what the enterprise feels like in 2011: it’s very busy, and if you happen to find a bit of free time there is always someone who can create a new requirement.
Without Continuous Integration the various teams have to factor in "integration time" into the schedule. Unit tests might be run by individual developers, but integration tests are only executed when the system is ready to be deployed. There’s nothing automatic about the process, and, very often only a handful of developers have the appropriate environment to build and deploy to production.
Each team, each group is responsible for testing the larger system and making sure that their changes don’t interfere with another group’s changes. One week before a release, everything stops so that a single "release manager" can build a canonical build on a single build machine and deploy this build to a testing environment.
Each time one group needs to deploy a specific part of the system, every group needs to stop, synchronize, and test. If the web application group needs to deploy to production, all of the other groups need to synchronize with that group and make sure that all of the tests pass during a build.
This constant need to stop, integrate, and verify introduces a synthetic limit to the scale of development. As development teams scale and grow larger, and as systems become increasingly complex and intertwined your team spends more time on integration and testing.
Without continuous integration, developing modern applications is practically impossible without dramatic inefficiency. There is too much unproductive down-time, and far too much waste. The development group described above would be hard pressed to release anything more than a global update once a month.
With Continuous Integration, the system completes a build, test, deploy, and integration in response to every single commit. If a developer in the web application group checks in code at 2 AM, Hudson kicks off a build, runs unit tests, deploys the code to a new server, and performs a set of integration tests. If this build fails or the tests encounter an unexpected condition, everyone is notified of this failure.
With continuous integration, no one needs to drop everything and run a release build, these builds are generated every single day, and in the most mature environments, a fully tested and verified system can be deployed to production at any time. In other words, when you automate build, test, and verify using a tool like Hudson you can continue developing your applications without having to wait (or synchronize) on some manual build, test, verify process.
Making these processes automatic has another important side-effect, it makes the development process more scale-able. When your teams don’t have to stop to actively test and collaborate with one another, it is much easier to add additional developers to a project. Without continuous integration you have to stop and synchronize release schedules. With continuous integration you reduce the risk associated with a particular software development cycle.
Another way to think about Hudson is in terms of technical debt.
When you amass "technical debt" you are making a decision that yields a short-term benefit while creating a problem to be solved in the future. If you’ve developed code under strict time lines, technical debt is nothing new to you. Every system has some form of technical debt. If you’ve ever sacrificed quality for the expedient solution, you’ve experienced technical debt: "the code had to be deployed tomorrow and the deadline was fast approaching".
From the perspective of builds and tests, it is much easier to develop a system without having to worry about unit tests and integration tests. You could speed up implementation time for a feature by just skipping the tests, but you’ll eventually pay for this approach ten-fold in the form of bug fixes.
Continuous integration along with a commitment to test-driven development helps minimize your exposure to risky technical debt. Because you are running the unit and integration tests every few minutes, you don’t have the opportunity to let problems sit and fester unaddressed. When you develop without continuous integration, you are amassing the potential for technical debt. You can choose to fix bugs as they are introduced, minimizing both the scope and severity of defects by catching them early, or you can develop for weeks and run tests only at the end of the development cycle.
While you might only deploy to production once a month, your Hudson installation deploys once a commit, and when you are dealing with increasingly complex systems this granularity makes sure that any glaring quality detours are confined to a single commit.
The infrastructure that powers applications has grown more and more complex over time. The web application that ran on a production network of 10 hefty Unix machines in 1999 has evolved into an architecture that can potentially span hundreds (or thousands) of nodes on a computing grid. Companies like Apple and Oracle are deploying infrastructure on a massive scale compared to where the industry was operating just a few years ago. This movement toward Cloud-based deployments and Platform-as-a-Service (PaaS) has meant that infrastructure is now more likely to be compiled and deployed on an as-needed basis.
Organizations are using tools like Hudson to setup production deployments and automate the management of cloud-based infrastructure. This carries the idea of Continuous Integration forward into the realm of Continuous Deployment. While the idea may sound risky if you work in a relatively slow-moving industry, social networks like Facebook and social media services such as Flickr have been building complex developer/operations (devops) systems that allow them to deploy code to production multiple times a day.
While this emerging trend is still confined to the largest deployments, the idea of connecting your Continuous Integration server directly to your production systems is gaining support. Hudson can be a critical part of automating these approaches to continuous deployment. Proven integration with tools like Puppet and Cucumber allow innovative devops staff to take continuous integration in some surprising directions.
Hudson can also be viewed as a general purpose scheduler, a replacement for cron or Quartz in your production network. While Hudson is generally viewed as development infrastructure, a class of infrastructure usually considered separate from the rarefied world of production, there’s nothing stopping you from using Hudson as a general purpose scheduler or server running within a production network.
If you have a requirement for scheduled services or if you need to perform a set of on-demand jobs in a production network, install Hudson and you’ve gained a friendly interface and a tool that can integrate with anything in the world. There’s nothing stopping you from using Hudson in production.
Hudson has an evolving community of developers and organizations interested in taking Hudson to the next level of stability and reliability. Following the creation of the Jenkins fork in early 2011, several members of the community decided to commit themselves to reconstituting the Hudson community and moving the project to the Eclipse Foundation. This move promises to create a more stable approach to open source governance while creating a level playing field for interested individuals and organizations.
Throughout its history the Hudson project lacked a formal structure and avoided process. Grown from a pet project, the governance of Hudson was ad-hoc. While this ad-hoc approach scaled for a few years, as more organizations decided to participate and contribute to the project the Hudson community identified a need for more formal procedures on issues such as release management, code provenance, and general technical direction. It is now the desire of the Hudson project to move toward a more formal process and structure with the end-goal of creating a healthy, open environment with well-defined rules, as well as a development process and release processes for building a high-quality and reliable continuous integration server.
Eclipse has a history of encouraging healthy environments for open source projects. It is an organization supported and respected by millions of end-users, thousands of contributors, and hundreds of active companies. In addition to being a great home for the Hudson project, Eclipse is also an important user of the software. Hudson has become an integral part of the development infrastructure at Eclipse, and the requirements of the Eclipse Foundation mirror the requirements of some of the largest enterprises in the world.
The main source of information about Hudson are the Eclipse Hudson website and the legacy website. The sites contain links to resources like the wiki, an issue tracker, the user forum, a few mailing lists, the source code and more. If you are looking for download Hudson or learn about participating in the, you can start with the Eclipse Hudson project’s web site.
At the time this document was developed, the Hudson project at Eclipse was focused on the following tasks:
Some of the larger, medium-term goals of the Hudson project are:
As Hudson has completed the transition to the Eclipse Foundation, the core of Hudson is covered by the Eclipse Public License version 1.0. The Open Source Initiative categorizes the Eclipse Public License as a "popular and widely used license with a strong community". The EPL is a limited copy-left license designed to be a "business-friendly" alternative to the GPL. More information about the Eclipse Public License can also be found on Wikipedia.
If you have any questions about the Eclipse Public License, the Eclipse Foundation also maintains a helpful FAQ about the license. A big reason for the move to the Eclipse Foundation was to make sure that the project was on solid ground for licensing issues. Eclipse takes intellectual property and code provenance very seriously and maintains a strict contribution review process to make sure that every project release complies with a set of licensing standards. If you download it from Eclipse, if it is stored in Eclipse’s source control systems, it is released under the EPL.
While the core of Hudson is licensed under the EPL, Hudson is a container for running various Hudson Plugins. These plugins may be covered by a variety of licenses, and if you are redistributing Hudson along with a set of 3rd party plugins, you should consult each individual plugin project to find a specific license.