~“Scrum is only useful in a college programming environment”
When I was in college, doing full-blown project management seemed like a far-off dream that was only available to “professional” software engineers. I had studied hard about the science of software engineering. I read all those books that talked about the 5 phases of software engineering: requirements, design, implementation, verification, and maintenance. It was clear that this was the process that “real” software engineering projects used to achieve success.
In the “real world” of software engineering, a development team would take months putting together the perfect requirements document that would drive the entire development project. There were many different ways to denote requirements and an entire language to describe them: BRDs and MRDs, Use Cases and UML diagrams. These requirements would be paired with a development timeline that would perfectly, and elegantly, lay out the entire development schedule. Requirements documents would flow into Gantt charts, Gantt charts would describe features and dates. PMPs would feel the rush of victory as the development team put together meticulously thought-out designs and fed a team of programmers with requirements as the software architects churned-out technical specifications. I knew in my heart that this is how aircraft control systems and rockets were built!
I read deeply into Frederick Brooks’, “The Mythical Man Month” and the first edition of Steve McConnell’s, “Code Complete.” I learned about requirements and design, estimation and planning. By the end of my studies in college, I felt that I had read enough about the foundations of software engineering that I would one day grow up to be a fine programmer. Unfortunately, none of this mirrored my actual college experience building software products for my various engineering classes.
My real experience building software in college went something like this: time-box everything and do all the “required” stuff first. As a student, you know when your project starts (you get the assignment) and you know when it ends (the due date). There was no “letting the date slip” in this environment. Failure to deliver working code resulted in a grade of “F.” Because you couldn’t let the date slip, it was important to focus on the most critical functions first. Do the 20 percent of the work first that yielded the 80 percent of what the professor was looking for in the assignment. If there was time left over, you would focus on the “polish.” This sometimes left time for other activities like LAN parties, sports, or dating.
This programming work was usually done alone or in small teams; it was the team assignments that were always the most challenging. The team would take the assignment (requirements), break it up to a high-level design, and divide up the work that needed to be complete by feature or technical area. Then everyone would code like crazy to get their part done and merge all the code back together. Then test everything and fix any obvious bugs. This usually left the minimum amount of time for documentation, which was always required for a project submission. So the documents would be very focused on specific aspects of the software and usually done at a high-enough level that if the software had to change at the last minute, the documentation would still be accurate. This documentation would basically be used to remind myself what I did and explain to the professor the purpose of the software and design. I did a lot of diagrams and high-level feature explanation back then. This “just get it done” mindset, where we tried to get the most important requirements from the assignment complete and submitted to the professor before the due date, did not fit into my vision for how software engineering should really happen. This was just “college work.”
Fast-forward to “the real world” of software development in the mid-to-late 90s. Before the Internet bubble burst. Back when IE3 and Netscape were battling it out on the desktop for the soul of the Internet. Back then I was working in Seattle at a company called IDX Systems. IDX had recently purchased a company called PHAMIS in the mid-1990s, and I was a widget in that acquisition. I had worked on a number of different projects, and they were all run in a similar way. Start with the business requirements document (BRDs), put together technical specifications, estimate timelines and put these timelines and milestones into Microsoft Project. This was followed by development and testing, then into the maintenance phase.
This was just like I expected software engineering to be done when I was in college, except for one thing – the results were not what I had expected. Instead of an elegant flow from requirements to software, all flowing through a tightly controlled project plan, I had encountered the realities of software projects. Pinning-down both the requirements and the release dates lead to a number of strange circumstances. Often times the release date slipped. It simply took longer than expected to get all the requirements completed. Sometimes there were “missing” requirements and a special team had to be put together to do the gap analysis, leading to further project delays. Sometimes the projects were released on time, but since the development time ran longer than expected, some “optional” phases were compressed, such as testing or documentation. Was this better than slipping the date? Who knows? At least we were able to say we delivered all the features on time. Sometimes all the pieces would come together perfectly, but this was a rare event worth celebrating.
IDX was a company who had its headquarters in Burlington, Vermont. One day in 1997, the technical leadership in Burlington sent a guy by the name of Jeff Sutherland to Seattle to talk to the engineers about a new way of doing software development. He had this “crazy idea” that software development should be done more like I did it in college and less like we did it in “the real world.” He didn’t call it “the college way,” like I thought it mirrored. Instead, he called it “Scrum.”
Developing business-focused software applications is more like programming projects in college than I would ever have imagined. It is important to deliver software early and often to get that critical feedback needed from the business owners. Did the new features meet their needs? Have their needs/wants changed with the delivery of new functions? The only way that you’ll know if your customer’s will give you an “A” is by ensuring that you are on-track with their needs and vision. I know that in college, I regularly discussed the state of my projects with my professors to try to understand what was needed to achieve an “A” in the course.
Time-boxing the development (called the “sprint” in the language of Scrum) is as critical in professional software development as it was in a college classroom setting. When you say that you are going to deliver software, you’d better be delivering software. Failure to deliver is one sure way to get an “F” from your customers, especially if they have changes to their business processes dependent on a software release. What exactly will be in the hands of your customers? It’s sometimes hard to say, because it is possible that not everything that was put into your backlog was completed. But one thing is likely: if you focus on doing the most critical features first, your stand a good chance of making your customers happy. Something is better than nothing, both in college courses and in software development. And if your customers derive 80 percent of the value from 20 percent of the features (like most customers do), then delivering features early and often will provide maximum benefit, in the same way that this allowed your professor to provide maximum guidance to your programming class software project. Additionally, time-boxing the development cycles will also leave time for other activities, such as keg parties, family functions, or more time at the Aikido dojo.
Time-boxing the development (called the “sprint” in the language of Scrum) is as critical in professional software development as it was in a college classroom setting. When you say that you are going to deliver software, you’d better be delivering software. Failure to deliver is one sure way to get an “F” from your customers, especially if they have changes to their business processes dependent on a software release. What exactly will be in the hands of your customers? It’s sometimes hard to say, because it is possible that not everything that was put into your backlog was completed. But one thing is likely: if you focus on doing the most critical features first, your stand a good chance of making your customers happy. Something is better than nothing, both in college courses and in software development. And if your customers derive 80 percent of the value from 20 percent of the features (like most customers do), then delivering features early and often will provide maximum benefit, in the same way that this allowed your professor to provide maximum guidance to your programming class software project. Additionally, time-boxing the development cycles will also leave time for other activities, such as keg parties, family functions, or more time at the Aikido dojo.
One important factor that allows professional software development to be run more like college is the rise of web-based applications run over the Internet (or intranet) in the browser. Since, in this environment, every user can be upgraded or updated simultaneously with very little effort, it has changed the economics of software deployments. In the web space, there is no ability for users to “opt out” of an upgrade or to have multiple users with different versions of the software. The software is just a single version, and it can be refreshed very frequently so it makes sense to release software as often as possible. In this environment, bug fixes have a lower cost, the benefits of incremental product improvements can be realized almost immediately, and users can be involved in constant, ongoing self-training on these incremental improvements rather than waiting for a huge class on new versions.
In this way, software – like people – is constantly evolving and improving. It is less focused on the “end product,” and more focused on continual improvement; both for the software features and for the customers who consume them. Constant incremental improvement of the software allows the software to closely mirror the needs of the customer and follow them as their needs and desires change. To me, this is the Zen of the Scrum development process; process over product.
