You know, that project that you started with good intentions. You wanted to do well; you tried your best to meet everyone's expectations and have a good time along the way. Remember the scene where our hero Clark Griswald, burdened with many concerns, absent-mindedly ties Aunt Edna's dog to the bumper of the family car. As soon as they pull away, and for the next 100 miles, his car is making a noise that bothers him, but he doesn't know what it is. A funny scene follows where he tries to cover up for the fact that the dog has, well, worn off the end of the rope.

Have you ever done something similar on a software development project? You know that throwing out proven process will be hazardous to the projects long-term health, but you do it anyway, with all good intentions of fixing problems later. Despite my best intentions, I recently tied good practice to the bumper of my project, and six months later realized that my process had...well...worn off the end of the rope. This article was written based on my experience; it is my hope that by reading this others might avoid killing Aunt Edna's dog.

Steve McConnell has written an excellent book, Rapid Development: Taming Wild Software Schedules, in which he outlines some common pitfalls in a rapid development project. Unfortunately, no one on our team read this book until the pilot release of the product was available. We fell into many of the pitfalls Steve mentions in his book. The seven that directly affected our project were undermined motivation, weak personnel, shortchanged quality assurance, lack of user input, insufficient planning, inadequate design and working in noisy, crowded offices.

In March of 2000, I began working for a startup company on the development of a new system that would be deployed on the Web. My client was very anxious to have a working prototype available as soon as possible. They needed something to demonstrate the product to investors and potential customers. Our first prototype was due by the first week of April. An aggressive schedule, but we were up for the challenge.

After working through five requirements gathering sessions, we started developing the prototype. Our team of three put together a database schema and story-boarded a few of the initial screens. After completing an initial rough guess, we threw together some feature functionality for the prototype.

We met the initial deadline, somewhat surprising considering our development environment. Our client didn't have facilities for us to work in, so we began working at our own office. Our office was traditionally set up for training consultants, not for development projects. We had old workstations, and even our ORACLE server was a 250MHz PC with 120MB of RAM.

The team was highly motivated for the first few weeks after the initial deadline. One of our developers lived 100 miles away. She was so motivated that when she started working with us on a Friday and saw the pressure we were working under, she decided to stay and work over the weekend.

From the very beginning, we planned to develop a prototype that could be used as a demo and then redesign the system with a middle tier in the architecture. As we frantically tried to get as much done as possible in a short period of time, we would say things like, "This is rapid development mode. Just make it work for now, and we'll fix it when we rewrite it." When our client saw something working that only took us three weeks to build, they wanted to release a pilot version of the product in the middle of June (10 weeks later).

To meet this new deadline, we chose three methods to push back the schedule: first, we trimmed down the requirements; second, we threw out any formal process, including any redesign; and third, we worked 70 hours per week. As we made these decisions, there was a haunting noise: could we be dragging something behind us?

When the June deadline, came we were still waiting for our client to decide on a third-party product for a major component of the application. We knew this was on the critical path, but we allowed them to stall on that decision, taking it on the chin to please the customer. Also, as we continued to plow ahead and put more poorly integrated pieces together, we discovered that we had taken out so many requirements, the system had become practically unusable.

We knew all along that the product never really had a solid vision. Our hope for a concrete product conceptualization was never realized, as our key point of contact never clearly communicated to us what the system should look like when it was finished.

No one on our development team had business experience with the market that our software was targeting. We scheduled meetings with people who knew the business, but key people were generally unavailable. So, we would guess and try to keep moving forward. As Clark Griswald tried to cheer his family up after a series of crazy and sometimes hilariously bad circumstances, we kept ignoring the real problem and continued on with the work. This, of course, led to a lot of rework on a couple of major components and slowed us down.

The next major delivery date was set for the middle of August. Before then, we were to integrate all of the third-party products into our system, and test it enough for a pilot release. When the deadline came, our client did not have a production environment available for us to release our software, and we continued to have integration problems with various products for two more months.

At the beginning of the project we made some architectural and implementation assumptions. We assumed that the production environment would be based on Linux and ORACLE. As we worked through the design and considered implementations of third-party components, we found that most of the viable solutions were implemented on Windows NT, and we would have to interface with COM objects. We spent quite a bit of time designing a few elaborate communication methods to handle the interface from our Linux server to our NT server. As fate would have it, the final database platform changed to Windows 2000 an with SQL server.

If we would have redesigned the system and implemented a middle tier into our architecture four months earlier, porting from the ORACLE to the SQL server would have been much simpler. More than likely, we would not have adversely affected the final release date, and we would have had a much more robust system. Our system was finally ready for pilot release on October 1.

This whole experience taught us lessons in rapid development that had been learned before. In our experience, the following dynamics played an extremely negative role in our project.

As noted earlier, the team's motivation at the beginning of the project was very high. We knew our team was top notch; each team member brought considerable software development experience that promoted our confidence at the outset. As the project went on, there were a few things that undermined this motivation.

The big motivation killers were the many long hours of work and human life itself. We were all working about 70 hours a week. It is a rare exception to find an entire team who can handle this workload for many weeks at a time. One key developer was trying to finish graduate school; another had to go to India for 6 weeks due to a death in the family; and a third left our consultancy for other opportunities in the middle of the project. These are the things that no project manager can predict or control, but as seasoned software people know, they happen on every project.

We had an extremely slow procurement process. For the deadline's sake, we were forced to use evaluation copies of software for development until full licenses could be purchased. Imagine the frustration when, with one week to go, one of our licenses on a key tool expired. When we finally did get a proper infrastructure going, motivation improved significantly.

In retrospect, the best thing we did to foster motivation was to buy toys --dart guns, a soccer ball and goals, etc. This allowed the team to play together as well as work together. I am surprised to hear of Fortune 500 companies that are only recently beginning to understand this. One key thing for managers to realize is that toys foster coworker relationships that are made when the team plays together. People love to work together, but they get tired. Giving them an opportunity to play together oils the machine and makes the whole team function together more effectively.

But, the most significant lesson we learned about keeping motivation up is to take the time to build a quality product. We compromised our deadline and as a result, our product was not as robust as it could have been, and the team knew it. People like building something they are proud of, and sloppy work never promotes pride. Had we planned for our team to work 40 hours per week, we would have kept them well-rested and creative. This affects software construction habits as much as anything else in life.

In addition to poor team morale, we were also not a very solid team. No one was familiar with the development tool we were using before we started. To our advantage, we had a lot of relevant experience, so the learning curve was actually fairly short to become productive. However, most tools are learned over time, and we did have to restructure the code quite a bit as we became more familiar with the environment. Having someone provide some guidance at the beginning of the project would have made a big difference in how we initially structured and managed the code, and it would have helped greatly with the more esoteric pieces of the application.

Every application with a graphical user interface needs some guidance from a graphical designer at various points throughout the project. We did not have a reliable person to give us input at crucial points of the UI design, so we guessed, using our opinions of other web sites we had seen and similar past experience from developing desktop software. Web-based applications are different from desktop applications, though, and it would have been nice to have an experienced person giving us some guidance.

Our worst personnel issue was that team members kept changing. We only had two developers who worked on the project from start to finish. We would bring a few people on at different times for specific jobs. If a person did a good job and blended well with the rest of the team, we worked hard to keep them on the team. If not, we let them go. (That is one nice thing about working for a consulting company, people expect to change jobs with no hard feelings.)

Our experience is practical affirmation that rapid development projects need smart and capable people. By not waiting for the right person for our team, we had a lot of activity (getting them up to speed), but not necessarily a lot of productivity.

Another area where we dropped the ball was in quality assurance. We let our client know early on that we needed someone to test the software as we produced new releases. The product manager was supposed to test the software for us, so he didn't want us to hire someone to test full time. When the product manager left the company, we had no one specifically delegated to test. Without a good tester to work on our project, we thought a good way to continue would be to have two of the senior people on our team test the system and check for inconsistencies. This worked fairly well under the circumstances, but we found quite a few bugs when we went to pilot that would have been found earlier by a full time tester.

Performing code reviews is a good practice that encourages quality and is helpful in maturing junior team members. One of our junior developers never contributed to his full potential, in part, because he did not have the knowledge needed to build a software product. Code reviews would have facilitated his learning, and by the end of the project he may have been able to contribute in a much stronger way. In a rapid development project, formal code reviews encourage quality. More importantly, reviews facilitate the growth of all the developers on the team, thus enabling individuals to perform at a higher level and the project to run smoother.

Each team member needs feedback at various times throughout the project if we expect the best use of the rapid development process. By having a full-time tester and requiring code reviews, you implement a plan for this communication to take place. Without the formal structure, the feedback will not be timely, if it happens at all.

User input is vital to keep productivity high and avoid feature creep. As noted earlier, our client did not give us a lot of direction for this product. Therefore, we would write code for the components that we understood. Then we would schedule more requirements meetings to understand the next part of the system to be developed. As a result, we didn't have a complete picture of the product until it was near completion.

When the client's product manager quit in July, we had no input from our client for over a month. This contributed to feature creep and the discovery of a significant architecture change late in the project. To have a rapid development project run smoothly, the project needs solid direction and managers who are not averse to making decisions.

When this project started, we didn't take the time to plan our approach. We knew that we were going to have to rework parts of the system, but we didn't realize the impact those changes would have on the overall integrity of the system. Our only plan of attack for working in rapid development mode was to code as fast as possible.

This project would have been ideal for utilizing extreme programming methodologies. Since we were never confident in the requirements, we implemented the simplest solution first. However, we never went back through and cleaned up the code when significant changes were completed. Had we done that, as extreme methods recommend, moving forward would have been much easier.

Since we didn't do what we were supposed to, release cycles were always difficult. It would take us four or five days to get all the components fully integrated and operational in our deployment environment. The system would have been much more robust if we had utilized another key component of extreme programming: unit and integration tests on a daily basis.

Ultimately, we did not have any plan to achieve rapid development, and so we hit a number of unnecessary speed bumps along our way. rapid development doesn't just happen. You need to utilize proven approaches to enable the project to run as smoothly as possible.

At the beginning of the project we had a high-level project plan that basically said, "Coding--four months". The technical leadership knew this was not a good idea, but we were under so much pressure from our client that we disillusioned ourselves into believing that we could operate in code-and-fix mode and get away with it.

In the first week, we had our database design somewhat stabilized and a storyboard laid out for our application. Then we started coding. Because we were developing a prototype and trying to get something out within a month, all other planning and design was thrown to the wind. As soon as we were done with one set of functionality, we started the next.

Operating in this mode was fine for the first month or two, but as we began to increase the functionality of our prototype for the next release, our lack of overall design began to bite us on the heels. We cleaned up the code in places but never implemented the three-tier architecture that we should have. In our desire to keep the boat moving along and keep our customer happy, we didn't think we had time.

As it turns out, our progress was slowed by not taking the time to maintain cohesive, robust code and make the general design changes. Late in the project, we needed to make a significant architecture change to the database. This change would have affected over 50 queries, so instead of being intrusive, we created a work around. This change would not have been as annoying had we implemented a middle tier in our overall architecture. Again, this is something we knew we should have done, but didn't.

Though some people find working with 12 other people in a cubicle a romantic and novel dot-comish experience--it's not. The final pitfall we fell into dealt with office space. Most high quality software development takes a lot of focused effort from each member of the team. It is hard to get the focus needed when there are many people working on independent activities in the same room. One of the rooms we used had five developers and three servers in it. This office was ventilated for only one person, so it got very warm, especially on summer afternoons when sunlight came directly in the windows. The developers who worked in that office threatened to strike in the middle of the project, unless we did something to cool the room off. (Not really, of course, but they were pretty miserable.) Though it is hard to measure, this environment takes its toll on anyone's productivity.

There were many other pitfalls we fell into, such as "planning to catch up later", "code-like-hell programming" and having "unrealistic expectations". We got caught in these traps because we had an inadequate design, poor planning and a lack of user input. Stumbling into one pitfall is like falling into quicksand--other problems emerge and make it very hard to get out of the quagmire.

Only highly professional people can work on a rapid development project and make it run smoothly. These professionals will be more apt to produce quality software in a shorter period of time, especially if they stay rested and motivated. They will also be inclined to follow a standard process: plan aspects of development, get user input and design a solid solution. By avoiding these good practices, we fell into a few potholes, but hopefully we have learned our lesson.

To really produce in rapid development mode, the whole organization must be able to respond quickly to changing needs. The purchasing department must realize that this project is different, and needs special attention so that it doesn't have to wait a month for new equipment. The people on the project need to be highly motivated and extremely competent. The customers should be dedicated to the project and, when needed, be able to give the project their full attention. If this environment is not in place, you may be running your train at full throttle, but there is oil on the tracks, and you will be spinning your wheels.