This is a rough output of the chapter I wrote, reproduced without permission. Later, I hope to clean it up, split it up, and add the figures. It's from mid-98, so it's rather low-tech, which is my general bias. I'm rather proud of its emphasis on the human/political issues.
Chapter 7 of Buying Web Services
Managing a Long Term Outsourced Web Project
Bill Seitz
(was) Medscape, Inc.
fluxent@yahoo.com
Sections:
About This Chapter
(This was written by the book's editor, not me.) Project management is one of the trickiest aspects of creating a successful, long term Web site. It gets even more difficult when you add outsourcing to the mix. This is partially due to the fact that both client and developer are likely to have an infrastructure and procedures in place to manage large scale, long-term projects. Outsourcing forces two distinct businesses to blend their approach to maximize efficiency when working with one another, which is easier said than done. Outsourcing also relies on outstanding and open communication--still a challenge in our high-tech, communications savvy age. Software tools, such as Microsoft�s Project Pro, offer some hope, but they don�t often reflect the dynamic nature of constantly evolving organizations. Finally, good management is hard, and the volatile nature of the Web doesn�t make it any easier.
Surprisingly, there has not been much discussion in books or trade publications on the topic of Web site project management. This chapter is an excellent step in the right direction. It begins with savvy advice on issues such as the evangelism of a project within a company to ensure its voice, presence and survival in an organization (yes, this is one of the many tasks that a good project manager must tackle). The chapter provides an overview of the project design life cycle and offers insight into procedures for ongoing site maintenance, semi-repetitive site development aspects, adding new project components and project portfolio management. Finally, the chapter concludes with an examination of how to manage the implementation of project elements that alter the structure of your Web site, incorporating less significant individual project considerations into your project, and covers the topic of �closing the loop� on Web site developments.
About the Author
Bill Seitz graduated from Cornell University in Operations Research & Industrial Engineering and later earned his MBA in Finance and Marketing from Columbia University before going on to work several healthcare, media, business, consulting and technology firms. He was an initial founding member of Medscape (www.medscape.com), where he serves as Vice President of Technology. Medscape, generally recognized as the Yahoo of medical information, is a premiere Web site delivering patient care information to physicians and other members of the medical profession.
A Healthy Amount of Skepticism
I�m extremely skeptical about outsourcing. Ronald Coase�s theory of the corporation says that you build a company to perform multiple functions when it�s too expensive to manage the communications about integrating those functions across company lines. In a new industry, where both technologies and strategies change quickly and significantly, defining a financial and operating relationship with someone outside the company is like riding the whip on a skating pond. The more important a Web site is to its parent business, the more important to have (a) long-term hands-on stewardship of the operation, and (b) integration with other parts and processes of the business. This is not inherently contradictory with an outsourcing process, but it creates an additional challenge. Do any of us need another one?
Yet outsourcing has been used by some major players (e.g., Charles Schwab). Are they crazy? Well, sometimes. But there can be good reasons to take this approach, such as the cost of bringing talent and expertise in-house. You may need to start delivering results so quickly that you can�t wait to develop the skills. You might anticipate a highly specialized need that may be very rare, and thus difficult to acquire through hiring or training. Or, despite the significance of your Web activities, they might still be sufficiently far from your core business that you don�t want to build up a staff of people with skills too specialized to be applicable elsewhere in your business. Even if a number of these reasons apply to you, just remember that there are costs to outsourcing (beyond your contractor�s profit margin), which should be considered in the planning stages.
My background is as the technical member of the team that founded the Medscape medical Web site (http://www.medscape.com/). We started doing research in late 1994, and launched the site in May 1995. We began as part of SCP Communications, Inc., a medical print journal publisher and meetings coordinator, but later spun out into an independent company now numbering 40 people (the technical team includes eight people). We�ve contracted out development of some client sites, and hired freelancers for some specialized client work. During our own site development, we used only one consultant, and that was to assist in managing the transition from our original Macintosh server platform to Windows NT. We are only now beginning to consider outsourcing for some non-integrated new features.
To frame your thinking, start with defining your title or role. Think of yourself as both Liaison and Project Manager. This emphasizes the two major aspects of your job. On the one hand, you are the link between your business and the contractor you're working with. At the same time, you must remember that you are personally ultimately responsible for the success or failure of this enterprise.
This may seem unfair, since you don't directly control the contractor's staff. This is the tension inherent in outsourcing. Still, you must define and maintain the vision for the project, set priorities for delivering specific results, and make sure that those results get delivered.
Most of this chapter will focus on the contractor and the project itself, but it's worth first considering how your role as project manager relates to your company. Recognize that your project is just one within the corporate ecosystem. (For background on the "ecosystem" metaphor, read Kevin Kelly�s Out Of Control Addison-Wesley 1994 or any of the other chaos/complexity or bionomics books.)
Before spending too much time on the vision for your project, consider the direction of your company. What major goal or value does your project support? For instance, if your company's strategy is based on low-cost manufacturing, and your site makes more customers aware of loss-leader products that the company would rather stop making altogether, then your project may not have a bright future.
In some cases it�s OK for a Web site�s direction to be somewhat mismatched with the company�s direction, as long as (1) your project is accepted from the outset as belonging to at least one group within the company with whom your project's vision is congruent, and (2) you report into that group. If you work in the finance department and show interest in the Web, then find yourself running a site, but the project ends up with a marketing focus, you might want to feel out the idea of reporting into marketing. If marketing doesn't want it that way, then who is really the sponsor for your work?
Be especially wary of having your project deliver nothing but strokes to the egos of senior management (just check out the annual report sites of many of the Fortune 500). At some point, those bosses might need tangible successes, might be replaced, or be embarrassed by a scathing review in a magazine (see www.metoo.com).
How important is your project (explicitly or at least potentially) to the achievement of a goal? You probably don't want your project positioned as the only hope for achieving a crucial goal (this smells like being set up as the fall guy). If the outsourcing success of your project isn't going to matter to anyone, then you are at risk. As companies redefine their focus and/or cut overhead costs, insignificant distractions become targets, and rightly so. Companies and departments change direction, or at least priorities, on a regular basis.
The keys to operating in a corporate environment are (1) always think in terms of what matters to the business, and (2) maintain strong communication with the key players who can support or kill your project. Regularly review your thinking on what matters to the business and how your work supports that. Evaluate whether your project is making or will make a real contribution. Documenting this thought process will both clarify it in your own mind and communicate it to others.
Make sure both internal stakeholders and your contractor share your vision. If your vendor doesn't believe in what you're doing, he or she may end up undercutting you. When it comes to vision, you really must be partners.
Remind both your internal stakeholders and vendors regularly of how the vision for your project fits into the big picture, and what your short-and medium-term plans are for fulfilling that vision. Try to avoid doing this in formal large-audience dog & pony shows (or, if this is important in the culture of your company, don't let it be your primary avenue of communication); meet or chat informally. If short, chatty emails are part of the culture, then do that. If communications are a little more formal, then focus on communicating when specific milestones are achieved. (A specific milestone is �prototype completed and demonstrated to 3 focus groups;� a meaningless pseudo-milestone is �trading system 50% complete�.) In these cases, make sure to place that announcement in the context of the planned overall schedule and how that contributes to the major vision.
Use the Web to document the various perspectives (overall vision, next small step, etc.) of the project. Keep individual documents concise and focused. One of the joys of hypertext linking is allowing the reader to re-achieve context. For example, a status update about a narrow issue can link to the higher-level project component it relates to, which links to the part of the vision which it helps support, etc. A big, long-term project has many complex components, which relate in subtle ways, and anyone not involved on a daily basis is going to lose track of all those pieces.
To make this tactic effective, and for other obvious reasons, try to make sure that your key reviewers are comfortable with using the Web. A formal executive training program will probably not work: I recommend (low-key) guerrilla evangelism. Assuming such key people at least have PCs on their desk and use them regularly, get whatever they need installed on their machines. Offer to show them things. Write little cheat sheets if necessary ("first check that the modem is on � you should see at least one red light..."); don't be condescending, but recognize that tiny details often get in the way of a new user's satisfaction.
Seek and maintain alliances with other players in your firm. Understand how the work you do relate to the goals they're trying to achieve. It�s entirely possible that your goals may conflict with theirs (for instance, your attempts at direct sales online will not sit well with the manager of the reseller channel). Both your efforts should fit together somehow.
Listening is as important as talking. Listen carefully for signs that your audience disagrees with a key assumption behind your work ("online support will result in fewer phone calls"). Or, maybe worse, that they just consider your project and its goals irrelevant.
The above recommendations are not about politics or manipulating executive egos. They're about recognizing that your work has to fit in with other people's goals and priorities. Without this process, you might find that you are meeting your self-perceived objectives only to have your project canceled without warning.
Project Design: The Life Cycle
Your project is going to pass through multiple stages. Work that you do early on will have to be either maintained, integrated into later work, or discarded.
Begin with the "big picture" vision of what you hope to achieve and think about how long it might take to reach that point. Then consider whether you can wait that long to have any "deliverables" (launched features delivering benefits to users and your business). You might envision an online database-driven library of every product produced by your company, fully integrated with your sales support systems. That could take a long time to finish, but if there are a small number of less complex goals that will deliver a lot of value with little effort in a short time, then get those done first, even though you might have to re-do that work later in a more elegant and scalable way. For instance, marketing or the tech support group may be able to identify a small number of resources that they know are greatly needed by users, and it would make sense to get those online quickly, even if they are later re-organized as part of a coherent and more expansive collection. Think of those first steps as prototypes and market research. They can help challenge or confirm your understanding of the audience's needs, and also clarify issues that you will face in the implementation of your more complex product. Plus, if all goes well, early success will reduce concerns that your project may be a pipe dream.
Another approach to the life cycle planning is to clarify the specific goals for different phases of the project. In Crossing the Chasm (Harperbusiness, 1995), Geoffrey Moore discusses a technology company that started out by concentrating on achieving "design wins" (favorable reviews, industry buzz, etc.), then later revenue, and much later profitability. The danger with this approach is that it can be difficult to define objective or measurable goals in the early stages of this kind of project, but a reasonable spin on this idea is to focus on short-term deliverables that test the concept of your project (design wins) and begin delivering value (revenue) even if not through the most efficient (profitable) process. In Medscape's case, this has meant that we have skimped on automation of our production processes (a good deal of our HTML conversion process still involves manually cutting and pasting chunks of template code from cookbooks), because our development resources are better invested in features that are visible to our site's users (an improved search engine, greater personalization features, etc.), rather than in reducing our production costs.
The life cycle planning process should help you refine your plan, but don't expect that you'll have a detailed road map of everything you�re going to deliver over the next two years. Business changes too much, and the Web is still not sufficiently predictable. Again, discuss this plan with your contractor and the appropriate internal staff.
As you begin to move through the life cycle of your project, you'll find that your contractor's tasks will range from the repetitive to the new and unpredictable. You can think of this as a portfolio of activities that have to be managed. It is your responsibility to recognize the different management issues associated with these different types of activities.
Different skills are needed to perform different types of tasks. Some contractors excel at building complex applications, but don�t have the low-end staff to cost-effectively handle repetitive production tasks. Others design great static sites and can code HTML pages, but cannot build an infrastructure that will grow with your site's scope and traffic.
Your monitoring controls must fit the various tasks. Repetitive activities can be measured in volumes of flow (e.g. �3 new documents added per week�) and/or unit cost (e.g. $20 per document). Development of new significant new functions must be tracked against scheduled progress. You may even set up different fee schedules with your contractor to reflect the different components of the project.
As you discover the extent of the maintenance activities needed to sustain the site, you might even reconsider whether they should be outsourced at all. If your contractor has built the appropriate infrastructure for your site, and you've found ways to integrate that into your company's internal activities, it might begin to make sense to "in-source.� Discuss the potential for this with your contractor from the beginning, and be sure to structure the project and its fees so that neither of you has a vested interest in pushing the decision in one direction or the other.
Managing Maintenance Activities
The archetypal maintenance activity is content production. In a publishing or entertainment-model site, it's clear what content is. In other cases, like a technical support site, content might include: product descriptions; product documentation; enhancements such as software drivers or add-on utilities; and records of previous customer questions and their answers. In yet other cases, such as financial data sites, online communities driven by discussion-forum software, or sites where the content varies so significantly in structure that its production is always a technical craft, there may not be any of this manually handled repetitive activity at all.
As previously mentioned, this type of maintenance activity is relatively easy to quantify. How many documents get added to the site each week? What's the average cost per document? How long does it take to get the average document through the pipeline? What's the backlog? Just because you�re able to measure these activities, however, does not necessarily mean that they are useful in every situation, but it's worth consideration, if only to evaluate the performance of your contractor, or predict future costs or other metrics as your project's scale increases.
There are a variety of techniques and tools
available for managing repetitive tasks more effectively. Simple
checklist-style production procedures both improve and help maintain the
quality and productivity of repetitive tasks as the team faces staff turnover
(See Figure 7.1 and 7.2).
Figure 7.1 A simple and effective checklist of on how to check the production quality of a new article before posting it to the Medscape Web site. Documents such as these can be easily created in HTML and accessed by both internal and external teams working on a project.
Figure 7.2 A simple checklist for posting articles to the Medscape Web site (updating various navigation pages). Because these documents are in HTML they can be updated frequently as project management processes evolve throughout the life cycle of a project.
Various tracking software may be used to manage the flow of activity. Our production staff uses NowUpToDate (now called Eudora Planner), a calendar-scheduling program, to manage article conversion (much of our non-database-driven or static content such as journal articles is aggregated from print publishers). Different areas of content are assigned to different calendar categories (such as �AIDS articles� vs. �clinical practice guidelines�), so that they can be scheduled over the course of the month and moved around to maintain a consistent flow in each area. Then various flags are used in the short title of each article being added to communicate the status of various required tasks for that article (e.g. �waiting for links to other sites�). This approach doesn't support structured reporting ("show me all the articles waiting for copyedit"), but it provides visual support for the scheduled rollout process, which is more important (see Figure 7.3).
Figure 7.3: Simple work group calendar tools, such as Eudora Planner, can be adapted to provide everyone working on a project with the opportunity to view due dates, milestones, and stay informed about the status of a Web project.
For processes that take place over a longer period of time, a more structured workflow system might be appropriate, as it can help delays at specific steps in the flow (e.g. in getting final edits back from authors). The original content we develop takes months from solicitation to the final production process, and there may be hundreds of articles at various stages in that process. Being able to report across articles to find those whose status hasn�t changed in a long time avoids the sudden discovery that "nothing's ready to go into production". Workflow systems are becoming more affordable, but you might still prefer to build a lower-tech system with a workgroup database like Claris FileMaker or MS-Access. Just remember that you want to be careful about turning management of the process into yet another project. Medscape uses custom database-driven software since it�s tailored to our processes (see Figure 7.4). If workflow tracking is going to be a big activity in managing your site, you might want to select a contractor based on their existing workflow system.
Figure 7.4 Medscape�s Article Tracking System uses an intranet to provide the status of each article being processed for uploading to its Web site. Medscape uploads several thousand articles a year and current status reports can be generated dynamically by the company�s system.
At some point in the maintenance management process, increased automation opportunities might become feasible. While we've put off writing programs to automate the production of our more complex content, we built the much simpler programs sufficient to make the production a nearly hands-off operation when we started working with a couple publishers of shorter and more simply structured bundles of content (e.g., news stories). In some cases you might suggest that your contractor to build (or buy) these tools on the basis that the results could be applied to other clients� projects, giving the contractor a cost advantage over the competition.
Make clear your expectations of the vendor to use an extranet (an Internet-based system run by a company to support confidential communication with specific outsiders such as vendors, customers, and partners) to make appropriate monitoring information available to you. If his group is generating production statistics internally, then you should be able to get those online, rather than having to attend a meeting to get an update. This might not always be appropriate, so be flexible in your expectations.
You should look for opportunities to integrate repetitive tasks with related activities performed within your company. In other words, if you want to generate a customer support knowledge base from every single customer support case that your staff handles, then it might be worth investing in building an infrastructure that manages this case information in a way that reduces manual management. Having support staff write small text files and save them into a search engine might work for a purely internal process, but having them entered directly into a structured database, and then indexed, might eliminate a future conversion process for the public site.
Semi-Repetitive Projects
Many projects run across various events analogous to a magazine's special issues. In other words, many projects share similar major issues and resource requirements, but each one can involve new twists that warrant attention. At Medscape, one of our custom services, Conference News Online, involves sending medical writers to a conference and publishing online session reports. A new client, for example, might pay for some special interviews with keynote speakers that require a different kind of preparation. In dealing with these new issues you can get distracted from another milestone's deadline. We have a generic plan template to start with, but there are always changes, and sometimes that can require some creative restructuring of the process if there is a particularly tight schedule (when the client is a week late signing the contract, the conference doesn't get pushed back). We use basic traditional project-planning software (AEC Software's FastTrack Scheduler Pro, a cheaper and easier competitor to MS-Project) to track task timeline dependencies.
The key to managing semi-repetitive tasks or projects is to have enough short-term, discrete tasks so that you're meeting milestones on a regular basis to confirm your progress toward the deadline.
Schedule frequent status meetings, either held at regular intervals or based on scheduled milestones. Use these meetings to identify and resolve issues. If your vendor is getting into deadline trouble, you can then redefine your expectations or find options that will benefit you both.
New Project Components
Implementing new features is one of the greatest challenges of Web site development. This level of project management gets you into the heart of true software development. I'll assume that while you may be semi-technical, you've never been a full-time programmer or manager of programmers, so I'll focus on some of the macro issues involved. A great book to read next is Steve McConnell's Software Project Survival Guide (Microsoft Press, 1997); it covers the full breadth of project management and provides plenty of checklists.
No other area of software development requires a greater focus on "time to market" than Web feature development. Rapid development is an excellent form of risk management. In software customized for a specific client, the requirements are understood and structured in advance. In developing retail software, market research can offer insight into your target market, at least after your product's category has been defined (it was a lot harder to design VisiCalc than Excel). Great Web sites are still at the VisiCalc stage: each one is trying to change the world. That's what makes Web site development so exciting, but the flip side is that it�s difficult to figure out what customers want, or more accurately, what they'll use. Since standard market research activities don�t apply in Web development, some people are trying to come up better processes (see Contextual Design by Hugh Beyer and Karen Holtzblatt, Morgan Kaufman Publishers, 1997). Still, the newer processes are very complex, and require that significant time be spent one-on-one with potential users, which is difficult to achieve. In addition, Web projects rarely have the development budgets of, say, the Denver airport, so to tie up multiple programmers for a full year on a single project is rarely feasible.
Some businesses require the development of extremely complex systems which warrant a substantial investment in development. We can break these down into "rocket-science" projects that break new conceptual ground, and "nasty" projects which are complex due to the number of components, rather than the complexity of any of those components individually. Management of both these categories of projects is beyond the scope of this chapter.
The most practical way to test a system�s concept is to build and release a working version, and see how much traffic it gets. The first release should include a number of new features, which should make it easy to measure how much traffic they draw, then you can decide which are worthy of the investment to improve them.
Your goal, in other words, is not likely to be "build a big system,� but rather "put together a bunch of small systems over time.� Think about the various stages your site will go through (design wins, then revenue, then profit), and match the types of projects to the stage. You are essentially a project portfolio manager, which begs the question of how best to build that portfolio. As discussed earlier in this chapter, score some early "wins" to satisfy the in-house skeptics.
Brainstorm by building a huge list of potential projects, even if you know many of them will never be built. Then, a couple of times a year, review the list and categorize projects as "hot" (those you want to accomplish based on the resources currently available), "maybe,� and "probably not.� As a manager, you�ll want to discuss the �hot� list with the people whose input you�ll want or need. Doing the categorizing yourself will avoid a drawn-out consensus process. Of course, you should consider the needs of the various stakeholders involved, and take input from others very seriously while noting that different people have different priorities largely driven by subjective or personal criteria, and that nobody can really predict what people want.
After finalizing "hot" selections, decide in what order to attack them. By starting with the highest priority items, you�re more likely to deliver maximum value by the end of the year. At the same time, stay flexible enough to adjust priorities according to unforeseen factors. A project that might not seem to be a priority may be a good test of, say, a technical architecture that needs to be built for a more valuable but complex project. By starting with a simpler and lower priority deliverable, you can test the architecture and hash out some of the unpredictable issues in a project that can be rebuilt relatively easily. Staying flexible might also mean delaying a valuable project because resources outside your control (designers, editorial content, outside partners, etc.) might not be available when you want them, and you don't want to spend time pushing resources that are out of your control.
Priorities will change, and resources will change, but the planning process serves two important purposes: First, it forces you to think through what's important to your site's success; second, the �hot� list gives you ammunition against inappropriate changes to the plan. For example, people will often come to you with new ideas, and your prioritized list gives you a basis for thinking "well, this might be a good idea, but is it a better use of scarce resources than these previously planned projects?" You can also ask that question of other people, and challenge their thinking. Again, I'm not recommending knee-jerk plan-following: just make sure that deviations from your plan are conscious and justified, rather than reactionary.
Selecting Your Site�s Architecture
Project scheduling may be significantly affected by the architectural decisions made at various stages of your site's life. Whether you're considering operating systems, development environments, or shrink-wrapped software, there are similar factors to consider (e.g. cost, quality, reliability, ease of integration with other systems, stability of vendor). Of course, all these decisions are related, because some applications are only available under certain operating systems, etc. Ideally, you want to consider all your major decisions at the same time so you can make a coherent integrated choice, but since many significant elements of your architecture are things you don't even know you need yet, and many of the products you'll eventually evaluate don't yet exist, you�ll have to make some very rough guesses about future markets for software.
You might think that your contractor should make these decisions, but your input into these issues should be significant. If you feel your contractor�s recommendations are based more on his or her familiarity with certain products than on your needs, then it might be time to re-visit your contractor selection. In fact, a good case can be made for working with a consultant to help you in some of the architectural decisions, and then finding a contractor who agrees with you.
Also note that I'm talking about architecture after talking about project prioritization. That's because having a sense of what types of systems you expect to buy/build/integrate on your site will help you to think about trade-offs.
We ran Medscape on Macintosh servers for a year and a half. As I mentioned earlier, we started out as part of SCP Communications, which had Macs on every desktop, and we saw integration of the production process as being important to our ability to bootstrap the operation. By the time we moved off the Mac, we were running two production Web servers, plus a dedicated server for our AppleSearch engine (we still run our email, DNS, and FTP on Macs). I don't regret the decision to use Mac servers, considering that Windows NT was in its infancy when we were getting started, so UNIX was really the only other alternative. I was the sole tech person, both systems administrator and programmer, for that first year; managing a more-complex UNIX environment was not a realistic expectation. Limits to file input/output speed curtail the Mac�s scalability, and as more people build new sites around SQL back-ends that render and deliver content dynamically rather than as static standalone files, the Mac becomes untenable due to its lack of high-performance database servers. (While it�s possible to run a Mac Web server connected to a SQL back-end running on another platform, that just creates another variable to worry about.) But when you're trying out a business model that's untested, I still think the Mac is a great Web-prototyping platform.
Once scalable performance and reliability became significant issues for us, and we realized that we�d want to be able to buy certain high-end dynamic capabilities (ad servers to dynamically insert ads according to business rules rather than just randomly, template-based middleware for developing SQL-based features, etc.) that were never going to be available on the Mac, we knew we had to switch. I found a consultant who had been working with Time Inc.�s Pathfinder site, so he had the chance to see many of our options already in action, since Pathfinder had the budget to buy things and spend time trying to make them work. We ended up settling on Windows NT, a decision I'm still pretty comfortable with. Note that I didn't say "thrilled.� We knew then that Windows NT was not as scalable as UNIX, but we were comfortable that we wouldn't run into problems based on our traffic forecasts. (Just as a quick sidebar, make sure that you do capacity planning calculations based on peak load, not average weekly or even daily load. Medscape doesn't have a huge daily variation: our peak load is typically double our steady daytime traffic, but some sites see huge peaks, based on either special events or time of day, such as lunch hour. We also knew that Windows NT wasn't perfect and that the stability of UNIX systems is highly variable, so there were risks on either side. Having covered those concerns, we chose NT based on hardware cost, software cost, support labor cost, and expectation that Microsoft's dominance in the corporate market would extend into intranet servers, driving many application developers to switch from UNIX to NT as their primary target platform. Software cost differentials have actually decreased as NT's strength has forced down UNIX pricing. Many vendors still start with UNIX, but the momentum of Windows NT is there.
When it came to Web server selection, it was clearly a Netscape vs. Microsoft decision. Performance was comparable. We chose Netscape because we suspected that their philosophy of open standards would make integration with 3rd party applications (such as the previously-mentioned ad server, etc.) easier, while we knew Microsoft would always be torn between working with vendors and competing with them. I still question whether Netscape was the right choice at times, but less because of any current dissatisfaction than from fear that Netscape will be marginalized over time as Apple was.
We bought two major server applications during our initial platform migration: the Verity search engine and the NetGravity ad server. Search engines are rather complex, and you certainly don't want to build one. Many of them are available for free, but you probably don't want to spend a lot of time maintaining or tweaking it. On the other hand, you might find that the vanilla search engines are inappropriate for your purposes. For instance, we define some custom fields in our documents to allow searching based on area of the site, publication date, and whether continuing education credit was available on the content. In addition, we were able to exclude certain pages (e.g. navigation) from the search engine entirely. No thanks, never read it. And I believe it�s about using public search engines like Altavista, not buying one for your own site.
Ad serving was a technology where we were tempted to build rather than buy, because we had specific requirements that would be tough to implement in a shrink-wrapped application. Ultimately, we needed the application on such short deadline that we decided to buy it. An added benefit of buying an ad server is that you�ll have legitimate excuse for not providing an unnecessary number of customized features (which salespeople tend to request) since a shrink-wrapped application has limited capabilities. (Salespeople have a right to ask for what they want, but they also tend to hyper-focus sometimes on narrow areas of functionality that aren�t crucial to their business success but increase the risk of project failure as the software gets overly complex, so it�s in your mutual best interest to avoid this.)
There are other application categories we evaluate periodically (e.g. workflow or content template systems, personalization technology, etc.), but so far have found those applications to be expensive, while the benefits are not proven. Also note that the applications we bought come from reliable vendors with proven track records; you don't want to buy software to implement technology that�s key to your site, and then spend all your time worrying about whether your vendor will go out of business (or get bought out by Microsoft).
To integrate these applications and build new custom features, we needed a variety of development tools. But we didn�t want too many, as that would be a burden on our development staff. The code, such as that handling our authentication control (to use our site you must create a free account and sign-in at each visit) and issuing of cookies, which has to run in the NSAPI (the Netscape API which allows you to customize the core functionality of the server with high performance) at high load, gets developed in C or C++; luckily there isn't too much of this, as it's the most expensive to maintain or change (because C++ is an overly complex language). Applications that are driven primarily by access to SQL-based data (such as our drug database), we've found are best handled with Allaire's ColdFusion, which allows us to focus on the SQL logic rather than having to write a lot of code. And for the applications that don't fall into either of these categories (such as middleware that communicates with a partner�s site to grab and re-organize Medline searches), we write server-side (pure) Java, as it's a well-structured and productive development language. We avoid browser-interface programming because of the unprofessional behavior of the browser companies in consistently supporting any language. On an intranet you can try to force everyone to use the exact same browser (and version of that browser); on a public site code that works with one version of Netscape will break with Internet Explorer, forcing you to write many things twice and spend a lot of time testing).
Again, note that we tend to stick with relatively mainstream, well-supported technologies. This (a) makes us less nervous about vendor viability, and (b) gives us a reasonable chance of hiring staff already experienced in a given technology. For instance, I'm often tempted to use the Python scripting language, which supposedly is as productive as Perl but more amenable to building well-structured maintainable code, but always stay away because of the scarcity of Python programmers. [2001 Postscript: note that I use Python at my current job.]
Managing Individual Projects
My approach to managing individual projects can be summed up as follows: plan for a multi-phased delivery, and try to avoid ever getting to the second phase. In other words, since we're attempting to deliver a large number of projects with limited resources, we'll settle for delivering "good enough" software (see the article by James Bach at <http://www.stlabs.com/testnet/docs/good.htm>), then getting on to the next project. Again, this doesn't mean we produce buggy software, it means we create applications that deliver the core benefits to the bulk of target users. Fred Brooks, in his classic book The Mythical Man Month, Addison-Wesley, 1995 reprint, describes a consultant who was renowned for coming into projects that looked like they�d never end, and bringing them to closure. When asked what his secret was, he said "I just keep throwing out requirements until I get down to a project that can actually get completed.�
Start applying this approach before you even being programming a project, when you write the requirements for it. You should have done some thinking about this during the process of prioritizing your projects for the year, since the complexity of requirements will define the cost of the project, and thus the evaluation of whether it's worth doing. But now it's time to get into the details.
The requirements might result in realizing that a project is much more expensive than you originally thought, forcing a reconsideration of its priority, or even its cancellation. One of the dirty secrets of the development world is that a huge percentage of projects are never successfully completed (and few are finished on time and within budget). So, canceling a project that�s unlikely to be completed before significant resources are invested in it can be a good decision.
Document all the features you plan to deliver. Distinguish between those that must be included in the first release, and those that can be held off for later releases. Then describe explicitly those features you have no intention of delivering (this process also helps you brainstorm about features you don't want to forget about). Structure your requirements around the different series of interactions between your system and its various users (these are referred to as "scenarios" or "use cases,� and there are a number of complex books available on their structure).
According to some formal development methodologies, you�d first finish the requirements document, then start on the design process (data structures, interface, etc.), but I find that making the design part of the requirements process helps keep everything consistent. So I'll typically cycle through different parts of the plan (what does the user want to do? what does the system have to do? how should it look? what needs to be happening behind the scenes?) one requirement at a time.
One of the joys of HTML is that you can easily build a prototype of your system simply by creating a series of static HTML documents for each use case, showing how the user progresses through the scenario (if there are significantly different paths the user might take, or special exceptions that need to be recognized, then you might have to make a nearly duplicate set of pages for each sub-case). Then you can link the documentation of each requirement to its prototype. This will both improve the thoroughness of your planning (you will discover special cases your system must react to, or you might realize that your user is going to be confused half way through a process), and communicate your design to your company peers and the people building the system.
This type of process may not work as well for you, because (a) you may not have the skills and experience to think through the details of the interface, data structures, etc., and (b) your contractor's project manager might want to manage these parts of the process, so the two of you might need to work on this together. Try to avoid the "I'll finish the requirements by myself, then toss them over for you to do the design,� because you'll end up having to cycle through plan changes anyway, so it's best to avoid over-formalizing what is bound to be just a first draft.
Once you (both) think this document is completed (it's never really done until the project is over), you should review it with the appropriate parties. You should have been discussing many of the issues during the writing of the document, but review the final results again anyway.
Another outcome of the requirements analysis and design should be a rough task list and schedule for the project. In "nasty" projects you would go through an extremely complex estimating process, building up an estimate of "function points" from which you would predict total development time, etc. But I've found (a) that our projects are too small to warrant that analysis investment, and (b) too likely to run into ugly surprises due to the immaturity (bugs) of the development tools or applications being integrated. Breaking things down into a reasonable level of detailed tasks (to the extent that they are predictable), however, will help build a calendar of progress.
By the time the task list is ready, the actual programming is taking place. The more complex your project, the more important a role contractor�s processes play. But it's difficult to evaluate this from the outside. You could take Steve McConnell�s Rapid Development book, go through its list of best practices, and ask your contractor "Do you do code reviews?", "Do you having coding standards?", etc., but (a) he or she could easily lie, (b) he or she could go through the motions of some of these practices without actually getting the benefit, and (c) it's difficult to generalize about which practices are most valuable. Therefore, it's probably best not to meddle too much in their internal operations, but I'm going to immediately contradict myself and mention two particular practices which I've chosen to implement in our group, just because I thought they were the most important place to start.
Version-control software lets developers track changes in source code or other files (such as documentation) as the project progresses. The software can analyze differences between the different generations of code. This makes it easier for the contractor to (a) resurrect draft code that worked but was abandoned in favor of a new approach, (b) re-trace the steps of a programmer who leaves, so his or her replacement can see what was built over time, and (c) create an audit trail for tracking when a particular component was built or changed, and sometimes why that was done. In the case of project documentation updates (like changes in requirements), it's again handy to have an audit trail. The Versions product from StarBase (http://www.starbase.com/) is probably the most affordable market leader.
A defect tracking system is helpful for managing bugs, from when they are discovered through various cycles of repair. Even in relatively small projects it's easy to forget about a bug (the programmer might remember to fix it, but the tester might forget to re-test it), so some sort of structured log is helpful. There are a variety of high-end systems that can be bought; we chose to build a custom system using SQL and ColdFusion so that it would (a) support our chosen level of complexity, and (b) be Web-based (see Figure 7.5). But, again, it's easy to have this type of system and mis-use it, so evaluating a contractor's development processes from outside when you don�t have programming experience, is rarely a good idea.
Figure 7.5 A report that groups Web site defects by Requirement, enabling project managers to zero in on defects by importance, complexity, and date.
One of our system�s customized features for tracking bugs is to tie them to a test's "phase.� There are certain types of testing which can be time-consuming, and which you want to evaluate only after other tests have been passed. For instance, we'll start by using other members of our development team to test a system, and fix all the bugs found during that process before anyone else in the company tests it. Later on we'll get to more complex challenges such as (a) re-testing an entire system on a variety of browsers, versions, and platforms, and (b) load-testing the system to see whether it breaks when run with high frequency. Then, when the system to goes live (i.e., when it�s installed on the production server for use by the public), we'll (a) test yet again (at least some of the basic features), and (b) watch error logs carefully for the first couple days. We also wrote some code to take our daily server error logs and count the volumes of known error codes (to catch a sudden increase in frequency) and list new codes.
Encouraging feedback from users provides many benefits, from discovering bugs experienced only by browsers you didn�t test, to finding that people can�t figure out how to use certain features that you think are obvious (despite our best efforts at making it obvious that our site is free, we still get email asking �but how much does it cost?�).
On many sites, you�ll see a �mailto� link on a page which makes it easy to send a message to an identified support person in your group (or your contractor�s group). We�ve gone a bit further, creating a feedback link on every page, which takes the user to a form which will generate the email. This allows us to (a) grab the URL of the page the user is coming from, which helps make clear where they were; (b) grab the name, version, and platform of the browser they�re using; (c) let them direct the message to the appropriate specialized support person depending on the nature of the question (content vs. technical, etc.); and (d) provide a few different entry fields to structure the information they give us, helping to get a more specific, and therefore useful, question.
By the way, handling this email can easily become a significant maintenance activity� don�t forget to plan and budget for it.
In addition, we�ve created even more specialized service request forms (�I forgot my password,� �Help me unsubscribe from the �what�s new� email broadcast list,� etc.) for cases where forcing a certain structure to the email message will let us write programs on the other end to automatically process a batch of similar requests.
As I�ve repeatedly mentioned, the reason for my �portfolio� approach to site development is the difficulty in predicting what people will like. It�s almost as hard to figure out whether your site is successful (and why) after the fact! The portfolio approach helps set appropriate and realistic expectations. For example, sites that are designed as profit-making ventures (as opposed to those which promote an off-line business activity, or reduce its costs) have a particular challenge. Despite the �make money fast� aura of the Web, the only people getting rich so far are those who cash out after their unprofitable company is foisted onto an unsuspecting public, or the groups servicing those unprofitable sites (ad agencies, lawyers, investment bankers, consultants, headhunters, etc.). This doesn�t necessarily mean those companies won�t be profitable someday. But nobody can accurately predict how long it will take, and so measuring progress toward that goal is impossible.
To set clearer expectations for your site, start with a �future perfect� vision (Luke Hohmann describes this process a bit in Journey of the Software Professional, Prentice Hall Computer Books, 1996). Studies have found that people generate more vivid visions if, instead of thinking from the present to the future (�how will your site succeed?�) they think from the future to the present and near-future (�imagine it�s 2 years later, and your site is a huge success - what makes it a success, and how did you get there?�). The more you expand your vision into a story, the more detail you�ll be able to create. (Looping back to the beginning of the chapter, this story should help you define the goals of your site, and come up with ideas for what to add to it.) This vivid detail is especially helpful in creating a sense of how things might progress in stages from the current time to that successful future. Having your present-time, critically-thinking self interview your future self can help build the bridge between the two phases.
Having described a progression toward a successful future, you must then try and define quantitative stages of progress. Recognize that a bottom-line budget number can be met or not met for a variety of reasons, so you�ll probably want to develop a series of less-direct but more-descriptive measures. Some of these measures might be based on traffic figures (analyzing your server�s log files), but others might be gathered through other means (for instance, if your site is intended to provide customer support, then you might have your existing phone staffers ask customers whether they�ve tried finding the information on your site yet). For an annual goal, you want big ranges of values that you associate with levels of success (�if, after 6 months, less than 20% of customers calling in have tried the site, then we�re in trouble; anything from 20% to 50% qualifies as being on-track; if we�re above 50% then we�re lucky or heroes�).
As time progresses, compare reality to your planned indicators, and then figure out why you got the result you did. This is where things get challenging: lots of explanations are possible, and it�s difficult using any data source (traffic logs, surveys, etc.) to prove a particular theory. Likewise, figuring out what to do differently in the future to improve your success is more difficult than creating your original vision, especially if your original vision hasn�t been successful (how can you believe your next plan?).
The most credible process for figuring out what to do in the future is, again, to trade numeric precision for broad-stroke indications either supporting or challenging a mental model (back to the story concept). The way to drive this process is through external communication (asking site users, through whatever medium possible, without regard for measurability) and internal discussion (arguing about the implications of feedback gathered so far, asking the next question to be put to the users, etc.).
If you can master the process of investigation and discovery, everything else in this chapter (and book) will fit together.
Summary
(This was written by the book's editor, not me.) Many companies that outsource their Web site fall into what we call the trap of unmanageable moving target. The way companies fall into that trap is by believing that, since the Web is evolving so quickly and forever morphing into greater and more exciting areas, it is best to keep the management of a Web project loose as you move forward in your efforts.
This chapter suggests just the opposite. There are many aspects of Web development that need to be managed, and they can and should be addressed through organized and efficient systems set up and agreed upon by you and your Web developer.
Of course, sometimes the best plans do not work as intended. In our next chapter, �When Things Go Wrong,� we take a look at the likely fail points on a Web project and suggest strategies for avoiding them before they happen...and coping with them if they do.
Checklist
Have you identified clear goals for your project? Share these goals with your developer and your staff.
Identify a list of key people in your organization who need to approve and support your project. Find out how those key people define the success of a project and let them know about your project.
Create a list defining maintenance activities and discuss strategies for managing them. Are there procedures in place, and are those procedures accessible to your staff and your developer?
Create a list defining semi-repetitive tasks and discuss strategies for managing them. Are there procedures in place and are those procedures accessible to your staff and your developer?
Create a list defining new Web site projects and components and discuss strategies for managing them. Are there procedures in place and are those procedures accessible to your staff and your developer?
Create a list defining activities that alter your Site�s structure and discuss strategies for managing them. Are there procedures in place and are those procedures accessible to your staff and your developer?
Create a task list and work with your developer to set up a schedule.
Create a testing schedule and work with your developer to set up a process for implementing it.
Create a process for soliciting and evaluating feedback for your site.
Institute a formal procedure to review your deadlines, successes and failures. Always remember the key to project management is to �Close the Loop.�