PSI

While doing some more research on the web for the elusive probability of success of a project, I came upon the PSI. PSI stands for Probability of Success Indicator for a project. I have not gone into details of the same but apparently, it is a scoring system which can predict the success of a project. May be this is what I am looking for. It is in a book called How To Run Successful Projects III: The Silver Bullet by Fergus O'Connell. He has compared the software development project with that of film making and talks about a method called “Structured Project Management” in his book. There is a list of 10 steps which are advocated in the book. They are:

1. Visualize the goal
2. Make a list of jobs
3. There must be one leader
4. Assign people to jobs
5. Manage expectations, allow a margin of error, have a fallback position
6. Use an appropriate leadership style
7. Know what's going on
8. Tell people what's going on
9. Repeat steps 1-8 until step 10
10. The Prize

The author says that any impending disaster in a project can be identified much before it has struck.

Probability of Success for a Project

I have been thinking about something lately: How does a project manager know at any point of time in his project, what is the probability of success for his project? There is definitely a need for such a tool which will help the project manager quantify where the project is headed. This would help him in taking corrective and preventive actions in various areas so as to bring the project in track. As of now, there are various metrics collected for a project. The metrics collected differ in different stages of the project. But if you ask for a single figure which will gauge the probability of success, I am not aware if there is anything which can help. Presently, I would say that most of the project managers go by whatever discrete metrics which are collected and their individual gut feel to assess the pulse of the project.

The more I think about this, the more I am convinced of the need for such a tool. For example, during the initiation of a project, there are certain parameters that should be measured. Similarly, during the requirements stage, certain other parameters needs to be measured. Similarly during the design, development, testing and implementation phases, other parameters are measured. All these parameters should be combined in a way to give us the success measure of the project.

The risk metrics may be the one that represents the probability of success of a project closest. Somehow if the risk metrics can be combined with the review effectiveness at each stage, the defect leakage for each stage and the cost/effort and schedule variance, along with a parameter to quantify the trace-ability, and may be many other metrics, we may be able to come up with a general dashboard which can give us the pulse of the project. It should factor in any dependencies between various parties involved in the project.

Some of the things mentioned above are reported in status reports. But as someone had stated, Status reports are rear-view mirrors. They show what has already happened. It may sometime give us a leading indicator of the success or failure of the project. The Traffic Light reports with Green, Amber and Red colors tell us where the project is headed. But again, it is very subjective and not quantifiable.

Document Effectiveness

In my experience of being in part of development as well as testing projects, I have observed that about 50 – 60% of the effort is spent on preparing various documents like the Functional Specification, Design Specifications, technical specifications, etc. These are most of the time deliverables as part of the project along with various other artifacts. These documents may be the interim outputs which will be used to generate other deliverables / documents downstream or may be the end product itself for some projects. Almost all of the deliverables for a software testing project are also in form of documents: Test Strategy, Test Plan, Test results, Test Closure Summary Reports, etc. Unlike the development projects, where the code is the deliverable, in testing, the documents themselves are the deliverables. Each document is created based on certain inputs and assumptions, thoroughly reviewed may be multiple times, comments and corrections incorporated and finally signed off. Some documents go through many revisions based on updates received after the documents are released. Some others go through various revisions because of comments or corrections that need to be incorporated. I was wondering if there is something like a measure of the effectiveness of the document which can be assigned to each document that is a deliverable. This would help a team to track the quality of their output.

Any technical document can be evaluated for a set of attributes. The most obvious attribute for a document is the presentation and organization. The next general attribute will be the language. However, there are some other attributes which impact the downstream activities. These are: clear scope, coverage, completeness. To evaluate a document on the coverage and completeness, perhaps each section and each functionality of the document should be rated. For technical document scoring, the attributes have to be identified separately for each type of document and evaluated according.

Probably the person doing the reviews and giving the signoff for the document should rank the document for a set of attributes, which can then be given weights and a final score from each reviewer can be obtained. The average of the scores for all reviewers of the document can then be taken as the document effectiveness metrics. In addition to this, the feedback comments should be analyzed and defects related to the document classified. These should also be factored into the document effectiveness metrics. This can serve as a feedback to the author of the document so as to take corrective actions or improvement initiatives for the process as a whole.

Again on Functional Specifications

Then there is the issue of incomplete functional specifications. The functional specification sometimes does not cover many items which the author may have taken for granted. However, this may be required for development and test teams to come up with their downstream design and planning. For enhancement projects, many a times, the author of FS documents, do not start a document from scratch. Various FS documents are available in the organizational repository. Documents with similar functionality forms the base of a new FS. In doing so, the author unknowingly misses out to remove old references and sections in the document. Also sometimes, key areas that are supposed to be covered in the FS are left out by mistake. All these come into light during the review process. However, donkey’s hours are spent by the guys reviewing these FS to figure out what is correct and what isn’t.

More on Functional Specifications

Then there are FS documents which get into details of technical design. May be the author, with a good intention to give a heads up to the development team, points out the programs to be analyzed for impact and enhanced if required. Other times, the author is not aware of the existing system functionality and works backwards from the existing code and prepares the FS by analyzing the existing code. This is not the right way to write a FS. However, in absence of proper supporting documentation, and may be lack of time from the end users to explain the current system behavior, there is no other alternative for the FS author, other than looking into the existing source code to find out what the system does. However, if the author is not technically sound, he does not figure out completely what the functionality is, and just dumps the code into the FS for the development team to figure out what to do with it. This is not the right way to go about it! What the author of the FS does not understand is that the FS is supposed to give the behavioral aspect of the system and not how this behavior is to be achieved. How that behavior is to be achieved is a question for the design/development team to grapple with. A FS as the name suggests should give only the emergent functionality of the system. There should be mention of different interfaces in the system and how the interfaces interact with each other, but rarely ever any code to tell how it may be achieved.

Functional Specifications and Gaps

Most of you would have spent considerable time preparing, reviewing or reading through pages of Functional Specifications (FS) for your projects. In fact, functional specifications are the start point for most of the project leads, technical developers and system testers. If the FS is not of good quality, all the downstream outputs like the Technical Specifications, Functional Test Plans, etc will follow the GIGO (Garbage In – Garbage Out) rule.

In my experience in enhancement type projects, the FS, normally details all that is required from the system. It flows down from the business requirements. This would have been okay if the project is a green field new development project, where everything is to be built from the scratch. But in typical enhancement projects, where there are modifications to be made to an existing code base to satisfy the new requirement, if the FS is a compilation of all that the system should do, it becomes no more than a user manual. And subsequently, when the development teams and the testing teams review the FS to figure out what changes are required to be done or what is to be the focus area of testing, they again have to fall back on the author of the FS or some other person with knowledge of the existing system, to point out the specific area of change. This clarification takes a lot of time and effort which could have been saved if the author of the FS clearly specified the area of changes.

Ideally, the FS should contain a section on the gaps between the existing functionality and the required functionality. For larger implementations, the Gap document should be a standard output for each functionality that is required.

Between the Gaps (Interfaces)

Generally an interface means a junction where there is touch point between different systems. Typically, there is some exchange of data at interfaces. Most of us are aware that in any project, managing the interfaces between various systems is of critical importance. The same is also true whenever there is any transition between anything in a project. Be it a transition whenever the project leader changes or the test manager changes or even in transition between one phase of the project to another. Whenever there is an interface, there are chances of data / information slipping in the handshake and it may pop its ugly head up later in the lifecycle of the project. What does a project manager do to manage these interfaces seamlessly?

Whenever there is an interface, the first thing to do is to define its boundaries. This is pretty straight forward while dealing with IT systems; but may not be so while dealing with processes or people. Difficult thought it may be, that is the prudent thing to do. In terms of project phases, we need to clearly define what is the entry and the exit criteria. These are the ones documented in the project plan. Similarly, for people interfaces, it would mean what are the communication channels, what are the escalation procedures and what are the bare minimum handover activities in case of team member transitioning from the project. When the project starts, there is lot of pressure on the work to be performed and apart from defining the system interface boundaries and in some cases, the phase entry and exit criteria, the human interface part is the most neglected one.

Once the boundaries are defined, the same needs to be validated. We need to refer to this as per the need of the project when the project progresses. Many times, we document everything, but do not make use of the same, again with the pretext of lack of time for these activities. Following this simple step will help us prevent valuable time and effort by keeping the important things from slipping in between the cracks.

3 months x 6 resources or 6 months x 3 resources

The other day I was toying with a hypothetical idea. Suppose a project is estimated at 18 person-months effort. Now, there are 2 options for you. The delivery can be done in 3 months time frame with 6 resources or it can be done in a 6 month time frame with 3 resources. Agreed that most of the time, in the real world that we operate in, the option 2 is not there. But for the sake of our discussion, let us assume that we have the 2 options. Now if you were to execute this project, which of the 2 options would you prefer. One of my team members says that the 6 person, 3 months would be a better choice. The logic being, since the time frame is short, number of changes introduced by business will also be less and hence less complicacy. This may not be true always. Most teams already have a system in place to handle the requirement changes in the project. There was another justification to his choice also. A team of 6 members will command more attention than a 3 member team. And may be people like to handle larger teams.

But, actually we should take the decision by looking at various fact in hand. Say, we introduce one factor here saying that 80% of the team will consist of freshers or with people less that 1 year experience. Would you still be taking the option 1 i.e 3 months, 6 resources or would you move to option 2? My friend’s decision changed with this new fact. Of course, with freshers, there will be time required to train them and make them productive. So a longer timeframe helps. Similarly, assume that you get experienced resources to staff the project, but the project is something which the organization has never attempted in the past. So, do you still go with 3 month x 6 resources or 6 month x 3 resources? Here again the learning factor is critical.

Similarly, the decision can change depending on various circumstances and various facts that are introduced. But my 2 cents is that I will opt to execute the project in the longer duration. This is because, the deliverables will be spaced out in time and hence the team will be more relaxed and thorough in their job. Longer duration also entails the risk of resources going on leave, resigning etc. So these are also the factors that need to be thought of.

Differentiating oneself in project management

Most project manager practice the art of project management with proper planning, good communication and superb execution. But if all project managers do it well, how does a project manager differentiate himself from other project managers? This is where innovativeness comes into play. A project manager has to be innovative in his thinking, planning and executing. He has to constantly push limits of innovativeness to see if things can be done in a better, faster and cheaper way. A project manager should encourage his team to find out ways to increase their productivity. He should tailor processes in such a way that project team members are not bogged down with too much of unnecessary documentation.  The project manager has to find ways to reduce non-value adding activities from the process flow and maximize time spent on value addition.

Execution

Driving the execution: All planning and scheduling, creating MPPs, filling project initiation forms, risk management sheets, budget sheets, and different processes are just meant for EXECUTING the project. Execution is the Key. Project execution is where the rubber hits the road for a project manager. The project manager is the one who drives the project execution, scanning the environment for roadblocks and removing the obstacles so that the project team can have a smooth run. This is where, the project manager has to employ all his skills in negotiating, cajoling, inspiring, driving and if required turning the heat on to get the ball rolling smoothly. This is where he needs to make quick course corrections and certain trade-off decisions depending on the various available information at hand. He needs to ensure that all loose strings are tied up at the end and project is executed successfully.