One of the most challenging tasks that Project Managers face is Estimating the Effort, Time, Resources and Cost. This is because of the inherent uncertainty associated with many activities.
1. Identifying Project Activities
There are 4 types of activities and what type of estimating approach should be used with each of them. Those are the Stable Activities, the Dependent Activities, the Uncertain Activities and the Unknown Activities.
1.1 Stable Activities
Stable Activities are those that are well understood and predictable. Hence the estimating is usually straightforward. Generally analogous, expert judgement, a parametric model, or published estimating data are used for these types of activities. The appropriate technique is used based on the information available to the project team and set the estimate.
1.2 Dependent Activities
For Dependent Activities the time or effort is highly dependent on some project attribute or characteristic that is not yet known at the time of furnishing the original estimate.
For instance, the amount of time needed to complete testing will depend on whether the test is successful on the first try or whether a retest is required.
For these types of activities, an assumption is made that will drive the estimated effort, time, resources, cost and quality.
This assumption is a risk and should be tracked on the Risk Register. If the assumption is incorrect, the time or cost involved in doing the activity may vary from the estimate.
- If a conservative estimate is used, this is a positive risk.
- If an aggressive estimate is used, this is a negative risk.
For these Dependent Activities, use the 3 Point Estimate, Expert Judgment, or Analogous Estimate.
If a project activity is outsourced and it is a Dependent Activity, consider what assumptions are used by the supplier and do a Vendor Bid Analysis.
If there are a large number of Dependent Activities in the project, factor that into the Project Reserve Analysis.
1.3 Uncertain Activities
Uncertain Activities are the most difficult to estimate. Often there is very little data to support a precise estimate and there are many factors that could affect the estimate. Hence can’t just make one assumption and track that in the risk register.
An example of an Uncertain Activity is a requirements definition task on a Complex project that has numerous stakeholders who have different opinions of what is needed. Getting all of them to agree on the requirements will be an iterative process with the number of iterations being completely unpredictable. Yet if this task is not done well, there are likely to be major problems later in the project getting the stakeholders to agree that the project deliverables have been met.
Uncertain Activities typically are listed in the Risk Register since the timing and cost are impossible to estimate accurately.
For these Uncertain Activities, use the 3 Point Estimate to set the activity boundaries, although Published Data or Parametric Models can also be used to do this. Use Analogous or Expert Judgment to set the actual estimate. These activities must be considered in the Reserve Analysis. Often the estimate on these activities can be improved by breaking-down the work of the activity and conducting a Bottom Up Analysis on that work. This will isolate the uncertain portions of the activity and allows for accurate estimates where-ever possible.
1.4 Unknown Activities
The Unknown Activities can’t be estimated but must be accounted for in the project reserves.
2. Various Estimation Techniques
2.1 Analogous Estimation
Analogous Estimating is one of the most common forms of estimating project activities. This technique uses the experience from previous projects and extrapolates that onto the current project.
This technique is appropriate for those cases where the type of work is similar and the resources doing the work are the same between projects.
- Its advantage is that it is quick and, when the conditions are appropriate, it is usually accurate.
- The disadvantage is that the organization must have similar projects for comparison.
2.2 Parametric Model Estimation
Parametric Model Estimating is a very accurate and easy estimating technique. A formula is developed for estimating the time or resources needed to perform a project activity. The formula is usually based on historical experience. A PMO will often develop the parametric model based on lessons learned on many projects.
- The advantage of parametric model estimating is quick and accurate.
- The disadvantage is that models don’t exist for activities until there is a large experience base for the activity.
2.3 3-Point Estimation
The 3 Point Estimating is used to understand the level of uncertainty embedded within an estimate. In this technique three estimates are generated for the project activity using three different sets of assumptions.
- The first estimate is a best case or optimistic estimate.
- The second estimate is a worst case or pessimistic estimate.
- The third estimate is between the other two and is the most likely estimate.
The way those estimates are developed is by using one of the other techniques such as Analogous or Parametric Model. However, because of the high degree of uncertainty due to the risk assumptions, the three estimates are used to create a boundary on expectations for the activity.
A variation on this technique, the PERT analysis, uses a weighted average of these estimates to create a PERT estimate. When using this approach, the most likely estimate is normally what is put in the project plan but the optimistic and pessimistic estimates are used during the reserve analysis. Also, an activity that has a big difference between the optimistic and pessimistic estimates is an uncertain activity and should be tracked in the Risk Register.
- The advantage of this technique is that it provides boundaries on expectations.
- The disadvantages are that it is laborious – since three estimates must be created – and the most likely is still very much a guess – the actual could be significantly better or worse.
2.4 Expert Judgment Estimation
Expert Judgment estimating is easy to do – provided you have an expert on the project. This technique looks to the expert to create an estimate based upon their understanding of the project requirements. Many project estimates are created in this fashion.
- The advantage of this is that it is quick and if the expert is knowledgeable, it is often the most accurate estimate for uncertain activities.
- The disadvantages are that you may not have an expert available and even if you do, the expert often can provide no solid rationale for their estimate beyond, “That’s what I think it will take to do this.”
2.5 Published Data Estimation
Published Data Estimating is an excellent technique for those activities for which there is published data. In this technique, the activity is compared to the activities for which data exists and the actual cost or durations of the closest comparable activity is selected from the data and used as the estimate.
- The advantage of this technique is that it is very accurate when the project conditions match the conditions under which the published data was generated.
- The disadvantages are that data does not exist for many activities and that the published data that does exist is based upon the characteristics of the organizations who compiled and published the data – which may not correspond with your organization’s characteristics. (For instance you may have individuals on your project that are with different expertise levels and experience than those who were in the projects comprising the data.)
2.6 Vendor Bid Analysis
The Vendor Bid Analysis is a technique used when working with suppliers on uncertain activities. The analysis considers the assumptions the vendor worked with and does a sensitivity assessment on those assumptions. In addition, for effort that the buying organization does not have experience with, they can contract with a consulting firm that has experience to do a “Should Cost” analysis. This “Should Cost” estimate is compared to the suppliers quote to identify any shortcomings.
- The advantages of this technique are that it exposes supplier risk that can be accounted for in the reserve analysis and it increases the confidence in the supplier’s approach.
- The disadvantages are that this can take a fair amount of time and if a consultant is used to create a “Should Cost” it adds to the cost of the project.
2.7 Reserve Analysis
The Reserve Analysis is a fundamental technique for estimating. This technique considers the level of uncertainty and risk in the project and establishes a reserve pool of time, resources, or possibly performance that can be drawn upon to offset the un-estimated issues that arise.
With respect to schedule reserves, allocate the path float that is on non-critical path tasks to provide a buffer around those tasks with the most uncertain estimates. For critical path tasks, either use a conservative estimate or include a “reserve” task that is used to resolve any problems that arise. An example of a reserve task is “bug-fixing.” So include “bug-fixing” task as a reserve task to address the bugs with some amount of time and resources set aside to resolve software development problems.
With respect to resource reserves, set the level of reserve based on the uncertainty in the project and the risks. It is common to have resource reserves from 10% to 20% over the estimated cost of a project for those projects with high uncertainty. Projects with low uncertainty might have 0% to 5% reserve. Allocate that reserve to the areas of highest uncertainty, and allocate a portion of the resource reserve to each phase of the project. This is to better match the actual against the planned timing for the use of the reserve with when it is likely to be used. This avoids unnecessary variance in reports.
With respect to performance reserves, it is used when task deliverables are expanded to address unknown issues. For instance, a typical design with a 50% design margin on software architecture designs so that if there was a slight error in integration or configuration properties, the design would still be robust enough to perform as needed.
Hence the reserve analysis is a guess on the part of the project manager and stakeholders. If your organization has policies regarding reserves, use them. If not, use your best judgment.
2.8 Bottom-Up Analysis
A Bottom Up analysis is a technique to improve the accuracy of the overall project estimate. This technique requires the project team to break-down the work into very small work packages. Generally, the smaller the project activity, the easier it is to estimate because the work scope is very small. All of these estimates of small activities are added up into subgroups and finally into the project total.
- The advantage of this technique is that the estimate is usually more accurate since the work is better understood.
- The disadvantage of this technique is that it is very time consuming, and it may be impossible to break-down activities that cannot be easily defined.
2.9 Project Simulation
A Project Simulation is a way of combining the uncertainty in 3 Point Estimates and Reserve Analysis to understand the likely project outcomes. This requires that the project be populated into a PMIS, being careful to identify all relationships between activities. For those activities that have uncertainty, the degree of uncertainty must be modeled and entered into the PMIS also. The method for doing this will vary based on the PMIS software being used. A simulation add-on is then used to run the software through a Monte Carlo routine. The result will be a distribution of project time lines and costs. Based on the organization’s risk sensitivity, the overall project time line and budget can be set.
- The advantage of this approach is that it provides a global perspective on overall time line and cost uncertainty.
- The disadvantages are that it can take months to do this on a large project and the resulting estimates are only as good as the assumptions that are allowed by the software. In general, the software is seldom able to truly model all of the uncertainty and relationships.
2.10 Fixed Project End Date
In some cases, the project end date is set even before the scope and deliverables are defined. (Consider Y2K projects.) In those cases, a high-level time line is created starting from the end date and going backward to the present time. Given the amount of time allocated for the major activities, the project team considers the needed deliverables and available resources during the time period. Essentially, the schedule is fixed, and the scope and resource are varied so as to create a viable project. Often this will require an iterative estimating approach.
Once the high level plan is established, estimates for the activities are developed and then iterations are done varying resources and scope until a viable estimate can be created. The Risk Register will be dominated by schedule risk items.
Sometimes, an estimate cannot be created. In those cases, the project should not even be initiated, since it is doomed.
2.11 Fixed Project Cost
In some cases, the project total cost is set even before the scope and schedule are defined. (Just recall fixed-bid projects). In those cases, a high-level allocation of the budget is created between the likely project deliverables. Each major activity is then estimated and if the estimate is greater than the allocated cost, the timing of resources or scope and deliverables are varied until the project is able to meet the budget goals. This is often an iterative process that may take much iteration until it completes.
One technique for software project estimation involves the use of regression equations. These equations allow you to calculate an estimate for a particular project metric such as effort or duration by simply inserting the calculated, or estimated, size of your project into the appropriate equation.
This estimation technique is commonly used to produce indicative or “ballpark” project estimates early in the life of a project. This technique is not sufficiently accurate to produce an estimate that could be relied on for quoting or business case requirements. A “ballpark” estimate can be used for an early indication of whether a project idea is feasible, or when you are short of time and detailed information.
The ISBSG has produced sets of regression equations using the data in the ISBSG Repository. These equation sets are published in the Practical Project Estimation Toolkit. You can use these equations to calculate the following project metrics:
- Project Delivery Rate (person hours per fsm unit)
- Effort (person hours)
- Duration (elapsed hours)
- Speed of delivery (fsm units delivered per elapsed calendar month)
Equations are provided for:
- Platform, (Mainframe, Mid-range, PC & Mixed)
- Language Type (3GL, 4GL & Application Generator)
The combination of Platform and Language Type
It is easy to use the ISBSG equations. Having selected the appropriate equation from the tables provided, you insert the functional size of your project and/or the maximum team size, to produce your estimate.
Estimation using comparison allows you to achieve more detailed estimates than can be gained using regression equations. Estimates using comparison are aligned more specifically to the attributes of the project being planned rather than being based on those of the “average” project in the ISBSG repository.
Estimation using comparison involves a technique based on comparison of your planned project with a number of projects in the ISBSG repository that have similar attributes to the planned project.
Comparison based estimation involves considering the attributes of the project to be estimated, selecting projects with similar attributes from the ISBSG Repository then using the median values for effort, duration etc, from the selected group of projects to produce an estimate of project delivery rate and speed of delivery, and consequently project effort and duration.
The steps are as follows:
- Define the platform applicable to your project and identify that subset of ISBSG data using the ISBSG data on the Estimating, Benchmarking and Research CD.
- Define the other attributes of the project to be estimated, (eg Language, tools etc).
- Search the identified sub set of ISBSG data for projects with the same attributes. (There is a tool on the Toolkit CD that does this search for you).
- For each of the planned project’s attributes, obtain the median project delivery rate and speed of delivery for all the projects in the ISBSG Repository exhibiting that attribute.
- Determine the average of the medians of the project delivery rate and speed of delivery.
- The result is your estimate.
Because the resulting values are aligned to the specific attributes of the project to be estimated, they are better estimates of that project’s project delivery rate and speed of delivery than the values obtained from the equations that reflected the ‘average’ project in the database.
Analogy based estimation is another technique for early life cycle macro-estimation. Analogy based estimation involves selecting one or two completed projects that most closely match the characteristics of your planned project. The chosen project(s), or analogues, are then used as the base for your new estimate. The ISBSG Data Portal can be used to select a suitable analogue.
Analogy based estimation differs from the comparison based estimation above, in that comparison based estimation uses the medians from a group of similar projects. Analogy operates with one, or perhaps two past projects selected on the basis of their close similarity to the proposed project. Comparing a planned project to a past project is commonly used in an informal way when “guesstimating”; consequently it is a familiar technique to the practitioner.
Estimating software project effort by analogy involves a number of steps:
- Establish the attributes of your planned project, (e.g. size, language type, etc.)
- Measure or estimate the values of those project attributes.
- Search the ISBSG repository for a project that closely matches the attributes of your planned project. (There is a tool on the Toolkit CD that does this search for you).
- Use the known development effort from the selected project, (analogue), as an initial estimate for the target project
- Compare each of the chosen attributes, (size, platform etc.).
- Establish or adjust the initial effort estimate in light of the differences between the analogue and your planned project.
It is very important to use one’s judgement to exclude inappropriate analogues and not be tempted to adopt a “likely” analogue without due care.