Learning to estimate development time accurately
The last few months I've seen some debates pop up on the web about estimating projects. A lot of people appear to think that reliable estimates are impossible.
Comments like ‘programmers pull estimates out of their ass’ and ‘why does actual development always take 2 to 3 times longer than estimated’ seem to suggest that estimates are worthless.
Yes, some estimates are pulled out of the ass, this is an indicator that the task or feature is poorly understood. Development can take a lot longer than expected but this usually happens because of technical debt or when the project evolves without updating the estimates.
What can be estimated?
Choose an estimate that fits with how certain you are about a task. Communicate your certainty and don't let yourself be forced into an estimate when you're not comfortable giving one. Estimates for small tasks will be easier than large chunks. Estimates for projects lasting longer than a few weeks might need to be re-estimated regularly.
At Fingertips we estimate backend and frontend development in hours or days. For design we reserve a fixed amount of time upfront and for further development a percentage of the development time. We usually don't estimate time spent on project management, deployment, support, and research because it varies wildly depending on the client and the nature of the project. If we're forced to put a number on it we usually estimate around 15% of the development time.
So an estimate for a complicated search page might look something like this:
We expect to spend 16 hours on development and around 1.5 hours on top of that on design. We will also spend some time communicating with you and deploying the project, probably around 2 hours. Our total estimate for this feature is 20 hours.
Besides the estimate you could also mention to the client how certain you are about this estimate.
We've implemented search pages a lot of times. We're pretty sure we can get it done in the estimated time.
Once you start on a task you usually get a better idea how long it will actually take. If you expect to deviate a lot from the estimate, you can let the client know upfront so they can make an informed decision.
I started on the search page and I'm having trouble getting the indexing right for the legal documents. We will probably need an additional 2 hours to finish this.
Improving your estimates
You improve your estimates by practising. It's easier to estimate known tasks and you get better at imagining how much time unknown tasks take.
You can't learn without assessment. You must measure the time you actually spent on an estimated task to get better at it. Feedback from experienced estimators can reduce your learning time. Make a contest out of it with someone else on the project.
Tracking your time helps to get a feeling of how long things actually take. At the end of the week, sprint, or iteration you should spend some time evaluating your estimates. Your client will probably want to have this information anyway.
You can get an even better idea about how you spent your time by splitting it up into categories: development, deployment, feedback, project management and design.
The most important thing is to keep it fun. If you like stats you can go wild with graphs. If you don't like accounting force yourself to at least write down the weekly totals or start a stop watch or an egg timer when you're working on a feature. Try to find a process that works for you.
We track all our time with Freckle, which has a built-in timer. It allows us to send invoices for our work and keeps a record of the time we spent on a project.
How long did you actually work?
You need to use hard data. Boring or repetitive tasks might appear to take longer than they actually did. When you're ‘in the zone’ a day flashes past, but it's still 8 hours of work.
Let's assume again you have to implement a complicated search page. You've estimated that it would take around 18 hours to implement. After three working days you're finally done.
Does this mean your estimate was wrong? That depends.
During an 8-hour day you don't actually work 8 hours. One day you might have a longer lunch, arrive a bit late at the office, or have some sort of appointment.
I typically work around 32 actual billable hours each week, even though I'm at the office approximately 40 hours per week. In the three days mentioned in the example I probably only worked around 20 hours instead of 24 hours.
Figure out why your estimates were wrong!
There are three main reasons why estimates could be wrong: you didn't understand the task properly, you were less efficient than normal, or requirements for the feature changed after estimating it.
Estimating unfamiliar tasks is hard. Trying to visualize steps may help, but it's easy to miss details.
On some days you might be tired or constantly interrupted causing your efficiency to go down. For longer projects good and bad days even out after a while. For a short project a bad day might make you miss the estimate.
In any reasonably complex applications the requirements change once you start developing something. When is it time to throw out your estimates?
The more fine-grained your estimates are, the more you can re-use them once the requirements change. Fine-grained estimates will take more time to write down so you will have to find a nice balance.
In the example mentioned above an estimate like ‘three days’ might not hold up for very long. If you've included separate estimates for setting up the search backend and writing JavaScript for the frontend interaction they're more likely to hold when requirements for the data indexing changes.
Informed guessing
Estimates are nothing more than educated guesses, you have to assume your estimates are not going to be 100% correct.
Clients tend to think of estimates as a fixed proposal for a feature. They will mentally multiply the estimate with your rates and this price will stick in their heads. I do the same when I bring my car to the shop for repairs, there is no blame here.
Especially at the beginning of a project you should discuss features and budget in general terms. If you do find your client micromanaging the budget based on estimates ask them to take a broader view of the project.
Good estimates can help you determine which proposed set of features might fit a budget or help determine the order of magnitude of the price for a piece of software. The moment a project starts the estimates and reality will start to diverge. Insights and features change, so the estimates should change with them if they need to stay somewhat accurate.
Always remind yourself and others you can't predict the future.