I am wondering, how "real" work day (7-8 hours) relates to "engineer hours" term used in estimation of the time necessary for completion of some task. I think that estimated effort in EH (if correctly estimated) cannot be simply translated into work days by dividing with 8, and that effective work day of a programmer is shorter than the time he spends in the building in which he works. This can lead to big errors in estimates when estimating small chunks of tasks (i.e. what is scope of one iteration in SCRUM) and when there are no best/worst case estimates, but planning is done based on individual task estimates done by programmers. When programmers need to estimate time necessary to do some individual tasks, they usually estimate the time from the moment they start working on it to the moment they complete it. Needless to say, it is insane to expect that someone will do 4 tasks of 2 hours each in one day.
I would like to know what are best practices in successful companies for relating effective time vs. time spent "at work" and are there some books or researches which estimate what is average time programmer spend working (focused on the tasks at hand), not doing other activities which are not included into estimate, like reading/writing mails (except ones included in estimate, if it is support task or similar), brainstorming, meeting, drinking coffee, estimating tasks and updating status in bug/task tracking tool.
If there are no books or researches, any links to articles about this issue by respected members of community will be of help also as I couldn't find any, these keywords are too generic.
There are two things that we do when we program:
When we're repeating similar things, estimating - whether in points of relative effort / complexity or in hours - is usually fairly straightforward. Even when estimating in points, these can easily be correlated back to hours after a sprint or two, once the team has got up to speed.
The problem most teams face is that they're often doing new stuff. This is because:
Thus, a lot of what we do on software projects tends to be new. The usual same-old same-old frequently ends up being given to junior developers, for whom it's new.
The trouble with new things is that you have no idea how long they're going to take. You've never done them before!
I've seen a number of strategies for handling this, depending on how determined someone is to (force devs to) produce reliable estimates, how much trust you have with stakeholders, etc.
Pad the estimates horribly and then make the work fit them (this is what developers end up doing if PMs / SMs force them to produce reliable estimates, whether the devs realise it or not).
Kanban's "classes of service" allow you to mix some new things with some familiar things. This will tend to allow a less variable flow through the development pipeline, so you can base your estimates on that.
Do everything that's risky first (particularly if you don't know how risky it is!). Gradually the flow will stabilise. On a 9 month project, it took 3 months to start producing any kind of useful estimates, so careful with this one. The upside is that your stakeholders will see you addressing the things which keep them awake at night early, so trust can be high. Your PM / SM will go spare during this time. It's OK.
Accept that the old metaphor of "time spent = work done" is a myth in projects involving high amounts of learning and knowledge work. It might have been appropriate in the days when developers were treated like a factory - churning out the implementation of others' designs - but in the world of Agile and Scrum, it's completely inappropriate. Even Scrum.org have recently removed the necessity for estimation and velocity measurements from their guide. Find other ways to develop trust, manage risk, win budgets and have conversations around the work - since these are the only reasons you need estimates in the first place.