Graham Bleach

What do we know about software development?

2012-01-22T00:00:00+00:00

Just over thirteen years ago I finished an undergraduate degree in Physics and started working as a systems administrator. One of the tenets of science that I left behind was that claims require evidence. Robust evidence originates from repeatable, robustly designed experiments.

Thinking back over the hundreds of occasions when someone has suggested using a new technology, technique or process I can think of fewer than five occasions when that suggestion was backed up by robust evidence. This is a pretty awful state of affairs; I call myself an engineer and yet I make the majority of decisions based on little more than anecdote, intuition and poorly controlled experimentation.

Towards the end of last year, a colleague sent around a link to a talk [video slides] given by Greg Wilson at CUSEC. It’s called Bits of Evidence: What we actually know about software development and why we believe it’s true. It’s about an hour long and outlines some areas where evidence exists and what it seems to mean.

I’m currently reading Making Software, the book mentioned in the talk. Almost every chapter has caused me to discard some folklore or has explained some of my previous experiences and intuitions. Most importantly, it’s given me some hope that as an industry we can move on from anecdote-based engineering to evidence-based engineering.

I made a list of the publications that are mentioned in the talk; some are time consuming to track down, especially if you don’t have much experience in using academic search engines.

Lind (1753) A treatise of the scurvy
Doll & Hill (1954) The Mortality of Doctors in Relation to Their Smoking Habits, Br Med J. 1954 June 26; 1(4877): 1451–1455.
Evidence-Based Medicine Working Group (1992) (paywall). A New Approach to Teaching the Practice of Medicine, JAMA 268 (17): 2420–5
Cochrane Collection
Fowler (2009) (paywall) A Pedagogical Framework for Domain-Specific Languages, Software, IEEE, Volume: 26 Issue:4
Arunda & Easterbrook (2005) (paywall) Anchoring and adjustment in software estimation, ACM SIGSOFT Software Engineering Notes, Volume 30 Issue 5, September 2005. Read the thesis instead.
Sackman, Erikson, Grant (1968) (paywall) Communications of the ACM, Volume 11 Issue 1, Jan. 1968
Lutz Prechelt’s homepage
Boehm et al (1975) (paywall) Some experience with automated aids to the design of large-scale reliable software, ACM SIGPLAN Notices - International Conference on Reliable Software, Volume 10 Issue 6, June 1975
Why Aren’t More Women in Science? Ceci & Williams ISBN: 978-1-59147-485-2 Amazon
Woodfield (1979) (paywall) An Experiment on Unit Increase in Problem Complexity, Software Engineering, IEEE Transactions Volume: SE-5 Issue:2

…the 25 percent increase in problem complexity results in a 100 percent increase in programming complexity.
van Genutchen (1991) Why is software late? An empirical study of reasons for delay in software development, Software Engineering, IEEE Transactions Volume: 17 Issue:6

The two biggest causes of project failure are poor estimation and unstable requirements.
[Thomas et. al (1997)] - Can’t find this one

If more than 20-25% of a component has to be revised, it’s better to rewrite it from scratch.
Fagan, 1997 “Design and Code inspections to reduce errors in program development”. IBM Systems Journal 15 (3): pp. 182–211.

Rigorous inspections can remove 60-90% of errors before the first test is run.
Cohen 2006 Cohen, Jason. 2006. Best Kept Secrets of Peer Code Review. Beverly, MA: SmartBear Software

The first review and hour matter most.
Herbsleb and Grinter 1999 James D.Herbsleb, Rebecca E.Grinter. 1999. Splitting the Organization and Integrating the Code: Conway’s Law Revisited. Proceedings of the 21st International Conference on Software Engineering.
Nagappan et al. 2008 Nagappan, N., B. Murphy, and V.R. Basili. 2008. The Influence of Organizational Structure on Software Quality: An Empirical Case Study. Proceedings of the International Conference on Software Engineering: 521–530.
Bird et al. 2009 Bird, C., et al. 2009. Putting It All Together: Using Socio-technical Networks to Predict Failures. Proceedings of the 20th IEEE international conference on software reliability engineering: 109–119.
El Emam et al. 2001 El Emam, K., S. Benlarbi, N. Goel, and S.N. Rai. 2001. The Confounding Effect of Class Size on the Validity of Object-Oriented Metrics. IEEE Transactions on Software Engineering 27(7): 630–650.

Most metrics’ values increase with code size. If you do a double-barrelled correlation the latter accounts for all the signal.
The “beautiful” series:
- Beautiful Code Greg Wilson, Andy Oram, 2007.
- Beautiful Data Toby Segaran, Jeff Hammerbacher, 2009.
- Beautiful Visualization Julie Steele, Noah Iliinsky, 2010.
- Beautiful Testing Adam Goucher, Tim Riley, 2009.
- Beautiful Architecture Diomidis Spinellis , Georgios Gousios, 2009.
- Beautiful Teams Andrew Stellman, Jennifer Greene, 2009.
- Beautiful Security Andy Oram, John Viega, 2009.

Migrating to Jekyll from Personal Weblog

2011-12-04T19:00:00Z

This weekend I wrote some code to extract my posts from Personal Weblog and put them into a Jekyll-friendly format.

I suspect that few people were using Personal Weblog in the first place and of those, even fewer will migrate to Jekyll, so it’s unlikely that this will be of much use to many people, but I tidied it up a bit and put it up on github anyway.

The import isn’t very reliable, especially if there’s less than ideal HTML inside the blog posts and you might find yourself manually editing HTML to edit the post.

Moving stuff around

2011-12-04T00:00:00+00:00

For the past year or so my personal website has been in some disrepair; the RSS feed was returning HTML and there was no way to get to most of the blog entries at all. It was all on a domain called darkskills.org.uk, hosted by a third party. Both of these are things that I grew out of a long time ago and the source of some embarrassment. Sorting these things out has been too much like the day job for me to muster any enthusiasm.

I made a few attempts to migrate everything to an open source CMS, but these petered out after getting irritated with Wordpress or trying to glue the various bits of Django together to do what I want. Drupal, Wordpress, Django and all the other full-featured content management systems and frameworks I tried can be made to do what I want, but at the cost of spending hours learning their features, choosing styles and enabling plugins. Once that’s all done there’s the ongoing patching for security updates.

This weekend I really should have been doing something useful with my time, but instead I ended up migrating my blog entries to a static site, running on my BitFolk VPS, with pages generated by Jekyll. I combined this with a change of domain name and using Bootstrap as my stylesheet. The spartan design isn’t a great demonstration of what you can do with Bootstrap; there are some clear examples on their site that are well worth looking at if you like to avoid writing CSS at all costs.

One surprising outcome is that I seem to have regained some enthusiasm for writing on the web and making web pages. I’ve put this down to how easy it is to write in Markdown using a text editor and not having to worry about HTML. It feels like the Jekyll workflow is pretty close to my ideal way of making a simple site.

Static pages mean that I can’t host comments without using a javascript solution like Disqus, but that also means that I won’t have to deal with comment spam, which seems like another maintenance win.

I still have a few things to migrate over and then we’ll see how long my enthusiasm lasts.