A Developers Guide to Servers. What you need to know but were afraid to ask.

Servers are the backbone of enterprise computing today, most websites for example run on either Apache or IIS and will be running on a server of some description.

Perhaps because of servers ubiquity it is easy to become complacent about them especially as they are rarely seen but understanding what a server can offer you and it's limitations will definitely help with software development.

Servers on the whole are built more of the practicality side than for the esthetics, there is no need to make them look pretty if they are going to spend most of there time in a Data-Centre rarely seen. The server will only have a visit if something goes physically wrong, everything else should be looked after remotely. Ultimately the success of a server is measured in the the amount of time spent looking after it and it performing the task(s) you have set it.

The Differences Between Servers and Desktops

The main difference between Servers and Desktops are Servers are designed to run 24/7 desktop aren't. The components in a server can usually handle higher stress and handle redundancy by having two or more of critical components so if one fails there still is another keeping the server up and the service it is running available. 

The Different Types of Server

There are different types of server then mains ones are,

Pedestal/Tower

A Pedestal Server (which is often know as a Tower) isn't rack mounted as the main intention of it's design is to work primarily in an office environment not in a Data-Centre. These machines are usually found as file or print servers.

Rack Mounted

As the name suggests a rack mounted service lives in a rack. Placing servers on top of each other in a rack to support their weight and to provide them with services such as power and networking means that you can have lots of servers taking up less room and room in data-centre's are at a premium. A typical rack is 19 or 21 U (I will explain later what a U is)

Enterprise Class

Enterprise Class servers are free-standing like pedestals but you won't find them in an office! Because of their size, value and special requirements such as multi-phased power supply, larger than domestic voltages or specialist cooling they can only really live in a data-centre. As you can imagine this type of server doesn't come cheap. Enterprise Class machines would have been called 'Mainframes' in year gone by but as they run can happily run operating systems such as Linux, Unix and Windows (The IBM P Series pictured does not run Windows). Their main function today is as a consolidated platform as one of these servers can run the equivalent of several hundred Linux servers saving space and power.

Blade

In a typical rack you can get about 10 rack mounted servers. Blade servers on the other hand are designed for high density, i.e. more servers in the same amount of space. Take for example the HP C Class Blades, a U10 C7000 Blade Enclosure can have upto 16 468c G5 Blade Servers. So a normal rack can have 32 servers in it rather than 10. Blades servers make ideal candidates for hosting Virtualised servers such as Vmware, Hyper-V or XenSource.

The Worlds Most Popular Server

It is estimated by HP that the world most popular server is the ProLiant DL380 G5. As I haven't seen this claim undisputed I will use the DL380 as an example of a typical server as it is the one you are most likely to meet. This next section will give you a detailed overview of the machine.

HP's Official ProLiant DL 380 Overview

Front View:

1. Eight sockets for PC2-5300 Fully Buffered DIMMs (DDR2-667) - up to 64GB RAM is possible

2. Hot-plug fans, full redundancy - air is sucked through the machine from front to back.

3. Systems Insight Display

4. Quick release lever for rapid server access - lifting these levers will allow you to quickly pull the machine out of a rack.

5. Support for eight Small Form Factor hot plug hard drives bays - Typically SAS drive (discussed in the article on Storage) are installed here.

6. Front LEDs (show server status) and Unit Identification button/LED (for easy in rack server identification)

7. Two front USB ports (2 rear USB ports, 1 internal USB port)

8. Intel Xeon Processor (Performance models include two processors)

9. Hot plug power supply, redundancy option (High performance models include redundant power supply) - Yes two power supplies. One can be replaced whilst the server is still working for maximum up-time.

10. Three full-size PCI-E expansion slots in standard expansion cage (or optional mixed PCI-X/E expansion cage). Two additional low-profile PCI-E slots embedded on the system board. Four slots available for use; one consumed by Smart Array controller (Base and Performance models)

11. Quick removal access panel

Rear View:

1. Torx service tool - HP exclusively use a standard screw-driver head in their machines called the Torx T15 to help avoid screw-driver cam-out.

2. Optional pass through cable door

3. Two Embedded NC373i Multifunction Gigabit Server Adapters with TCP/IP Offload Engine

4. Hot plug power supply bays, redundancy option (High performance models include redundant power supply)

5. Integrated Lights-Out 2 (iLO 2) Remote management port. ILO is a remote management console that can turn a server on and off and take over the screen amongst other features. ILO usually has it's own dedicated VLAN. I will talk more about what that is in A Developers Guide to Networking. What you need to know but were afraid to ask.

6. Video Port

7. Two USB 2.0 Ports

8. Serial Port

9. Keyboard Port

10. Mouse PS/2 Port

11. Two low-profile PCI Express x8 slots. Slot 1 is consumed by the P400 Smart Array controller. (Base and Performance models)

12. Three full-size PCI Express slots in standard cage (or optional mixed PCI-X/E expansion cage)

Specifications

• Processors:
  • Quad-Core and Dual-Core Intel® Xeon® processors - systems support up to 2 processors
  • Intel® 5000P chipset
• Memory:
  • Up to 64 GB PC2-5300 Fully Buffered DIMMs (DDR2-667)
• Storage Controller:
  • Performance Models: 512MB (RAID 0/1/1+0/5/6)
  • High Efficiency and Base Models: 256MB Controller (RAID 0/1/1+0/5)
  • Entry Models: 64MB Controller (RAID 0/1/1+0)
• Internal Drive Support:
  • (8) small form factor (SFF) hot-plug drive bays to support Serial-attached SCSI (SAS) and Serial ATA (SATA) drives
  • Slimline media bay supporting Optical or Floppy drive
• Network Controller:
  • Two embedded Gigabit Network Adapters
• Expansion Slots:
  • Four PCI-Express slots
• USB Ports:
  • USB 2.0 support
  • 5 total ports: (2) ports up front; (2) ports in back; (1) port internal
• Integrated Hypervisors (Optional):
  • VMware & Citrix XenServer virtualization technology ... sadly not Microsoft yet!
• Redundancy:
  • Fully redundant hot plug fans (N+1)
  • Hot plug power supply with optional redundancy (Included in Performance models)
• Form Factor:
  • Rack (2U), (3.5-inch); Depth 26 inches (66 cm)

What is a U ?

A 'U' or Rack Unit is an Electronic Industry Alliance standard height (EIA 310) measure for a rack mounted server. The reason there is a server is so machines from multiple vendors can all use the same racking.

A U or 1U equates to 1.75 inches or 44.45 millimetres. A typical rack is either 19 or 23 U's high. Our example machine the HP DL 350 G5 is 2U (see Form Factor in previous section)

Good Questions to Ask

One of the most frequent question there is, is why is the system running slowly?

Often the answer to this question is one of the four resources on a server is getting maxed out. CPU, Memory are obvious but the problems usually lie with I/O in the form of networking and disk. So understand what networks the servers is connected to and what storage and how it is configured. Recently I found a SQL Server database was running like a dog because the mdb file was sitting on the same hard disk as many others so the disk was working flat out! However to solve the problem developers were off writing better SQL and .Net code which won't have helped at all.

Is the Live server the same as the Dev? If so, what else is the Live server running?

'It runs ok in development' is frequently heard and we all know that if development and testing environments could accurately simulate a Live environment then more problems could be found before go-live. A far easier thing to do is measure how much resource your new development consumes and profile it over a working cycle such as a day or through a month-end process, then examine the servers it's going to go live on, and watch the server through the same profile you will get a better idea whether your new development will fit, this will be easier than trying to reproduce a Live simulation in test or dev but if you can do that, that's great!

In Summary

I do like hardware! So I am bias but I do believe that getting to know hardware can help bridge divides and help solve problems because if programs don't work well infront of business users the whole of IT looks like smucks and the 'his fault not mine' looks extra lame.

via http://geekswithblogs.net/SabotsShell/archive/2009/01/24/a-developers-guide-to-servers.-what-you-need-to-know.aspx

ScrumMaster and ZenMaster: The joke of certification

Many people object to ScrumMaster certifications:

1. It’s a money making machine
2. Scrum Masters do not learn anything during classes
3. The certification is nothing worth - because nothing is certified

I have been a Certified ScrumMaster (CSM) and a Scrum practioner for some years. People who object to the certification do see it from the wrong angle. You need to understand Zen to understand the goodness in CSMs.

photo credit: darkpatator

Certification is a Zen joke, because the role of a ScrumMaster cannot be certified. It’s not about knowing some technical questions. What should a trainer certify in such a class? That you can lead an agile Scrum team as a ScrumMaster? No one can certify the fact that you’re a leader, catalyst and enabler. You either are or you aren’t. Zen masters (ha, another master without a master!) would laugh at the fun in the ScrumMaster certification. They laugh about the idea of certifying enlightenment.

Scrum without ScrumMasters

As another parallel, both in Scrum and in Zen, masters are only enablers. They are not needed after the act of enabling Zen/Scrum. My Scrum trainer told me, the goal of a ScrumMaster is to make himself obsolete. There is a Zen koan which goes like this:

If you meet the Buddha, kill him.
— Linji

If you see a ScrumMaster, kill him. Zen tells you:

If you are thinking about Buddha, this is thinking and delusion, not awakening. One must destroy preconceptions of the Buddha. Zen master Shunryu Suzuki wrote in Zen Mind, Beginner’s Mind during an introduction to Zazen, “Kill the Buddha if the Buddha exists somewhere else. Kill the Buddha, because you should resume your own Buddha nature.”

If you think the ScrumMaster is Scrum, you’re delusioning yourself. In Scrum the product owner and the scrum team can, and should from my view, act by themselves, without the need of a ScrumMaster. The ScrumMaster helps them achieve their Scrum. Helps them overcoming initial obstacles in their productivity.

Kick your ScrumMaster
If the ScrumMaster is not good enough for them, certification and coaching inside the company hasn’t helped, the Scrum team has always the right to kick their SM if he isn’t good enough for them. And they should do so. If in Zen a master isn’t good, pupils will just leave him. This might lead to problems within the organization, especially if the ScrumMaster is their boss, but that should be the problem of the organization, not a team problem.

Practitioner certification

There are many more certifications from the Scrum alliance. If you dig deeper, the real fun part is that CSM doesn’t mean anything, practitioner means much more:

The practitioner level of certification (CSP) is only offered to those CSMs who have hands-on experience using Scrum. Applicants must complete an extensive questionnaire with probing questions that focus on applicants’ real-world experience using Scrum on software development projects. Their application is reviewed for answers demonstrating competence and comprehension of principles that can only result from hands-on work. The applicant may be questioned to determine eligibility. To maintain CSP status, you must submit a new application every two years.

Is the certification any use?

Yes. The Certified Scrum Master training has several merits:

1. Calling the Scrum training “Certified” guaranties the quality of the trainer
2. It motivates the Scrum master to think in Scrum
3. If managers take part, it helps the organisation adopt a “we can do it” view about Scrum
4. Certification (CSM) seems to be one of the main reasons for Scrum success in the enterprise. The certification makes Scrum compatible for managment.

The view about acceptance is shared by Peter Stevens:

It is also about branding, and has been quite successful. The acceptance of the CSM program is high (especially from corporate customers, and this is where the money is). I believe the CSM program is an important reason why Scrum is better accepted than say, XP, in corporate management circles.

Scrum is successful. I’ve seen it help development departments gain productivity. If you do not scrum yet, go for it.

Future-Proofing A Design

Last time on FAIC a user asked for guidance on the potential pitfalls of refactoring an automatic property into a regular explicit property. This is just an example of a far more general problem: how can we design programs so that they are easy to get right when things inevitably change in the future?

This is an incredibly difficult question to answer, one which whole books could be written on. Today, I'll just give three general points to think about.

First: Premature generality is expensive.

Designing code to be future-proof without a clear understanding of what the future holds often leads to excess generality; generality has very real costs, and they might be higher than the benefit accrued.

If you do a good job of solving the problem at hand with the tools at hand, if the code is clean and organized and has few moving parts, then it will be easier to generalize it in the future if you need to in order to solve a problem then.

Design your code to solve the well-understood problem. If you design it to solve an unknown future problem, odds are good that you're going to solve it poorly, making even more work for the future.

Second: Represent in your model only those things which are always in the problem domain and whose class relationships are unchanging.

The relationships between the classes TrafficLight, TrafficSign, Vehicle, Car, Truck, Pedestrian, Roadway, Intersection, SignalTiming and RightOfWayLaw are likely to be eternal and unchanging.

Things change. The semantics of a particular RightOfWayLaw might change subtly or grossly as the city council does its job. There might be new subclasses of Vehicle created in the future. A specific traffic light might have a faulty implementation that the rest of the system needs to handle gracefully. But each of these situations is about changing the implementation details of a class, never changing its relationships with other classes.

If the class relationships are future-proofed then it is a lot easier to edit the implementations of those classes without having to worry that the whole system will thereby get messed up.

The easiest way to keep the relationships straight is to base them as much as possible on concepts directly from the problem domain.

Third: Keep your policies away from your mechanisms.

The mechanism of a traffic light works the same way no matter what policies like light timings at rush hour are.  If mechanisms (TrafficLight) are separated from policies (SignalTiming) then you can change the mechanism to a more efficient one without worrying that you’re going to inadvertently change policy, and can change policy without worrying that you’re going to break mechanism.

My earlier example of a bank balance was deliberately an example of what goes wrong when mechanism and policy become conflated. The “Balance” getter was originally a mechanism, but after the edit, it became a security policy enforcement tool.

You must then ask yourself, “where in this code do I care about enforcing the policy, and where do I care about executing the mechanism?” and make the appropriate edits. After the edit, everywhere that needs policy enforcement needs to use “Balance”, everywhere that needs to effect the action of the mechanism needs to use “balance”. 

What are the odds that there is going to be a bug introduced, given that we now have a difference which is important and almost invisible? Pretty high! This difference should probably be made more visible by renaming the backing store to something that calls it out as semantically different from the property accessor.

Let's bring this back to the question at hand. What guidance do we propose for future-proofing turning automatic properties into regular properties?  The guidance that I am proposing here is:

• First, think about whether making the design general now to solve an unknown problem that you might not even have in the future is worthwhile.
• Second, make sure that the design in general makes the relationships between classes unchanging, even if the specifics of each class change.
• Third, consider whether that hypothetical future change to the property will be splitting apart its role as a mechanism from its role as an enforcer of policy; if this is going to be an important distinction, then consider getting that distinction into the class implementation early so that you don’t have to retrofit it in a tedious and error-prone manner later.

via http://blogs.msdn.com/ericlippert/archive/2009/01/16/future-proofing-a-design.aspx

The genius behind the Google 20% time; It isn’t the time

Paul Buchheit has a really nice post on the power of code over talk and has one item that resonated with me:

This is where Google’s “20% time” comes in — if you want innovation, it’s critical that people are able to work on ideas that are unapproved and generally thought to be stupid. The real value of “20%” is not the time, but rather the “license” it gives to work on things that “aren’t important”. (perhaps I should do a post on “20% time” at some point…)

That is exactly how I felt about 20% time. It wasn’t about the time. In fact, I didn’t actually know that many people who took the time! The key was the following effect:

• In order for 20% time to work, anyone must be able to see what is out there
• In fact, if you want to get some people working “for free” you need to both advertise your project, and write it in such a way that it is easy to get ramped up and productive (end result: better code)

The end result is the culture HAS to be kept very open. The ability to see the projects that are worked on, check out the code, see presentations and design docs, is a key reason why Google does so well at engineering in my opinion.

Compare this to companies such as Apple where you don’t know what the chap sitting on the other side of the cube is doing. Therefore, even if you have some amazing wisdom that could be game changing for his project, it will never happen.

20% time means that sparks of genius can happen. If you copy anything about it, ignore the “time” part, and go with the open culture part.

Whenever I saw projects that were secret within Google it irked me. It stopped this from happening. It was as though that project was going walkabout instead of joining the party.

via http://almaer.com/blog/the-genius-behind-the-google-20-time-it-isnt-the-time

Sniff Out That Smelly Code

As time goes by, things go wrong with any body of code: we get lazy, bad developers contribute code, the original clean architecture gets forgotten, and so forth. Some bad code is easy to spot: it simply “smells” - once you see it, you know it’s bad.

Refactoring this sort of code to remove the smell isn’t often that difficult - and the time taken will be repaid next time you have to make modifications.

The following are my ten worst “odors” in code:

1. Long Functions

Over time functions often get longer and longer - extra functionality is added, edge cases are handled, and so forth. However, long functions are difficult to change, and difficult to understand.

Each function should ideally do exactly one thing: you should be able to sum it up in one sentence (and the method name should be a summary of that sentence).

If you find a function is getting too long, split it into sensible parts. Each part should follow the above rule.

2. Commented-Out Code

Some of these rules are open for debate. Not this one. I’ll go so far as to say that there is never an excuse for commenting out code.

• Are you trying to preserve some sort of history to the code? You should be using source control.
• You’re not sure if the code might be necessary: find out - and don’t commit until you’re sure.
• Perhaps you’re in the process of shotgun-debugging: step away from the code, and work out exactly what’s going on.

3. Copy-Pasted Code

Like commented out code, there’s really no excuse here. Programming IDEs should only support ‘cut’ and ‘paste’ - there’s no need for ‘copy’.

Instead of copying code, refactor it into a method, and call it. Copy-pasting is lazy - period.

4. Handled-but-Really-Unhandled Exceptions

I need to explain here: I’m not suggesting that every function should handle all exceptions that could be thrown. Rather, I think the handling of exceptions falls into two categories:

1. The function can handle a particular exception, and takes some action to correct it.
2. The function cannot handle a particular exception, and does not attempt to catch it.

What I’m referring to is code like this:

void Foo()
{
  try
  {
    // ...
  }
  catch( Exception e )
  {
    // Throw exception away, or just log it.
  }
}

In general, doing nothing with an exception is a mark of bad code. If the function can’t handle it, then it shouldn’t be caught. Alternatively, if it didn’t need to be thrown, then try to amend the throwing code.

(Especially bad is throwing and catching the same exception in a single function. Exceptions should not be used for flow control; in situations like this, the entire function needs to be re-thought).

5. Large Numbers of Parameters

Functions taking large numbers of parameters (say, more than half a dozen) are usually a bad idea, and are generally indicative of either a function trying to do too much, or poor class organisation.

Often a function will grow to accommodate new functionality, and hence grows parameters. Split the function into separate, smaller, simpler functions.

Alternatively, if the function absolutely cannot be split, consider introducing a new class which encapsulates some or all of the original parameters, and passing that class as an indirect parameter.

6. Non-Obvious Names

Variable or function names such as foo or bar, swearwords, names, or otherwise clever or amusing really don’t have a place in professional code. They might make you laugh, but when someone else tries to read your code, they won’t thank you. (And remember - that ‘other’ person reading your supposedly clever code might be yourself in six months’ time).

The use of i,j,k is prrobably a little more open to debate - however their use as loop indices can usually be avoided if your language or libraries support more generic iteration constructs.

7. Deep Nesting

Just as functions shouldn’t be too long, so they shouldn’t be too wide. There’s no hard-and-fast rule, but if your conditionals or loops are nested more than about three deep, you should consider refactoring your code. Either pull the inner parts of the code into separate methods, or pull the complex conditions into functions. A line of code should never exceed approximately seventy characters (the standard editor width).

8. Beacon Comments

Sometimes you don’t even need to smell bad code - the original developer has helpfully pointed it out in the comments! If you see comments like:

// This is hacky...

// TODO: This is bad. FIX IT!

// This code ought to be refactored

…then you know something needs to be done.

9. Narcolepsy

…by which I mean inappropriate sleep() ing. If your code requires an (essentially random) sleep interval to function correctly, it’s likely to fail if the user’s machine is especially old, especially new, or just busy. Try attaching to an event that signals when the situation you want to be in occurs.

Another related smell is relying on the speed of the computer, the speed of video frames, or similar, to achieve a particular timing effect.

10. Helper Classes

This is something I’m guilty of from time to time, and I’m looking for comments as to the best way to avoid it. Helper or utility classes always seem to grow in large bodies of code, usually containing unrelated static methods. These clumps should be analysed and split or moved into classes that have one task.

 

via http://www.hackification.com/2009/01/20/sniff-out-that-smelly-code/

A Simple Scrum Sprint Review

At the end of the sprint, the Product Owner, Team and Scrum Master meet to review the progress of the sprint. The product owner has to evaluate the state of the project so s/he can decide what to do next. How does the Product Owner ensure that s/he gets a complete and correct understanding about the state of the project, including all inconvenient truths? Here is a simple agenda/meeting template to follow, to make sure all the bases are covered.

Last week, I introduced that concept of a Sprint Contract to define the parameters of the sprint. The factors Scope, Quality, Time and Cost will be familiar to any project manager. Scope is defined by the stories and in particular their size. Quality is defined primarily by the definition of done. Time is fixed by the sprint duration. An upper limit on the costs is set by the team size * sprint duration, after adjusting for absences and other tasks.

A simple sprint review needs to

1. Confirm that the team has delivered on its commitments
2. Confirm that the overall project is on track
3. Examine the functionality which has been delivered.

Confirm that the team has delivered on its commitments

The Scrum Master should take a few minutes to review and summarize the agreement between product owner and team. How many stories? How many story points? What is the definition of done? How much effort did the team plan to invest?

Next the Scrum Master should present a summary of what the team actually accomplished. How many stories achieved done? How many story points? Did the team invest more, less or the same effort as planned? If there were any important differences, now is the time to make them visible.

Confirm that the overall project is on track

The Release Burn Down chart is the primary tool for measuring progress and scope of the project. Scope, progress and estimated completion date are all clearly presented in one easy to interpret chart. Everyone in the project should see the update release burn down chart after each sprint. It's even better to keep it posted on the wall.

Is this enough? Beyond the basic burn down chart, it might useful to keep an eye on the other parameters of the project, i.e. quality and cost.

Monitoring quality builds confidence in the product and keeps your technical debt under scrutiny. The number of unit tests and acceptance tests defined and passed should increase every sprint. Insufficient tests are warnings that technical debt is accumulating or that requirements could change suddenly as the project nears completion. A rising number of open bugs may be a sign that the quality is not sufficient. Your next retrospective would be a good to time ask yourselves how to produce fewer bugs.

I think the main reason for monitoring work invested is to make sure that your teams efforts are not being diverted from the project. Monitoring budget in actual money will keep your top management happy and make your Product Owner's life easier. Budget can be tracked with a burn down chart, just like scope, and the budget divided by burn-rate should be sufficient to cover the remaining sprints.

Examine the functionality which has been delivered

Your team is only allowed to demonstrate finished functionality, i.e. meets the definition of done. My experience has been that trying to demo unfinished functionality causes trouble -- things which haven't been fully tested often break. Everyone should get a chance to talk, so each developer should demonstrate the functionality s/he was primarily responsible for.

As a product owner, you should ask questions and explore. This is a demo of product which your users will use, not a tour through the mine field. So everything you see should work or fail gracefully.

Once you've seen everything that is done, you can discuss briefly anything which was not successfully delivered. Why wasn't it delivered? What needs to be different, so that it can be completed in a future sprint? Or do you need to rethink?

The Simple Scrum Sprint Review Meeting

Here is a sample agenda for the meeting, with timings for a 2 week sprint.

Present: Product Owner, Implementation Team, Scrum Master
Moderation: Scrum-Master
Duration: 1 hour per week of sprint length
Agenda:

What	Description	Who	Duration h:mm
Confirm that the team has delivered on its commitments	Review Sprint Contract Summarize sprint results (stories, story points, effort, etc) Note discrepancies between plan and actual	Scrum Master	0:05
Confirm that the project is on track	Review & Discuss Release Burndown Chart, other key points as needed	Scrum Master	0:05
Examine the functionality which has been delivered	A spontaneous guided tour, led by the developers, through the new functionality in the product.	Product Owner and Team	1:40
Discuss functionality which was committed but not finished	Understand what happened and why, but save deeper investigation for the retrospective	Product Owner and Team	Up to 0:10

 

via http://agilesoftwaredevelopment.com/blog/peterstev/simple-scrum-sprint-review

To Paraphrase: I'm Sorry I Invented Object-Oriented Programming!

Today is October 15th, 2007. Very few people celebrate this day, but I think its an important day to remember... because today is the ninth anniversary of the day that Alan Kay, inventor of the computer term "object-oriented programming" (OOP), apologized for inventing the phrase on the Squeak developer's list.

Apparently he was a bit sad to see how the term had become abused:

I'm sorry that I long ago coined the term "objects" for this topic because it gets many people to focus on the lesser idea. The big idea is "messaging"...

The key in making great and growable systems is much more to design how its modules communicate rather than what their internal properties and behaviors should be. Think of the internet -- to live, it (a) has to allow many different kinds of ideas and realizations that are beyond any single standard and (b) to allow varying degrees of safe interoperability between these ideas.

There's a theory in linguistics (Sapir-Whorf hypothesis) that says your ability to understand something is severely limited by the words you use to describe it... Some Amazon tribes don't have words for numbers, which limits their ability to understand how much they have...

For developers, who spend a lot of time in a 100% abstract world, using words severely limits how you can think about an abstract problem. By definition, abstract problems do not lend themselves well to existing language structures... and if you live and breath abstract problems, naming things significantly restricts how you can think about them. Now, sometimes its a great idea to limit abstract thought with concrete words... such as verbose variable names. Other times, it can be severely misleading or impair one's ability to understand the actual problem... such as verbose variable names.

There are a lot of linguistic traps in programming that affect how you can solve problems... you're dealing with a world that cannot be easily visualized or verbalized. This means that in order to communicate ideas between developers, you're always using bad analogies. These analogies get worse and worse as you try to communicate them to non-programmers. As long as everybody agrees that words like "objects" and "layers" and "logic" are necessary evils, we're OK... but when people start obsessing about the right way to do "pure object-oriented programming", or they start talking about "business logic" as a layer, then we have major problems.

A lot of the Java object-oriented patterns books make people unlearn what they have learned... the right way to use factory objects, interfaces, inheritance, etc., they all seem to be attempts at messaging. Either informing somebody of what they can do (inheritance, strategy), or informing somebody how their behavior can be modified (inversion of control, dependency injection).

The latest pony in this show is probably Google Guice: a slimmed-down version of the somewhat bloated Spring Framework for Java. They claim "pure dependency injection", so much so that you never need factory objects or static methods ever again! Sadly, my sources say the truth is vastly different.

Of course, Python and Ruby fans would mock people for using a language like Java that needs so many patterns in the first place... whereas with scripting languages and data-driven programming, such ideas are nearly intuitive...

Regardless, I'm glad things are moving in the right direction... All this "giving things names" does help when it comes to communicating how software works, plus people seem to be aware of the inherent dangers of verbalizing code... the "patterns for patterns sake" crowd seems smaller than the "objects for objects sake" crowd, which is A Good Thing ©

Anyway, happy Ninth "I'm Sorry I Invented The Term Object-Oriented Programming" Anniversary!

I wonder what the software world will be like on the tenth anniversary?

via http://bexhuff.com/2007/10/to-paraphrase-im-sorry-i-invented-object-oriented-programming

Requirements and you

Part 1 - Introduction and Description of the problem
Part 2 - Requirements and you
Part 3 - Tracking Features and Bugs
Part 4 - The little black book that could
Part 5 - Your developers - Don’t be a prick
Part 6 - Wrapping up

Part 2 - Requirements and you
When you give away a project to be done by someone else, the first thing that you will have to do is to come up with the requirements.
This seems to be easy, and everyone wonders how their projects become so garbled in the process.
Now, here is a little secret:
Writing requirements is hard.

Writing good requirements takes practice, patience and a little secret.

The secret to a good requirements document:
Treat the requirements document as if it would be the only thing that stands between you and checking off the task on a list.
Because that is what it is.

Someone who would not know your project or anything else about what you are trying to do has to be able to look at the list and see if an item has been done. Because this is what is going to happen the moment you enter a dispute over a project outcome and - ultimately - your money.

As this all seems cut and dry, here is a real-life example.

I was working for one of the top international banks as a project manager at the time when this happened.
One part of the project I was involved in (thank you, I was not involved in the part that went south) had the following requirement written down and sent to India.

Requirement 3465x/b: Implement a print button as seen on graphical mockup 577254b.

What happened?

Well, the print button was implemented as per reuirement and mockup from the graphics department. One tiny thing: It would not do anything when clicked on. Yes, you heard that right: Nothing would happen.

Whose fault?

Although it would be easy to blame the programmer (and it was tried) He did NOTHING wrong.
He implemented the feature exactly as requested and specified.

Yes, you could say “But he should have known that it should print the page when clicking on the button.” But I disagree.

Why? Because there are easily a few dozen questions like this:

• How should the printout look?
• Which print function should I implement? Is there a print function available for this program already?
• Should or could we use a third party library?
• Should it print physically or convert the page into a PDF?
• Which part of the pages am I supposed to print?
• Should a legal notice be attached to the printout?
• Which font should I use for printing?
• Should the logo header be included? Where is the high resolution logo?
• …

This is what happens when you write shitty requirement documents.
Another drawback is insecure and less productive coders, as they are having to guess your intentions along the way.
Time and cost estimates done on bad requirements are also way off.

How to write a good requirements document

• Treat the document as what it is: The only evidence of what you want to have done. Telepathy does not enter the picture at any point.
• Divide the document into sections according to screens / pages
• Divide the sections into features / work packages to be done.Each feature should be described in a way that makes it easy to identify it as “done”. For a bonus, it should be possible to identify a good implementation from a bad one.

In the above, badly done requirement, implementing the button was “done” flawlessly.

Also:
Update the requirements document with every change and communicate this clearly to your programmers.

Another example:
I want a button on my landing page that leads to my affilate offer versus a redirect page.

Bad:
Make a green button, leading to redirect.

Better:
Put a green button on the lower page, leading to redirect.php

Better still:
Put a green button on the lower page, just before the footer.
Clicking the button will lead to redirect.php.

Best:
- At the end of the text container, put a green button. The button should be labeled “Claim your prize NOW!”. The color code for the button is #00FF00. See attached screenshot of layout (layout_mockup_01.jpg).
- When the user clicks the button, he is directed to redirect.php in the same directory. The link opens in the same window.
- Please contact me with any questions.

Drawbacks to this method
I am tempted to say “NONE” but of course there are.
You will be spending more time on your requirements and you will still mess it up sometimes. Live and learn.

Advantages of this method
Your coders will know exactly what is required of them, leading to them being more productive and feeling more secure in what they are doing.
You will get better results and the chance to dispute badly done work.

 

via http://blindapeseo.com/webmaster-life/requirements-and-you

Getting Management and Engineers Out of Each Other's Hair

This is the fifth in a series of posts on self-management that began with "The Use of Self-Managing Teams Outside the Context of Software Development."
Short Iterations Increase Management Leverage
One of the hardest jobs of a manager is discovering and resolving problems. If you don’t know about a problem, you can’t resolve it. If you discover a problem late, you can’t do much about the negative effects it has already had. If you don’t get feedback on the results of the actions you take for a long time, it is difficult if not impossible to judge the effectiveness of your actions. Short iterations, the practice of breaking up a release into 1-4 week development cycles that each produce a shippable increment of work, vastly increase the amount of feedback available to management for making good decisions. It also provides more opportunities to make adjustments and judge their effectiveness.

Burndown Charts are a Management Dashboard That Everybody Loves
A burndown chart is a useful tool for measuring the amount of progress, positive or negative, during an iteration which clearly illustrates trends which may indicate problems. It is simply a day-by-day chart of how much effort remains. At the beginning of an iteration, 100% of the work for that iteration remains. At the end of the iteration, there should be 0% remaining.

An important multiplier of the effectiveness of a burndown chart is its visibility. A clearly visible burndown chart makes it obvious to everybody how the project is progressing. Nobody wants to see a burndown chart that isn’t making progress, and nobody wants other people to see lack of progress for a significant period of time. The burndown chart helps to motivate and inspire. If your own work is going well, but the chart isn’t going down in general, you’ll be more engaged in helping to solve problems that the team is having overall. When used effectively, the burndown chart reduces the effort required for a manager to determine that there is a problem and also reduces the need for the manager to cajole the team into taking corrective action.

Retrospectives Emphasize That You Are All In This Together
A retrospective provides a forum for the team to talk on a regular basis about what worked and what didn’t work as a team, to discuss ideas for improvement and to work out problems. Instead of a manager having to unearth all problems and come up with all solutions, a retrospective accomplishes much of this, providing the manager with more bandwidth to deal with sensitive issues and to focus on people management. Retrospectives are particularly effective when combined with short iterations because problems have less time to fester and there are more opportunities to take corrective action.
User Stories Reduce Micro-Management
Agile prefers user stories over requirements. User stories are simply stories that describe what is needed from the user’s perspective. User stories help to separate business value from implementation and focus all parties on the desired outcome. The separation removes the invisible constraints which can appear when the engineer assigned to a task is given a specification which completely constrains the problem such that only one solution is available. By focusing on the outcome desired by the user, the engineer working on the task has more freedom to utilize their creativity and ability to innovate without the risk of implementing something that the user doesn’t want.
There are two reasons that user stories support self-managing teams. First, they reduce the temptation of technically minded managers from becoming an implementation bottleneck. Second, they encourage team members to innovate and take on more responsibility which increases their self-sufficiency. This is much easier to do within an Agile environment because there are more practices in place such as short iterations, burndown charts, and retrospectives which encourage learning while reducing the risk of making mistakes.
I hope you have enjoyed this series on how Agile practices reduce and redistribute the job of management and thereby increase your capacity for self-management. If you missed the beginning, the first posting is "The Use of Self-Management Outside the Context of Software Development" .

via http://damonpoole.blogspot.com/2009/01/getting-management-and-engineers-out-of.html

No-Brainer Self-Management Practices in Agile Part 2

This is a continuation of "No-Brainer Self-Management Practices in Agile Part 1"

Whole Teams Improve Coordination and Communication

A whole team is a small group of people (no more than 12) that work together towards a common purpose, primarily spend their time as part of the team, and as a team have all of the skills they need in order to be self-sufficient. These skill sets may include server side programmer, web designer, tester, technical writer, scrum master, product owner, etc. The whole team practice contributes to self-management by reducing the amount of coordination required in order to achieve a goal. For instance, instead of having to coordinate with a QA group in order to find somebody that can test a piece of software, having a QA person as part of the team means that there is always somebody familiar with the software available to test it.
Another benefit of having a whole team is that it makes it more obvious if you have resource imbalances. If the QA folks in the team can’t keep up with the developers, then you either have too many developers on the team or too few QA folks. If your front end developers can’t keep up with the folks working on the back end, then you have a resource imbalance. You either need to add more folks with the right skill set, or you need to reduce the amount of work being done on the back end.
When you have a whole team, you spend less time waiting for other groups and bringing part-time participants up to speed, you lose less time due to communication delays, and individuals spend less time multi-tasking between multiple projects. Taken together, the benefits of whole teams can have a significant impact on reducing management complexity which makes it easy for teams to do more self-management.
Collocation Further Improves Coordination and Communication
Collocation is closely related to whole teams. Collocation is simply having everybody in close proximity to each other. This compounds the coordination benefit of whole teams.
If you have a large group of people that are not all near each other but they are all working towards a single deliverable, collocation may seem impractical on the surface. However, collocation at the whole team level is more important than collocation of multiple teams. Focus on creating whole teams that can collocate rather than collocating the whole group of people.

The challenge with both whole teams and collocation is that they often require either relocating people, changes to management structures, or both. However, the benefits of having a tightly coordinated team working towards a common goal far outweighs the difficulties associated with creating collocated whole teams.

 

via http://damonpoole.blogspot.com/2009/01/no-brainer-self-management-practices-in_19.html

Architecture Considerations

One of the common questions I am asked in the course of talking to organizations interested in document management is that of architecture.  Where does the software reside, how is it maintained, etc.  This is a good question, and the answer depends a lot on the organization’s IT infrastructure, support staff, and how the organization wants to interact with their document management system.  There are typically three different architectures an organization can choose from, each with advantages and disadvantages.   While exploring every single advantage and disadvantage of each is beyond the scope of this entry, I’ll try to highlight some of the most important differences in the following summary of each.

Traditional Client / Server Architecture: In this arrangement, the document management software is comprised of a server component which resides on a server within the organization.  The server software usually interacts with a database server also residing on the network, though some applications will use a built in database engine.  A client application is installed on each workstation, and configured to interact with the server.  Sometimes a client application installed on an end user’s workstation is referred to as a “thick” client.

An advantage of this configuration is that often times there is more functionality available to the end user within a thick client opposed to other configurations.  Heavy duty document editing or redacting is usually easiest within a thick client, and when it comes down to page level security usually your best bet is to use a thick client.  Probably the biggest disadvantage of this configuration is the deployment and maintenance of the client software on each workstation.  Every update or hot fix needs to be applied to each workstation, and if the organization isn’t using automated deployment tools, this can become a big task.

Web Server Based Architecture: Many document management solutions provide the ability to access information through a web browser rather than an application loaded on the end user’s computer.  This architecture is often referred to as a “Thin” client.  Any workstation in an organization with a web browser, and possibly some browser plug-ins, will have the ability to log into and access the document management system.  Quite often, organizations can provide for access to the system from outside of their firewall for employees working from home or traveling, or to provide portal or collaboration functionality to individuals within and outside the organization.  Just as with the previous configuration, the server software will reside on a server within the organization, and will interact with a database of some kind also on the network.  The addition of a web server, such as Microsoft’s IIS, Apache, or a proprietary web server will be needed to serve the information to end users.

The overwhelming advantage of this configuration is the ease of maintenance.  From an IT perspective, management of the document management solution is reduced to very few applications and devices.  Rather than a visit to each workstation PC to apply a hot fix, upgrading the server application is all that is required.  A disadvantage of this configuration is a mirror of the advantage of the previous configuration.  More often than not, functionality is streamlined to work within a web browser, and the most granular levels of security get harder to manage within a thin client.

One important consideration is that many document management solutions will provide for a mixed environment of both thick and thin client users.  If there is a need for both, be sure to inquire about this with the software vendor.

Hosted / ASP / SAAS Architecture: With the ever growing speed of internet access, there is a growing demand for hosted software applications.  Document management is no different in this regard.  Similar to a thin client architecture, a hosted solution provides access to the document management solution through a web browser.  The difference between the previous two configurations is that in a hosted solution, an organization pays a third party to host and manage the server side of the solution.  Organizations providing hosted solutions have taken great pains to provide the highest level of security, end to end encryption of data, speed of access and guaranteed uptime.  For many small organizations without  an IT staff or the internal architecture to support a document management system, this can be a very nice option.  Server side software updates, patches, nightly backups, etc., are all taken care of behind the scenes providing for a relatively low maintenance solution while still providing a very robust document management system.

The main advantages of a hosted solution are as stated, the relatively low maintenance required on behalf of the organization, and the ability to access the system from wherever a user may have internet access.  The main disadvantage is the same with any thin client configuration, the functionality is streamlined to work in a browser, and the most granular of page level security is difficult to manage.  Also a disadvantage, while not always being a fault of the host, is the speed of access to the system.  If an organization or user is connected through a slow or unreliable internet connection, the quality of access speed made be an issue.  Most solution hosts will work with an organization to determine a solution to any connection issues.

While I’ve mentioned three main architectures, there is also the possibility of overlapping architectures.  One arrangement I’ve seen more and more of is an organization who has a traditional document management solution for their day to day operation, that also makes use of a hosted solution to provide an off-site backup of their information (using automated tools), and to use a hosted solution to provide a publishing portal for information needed by users outside the organization.  The bottom line is there are a lot of interesting ways an organization can address their document management needs, and usually they’ll make use of one of the above mentioned architectures.

 

via http://dmtalk.com/index.php/2009/01/architecture-considerations/

Don't Use Scrum

 

Models

Once upon a time, people believed that the earth is flat and the sun revolves around it. Then Copernicus came and offered the theory that matched the observed reality better. Then Newton attempted to explain why planets revolve (among other things). Then Einstein pointed out where Newton was wrong and straightened the theory to match the needs of his contemporaries – very soon to be superseded by a bunch of crazy physicists who invented quantum mechanics. These days, we pretty much know for a fact that quantum mechanics is wrong and we are in a search for a better theory. Now, all of the above theories have a couple of things in common:

• they are purely theoretical and idealized models of the real world (yes, even the flat earth theory)
• they were significantly better than their predecessors
• initially, they fit the needs of their creators
• they became the model to use, the latest craze, the offcial religion, the mainstream, displacing the previous models
• over time the number of corner case where the models broke increased to the point where they were unusable
• yet, the evangelists of the existing model fought hard to preserve them and to prove that suggested alternatives were pure lunacy (which, to be honest, 99% of them were)

What does this have to do with Scrum?

Well, everything – you saw that coming, didn't you? All the development process definitions are theoretical models of the real world, just as physical models are. And they break in very simmilar ways. And the new, alternative methodologies are ridiculed and fought with similarly. And eventually become mainstream and break too.

So, don't believe for a second that Scrum (or any agile methodology) is immune from this process. Scrum will be replaced by something better – just you watch. Right now, agile methodologies have just become mainstream, having relegated the previous models to the status of the 'flat earth' theories. Which is precisely the moment when they are doomed to start breaking in more and more situations – and the paradox is that many these scenarios were created by the very fact that we have become so much more efficient thanks to using agile development methods!

So the question is: does Scrum (or any other agile method) still work now? Well, I would say that it mostly does, but in some scenarios – it works just barely. And if you attend the ScrumMaster training, they will introduce you to all the caveats, patches and workarounds that are being currently applied to the „real thing” to make it still work in a real world. Like – what to do if your customer or a product owner is not embedded in the team – in the case of my team, they are on the other side of the planet! Or, what do you do when the customer will absolutely not accept anything else than a fixed price contract? Pass on the contract, as the Good Book seems to suggest? But see, I happen to like the money that the customer is offering, so I thing this is just crazy talk.

The Alternative

I can safely say that there exists one development methodology that is guaranteed to be matching your needs always: it is called „common sense”. Really, you happen to know much better what the needs of your projects are then Scrum consultants. You do need to familiarize yourself with what they are offering, you do need to read all these books. There is a lot of good practices described there, that will make your life easier. If you are crazy about certifications, by all means, go on, make yourself a ScrumMaster. But then – pick and choose. It is ultimately your responsibility to design your process so that it works for you. If the result ends up not resembling the „standard” process – that's too bad for the standard.

via http://agilesoftwaredevelopment.com/blog/janusz-gorycki/dont-use-scrum

Programmer’s dozen - programming best practices

Take control of your code with these programming best practices from Kevlin Henney. At JAOO Aarhus 2008 Kevlin used a trash can, vampires, a train wreck, whiskey and much more to make you understand and remember his 13 constructive points (a programmer’s dozen) about programming and code smells. As this is a must see, we made a video of the presentation for you:

The 13 points made by Kevlin are:
0. Prefer code to comments.
1. Follow a consistent form.
2. Employ the contract metaphor.
3. Express independent ideas independently.
4. Encapsulate.
5. Parameterize from above.
6. Restrict mutability of state.
7. Favor symmetry over asymmetry.
8. Sharpen fuzzy logic.
9. Go with the flow.
10. Let code decide.
11. Omit needless code.
12. Unify duplicate code.

Is it a peacock and a rooster?

Some 20-25 years ago as a teenager in School, a teacher once told me and the rest of the class the following metaphor (as I recall it now).

Two persons look at a roof top and distinguish a bird sitting at top of the roof. The distance is far so it is difficult to see the details, but one of the persons suddenly says: "what a beautiful peacock!" The other one looks at him bemused and says "that's not a peacock. It's a rooster!"  Then the two persons burst into an argument on this matter which lasted for quite some time.

What my teacher then told us is something important I think: Sometimes the truth is different for different people, but it still is a truth. It is important to remember, and respect, that there may be different versions of the truth which are correct in their own way.

I have come to reflect upon the essence of this very often over the years and I feel I would like to write some fine grained aspects of this:

The "truth" is context dependent and it is not always absolute.

Both persons arguing actually believe they are right, and in a fundamental way, they are therefore both correct. In order to understand this perspective it is vital to consider the context. The context in this example is quite complex since you must take into account things like eye sight capability and what the actual conception of what a rooster and a peacock are. Nevertheless, the person with poorer eye sight or perhaps lesser bird knowledge, still perceive his belief according to his own context. That makes it a truth.

Most likely, you are thinking the same thing as I have been thinking for so many years. But. There is an actual correct truth. Take mathematics for instance. There always a correct answer there, right? I mean, 1+1 will always be 2, correct? However. If the context is Boolean logical mathematics, 1+1 is actually equal to … 1. So whatever truth you have, for sure someone can probably pull a rabbit from a habit by bringing a completely different context into the picture which will make your established truth completely wrong.

For sure, if the two persons go closer to the building they probably will discover the correct truth. And this is actually where the story continues:

The two men found they could not reach an agreement so they decided to walk closer to get a better look. Soon they realized that it was not actually a real bird, but a statue of a bird. More than that, once they got to see the details neither of them could really tell what kind of bird it was.

Ah! Now we are getting closer to reality I think. For sure, there are often truths out there which can be found when digging deeper. But very often we find ourselves in a situation where we have to accept a good enough truth. One that works for the context we are in. This is a practical truth. But let’s turn our attention in another direction for a while.

Always choose the most important topic for discussion.

Have you ever found yourself having several discussions going at the same time with someone? Well, it happens to me unfortunately. Before having agreed upon an initial deviation of opinion, the next deviation comes up. To make things even worse, the knot to untie the first disagreement depends upon the second one to be solved. This can continue in a discussion until you find yourself arguing over 5-6 very different topics at the same time. Don’t do this. Set your foot down early and stick on track is my advice.

That’s one of the things that make discussions such a difficult art. Having several different unfinished topics going on at the same time I consider a sure path to hell…

But let’s us turn our focus to the two persons again. The first person said that it was a beautiful peacock. His main point at the moment probably was that it was beautiful, not necessarily that it was a peacock. That is a sensation he had and one that he felt he wanted to share. By doing that he found himself ending up in a discussion that probably had nothing to do with his original sensation. I think both persons could have given in to this less prioritized matter by saying something like “oh, perhaps you are right”. Then they could have focused on the original and probably more important message: beautiful or not…

I’ve met some people who loves throwing arguments in a heated discussion and who might even provoke a person in order to get it. I’ve also come across some people who have even lied about their true opinions in a discussion since they wanted to test the opponents’ arguments. Truly, this is a dangers road to walk I think. In the end the person often end up confessing that he had not really been speaking his true opinion. In most cases such a person end up losing my trust. And whoever listens to a person that you don’t trust?

So let’s get back to a final aspect of the considering truths in a discussion:

Considering someone else's truth is about showing respect and having patience.

With the above aspects in mind it becomes a whole lot easier to actually listen to people in a discussion/argument. Quite clearly the more confident of an opinion we become, the more difficult it is to be open and appreciate different ways or slightly different truths.

In a very important perspective, listening and appreciating other peoples truths is the most important component in a discussion. You will never truly "win" a discussion if you don't pay the other person respect by giving him the proper feedback that you are actually listening and understanding his truth. I also think it is best if this effort is sincere. Be open to that you both have a version of the truth (or even that you can be wrong) and you will both probably gain a better understanding of a deeper truth in the end. I firmly believe this is the key to successful communication and I am constantly trying to become better at. And I must admit I sometimes fail due to lack of patience... But, hey, the road to perfection is a long and fun one!

 

via http://softwarearia.com/2009/01/17/IsItAPeacockAndARooster.aspx

Rules of Database App Aging

I mentioned I’ve learned some rules of how database apps change over time, now that I’ve done a few dozen. They are:

1. All Fields Become Optional

   As your dataset grows, exceptions creep in. There’s not enough research time to fill in all your company profiles, there’s one guy in Guam when you expected everyone to be in a U.S. state, there’s data missing from the page you’re scraping, you have to pull updates from a new source.

   Every field eventually loses that beautiful NOT NULL sheen, your code gets filled up with guard clauses of one kind or another and every <div> in your template is wrapped by an if statement. And this happens to foreign keys, too, so OR IS (NOT) NULL sneaks in and left joins mutate into outer joins.

   This is by far the biggest effect on apps over time. It’s getting to the point that I can gauge age by eyeballing the number of fields that retain their NOT NULL constraint.

2. All Relationships Become Many-to-Many

   Some guy works in DC but lives in Virginia, so he needs two Locations. A new type of incoming email needs to be shoveled out to different feeds. A state has both a primary and a caucus. Someone eventually realizes categories never really were mutually exclusive.

   The modern database paradigm is defined by relations, so of course that’s what falls apart as soon as you get an app into production. The urge to hack is overwhelming, fudge in a little denormalization or duplicate a row and the pressure’s off for now, but it’s like freezing a bottle of water, it always grows and breaks worse in the end.

3. Chatter Always Expands

   All the little oddities that change database schemas affect the user presentation as well. Chatter is the intro and outro text around the content of a page that almost no one ever reads. But it has to be there to explain what’s going on, the source of information, why things may seem peculiar, the limitations of the dataset, etc. Add in the difficulty of writing succintly and chatter grows until you burn it all down by rebuilding the app.

And when you do rebuild the app from the ground up, you have your chance to slip some NULLs back on, renormalize your data to have easy one-to-many relationships, and present the data in a self-evident and consistent fashion. Then, about a week later, there’s a politician who’s a Democrat but running for re-election as an Independent…

via http://push.cx/2009/rules-of-database-app-aging

Worse is worse

Recently, I was interviewed by a reporter who was doing a story on the difference between east coast engineers and west coast engineers (yes, that old chestnut is again being revisited). This in turn got me thinking about Dick Gabriel's classic note "Worse is Better", which is often credited with first articulating the distinction between the MIT (or east coast; after all, what else is there on the east coast) approach to engineering (do the right thing, no matter how complex it makes the code) and the Berkeley (or west coast) approach to engineering (make the code simple, even if it makes the user do more work).

The notion that worse is better has become something of a truism in the programming industry. The usual examples are the C language (worse than lisp, but it dominated anyway), Unix (or more recently, Windows) as opposed to Multics or VMS, and (in a completely different arena) VHS tape over Beta. Each of the dominant technologies, it is pointed out, was worse than the alternative, but the worse technology became the standard anyway. The moral to the story, or the reason that people bring the principle up in argument, is to convince whoever is on the other side of the argument that we should set our sights on the quick and dirty, less elegant solution to a problem, because "worse is better."

Of course, this received wisdom is just so much crap. The arguments simply don't hold up. But the damage that this principle has done to the technology industry is real, and we should start pushing back before we do any more harm than we already have done.

First, the arguments. Gabriel's original writing (which can still be found, like everything else, either through Google or by going here) makes an interesting read, and a number of good points. Reading it again not only reminds one of how well it is written, but debunks a lot of the usual common knowledge about the article. For example, the piece never contrasts the east coast and west coast engineering styles; the two groups it talks about are the MIT/Stanford style (one group) and the New Jersey style of design. Not nearly so interesting in these days as the notion that the contrast in styles is between west coast and east coast.

The rest of the paper is an excellent analysis for why Lisp lost out to C as a programming language, even though Lisp was a superior language. Or at least superior on the grounds that Dick found most important. But this doesn't necessarily show that Lisp was in fact superior to C; it can just as easily be taken to show that the metrics that were cited in the article were not the ones that were taken to be most important by those choosing a programming language. The fact that C produced faster code, was easier to master, was easier to use in groups, and ran well on less expensive hardware were not considerations that Gabriel found important. But others did. On those metrics, the dominance of C as a programming language was an example of better is better, not worse is better.

The old chestnut of beta and vhs is open to the same sort of alternate interpretation. On the "worse is better" interpretation, the superior quality beta was beaten out by the clearly inferior vhs tape format because of some inexplicable perversity of human nature (or the machinations of clever marketing people, or the hubris of Sony, who owned the Beta brand). But the vhs tapes were capable of recording longer programs on a single cassette, and could be played on cheaper recorders, and had a number of different suppliers. Thus there were a set of metrics on which vhs was the superior technology, and these metrics seemed to be the ones that most in the market found to be important. Vhs beat out beta not because worse is better, but better in some dimensions (cost, time to record) beat out better in other dimensions (picture quality).

Even the case of Unix vs. Multics misses an important point. While Multics may have been a better operating system, Unix had the twin advantages of actually existing, and running on a wide variety of fairly cheap hardware. Windows (and DOS) ran on even cheaper hardware, and though it was easy to argue on any technical grounds that you wanted that Unix was a better OS than DOS, the property of existence on really cheap hardware turned out to the the metric of goodness that appealed to the customer. The emergence of Linux as a real choice is beginning to give us more evidence in this particular choice; perhaps a better OS is something that people will choose when other things are equal. But when they chose DOS over Unix, things weren't equal.

In all of these cases, there is an alternate interpretation of the choices that were made that lead us to the conclusion that worse is not better. Instead, what we see is that better is a complicated notion, and can depend on a variety of different metrics. It may be disappointing to find out that what we geeks think of as better may not be what our customers think is better. But finding this out shouldn't surprise us too much.

Of course, worse is better is a much catchier slogan than better depends on your goodness metric or some other, equivalent phrase that would actually reflect what was going on. And there is wisdom and insight in the original article that can be used by designers, even under the catchier (but less historically accurate) slogan.

My problem with the slogan is that it has become a catch phrase for those who either haven't read the original article, or have read it and either have forgotten what it really talked about or never understood it in the first place. As a catch phrase, it is often used to justify shoddy design, or following the crowd rather than doing what is right, or as short-hand for the real claim that our customers are too stupid to either appreciate or deserve high quality products. Why spend the time doing things right, this line of reasoning goes, when we all know that worse is better. You are far better off giving the customer something that you know is less than what you could produce, because they (those simple customers) will think that it is better.

The end result of this thinking is sloppy products that don't work, are hard to use, or are unreliable (or all of the above). We try to convince our customers that this is the way software has to be, and then turn around and convince ourselves that they won't pay for anything better. But we short-change our customers, and we cheapen our craft, when we put up with this sort of thinking. When the original article was produced, I don't think that this is what the author had in mind; even if it was, it is time for us to reject the simple-minded interpretation of the slogan, and start putting out software that really is better (on the dimension of goodness that our customers have, not necessarily our own).

via http://www.artima.com/weblogs/viewpost.jsp?thread=24807

“No-Brainer” Self-Management Practices in Agile

While there may be some confusion over the meaning of self-management in Agile, not to mention some contention as to whether or not it is a good idea, many of the Agile practices contribute to self-management without requiring the wholesale adoption of self-management. For simplicity, I will define “self-management” as anything that is done by a team member that would traditionally be done by a manager. The practices contribute to self-management either by reducing the amount of management required or by encouraging the delegation of management tasks to team members. Some practices do both. As a result, managers have more bandwidth for dealing with things which are a more leveraged use of their time.
The Agile practices related to self-management include:

• whole teams
• collocation
• short iterations
• using a scrum master
• using a product owner
• user stories
• maintaining a product backlog
• assignment of tasks by team members
• estimation of tasks by team members
• producing a burn down chart
• retrospectives

Simplifying and Redistributing the Management Load
Let’s start with a simple example: task assignment. Traditionally, tasks are assigned to engineers by their manager. In Agile, tasks are selected by the engineers themselves. It may take some time for everybody to get used to this and to work out the kinks, but the practice itself is not particularly earth-shattering.

Certainly, somebody may find themselves struggling with a task they don’t fully understand, or one engineer may accomplish a task more slowly than whoever the manager might have chosen to do the task. But is that really all that different than what typically happens when the work is assigned by a manager? How likely is it that a manager has enough time to devote to the minutia required to optimize task assignment? Wouldn’t it be better for the manager to spend their time mentoring team members on how to pick tasks that are appropriate for them while at the same time encouraging engineers to stretch their capabilities?

 

via http://damonpoole.blogspot.com/2009/01/no-brainer-self-management-practices-in.html

How Unit Testing Could Have Saved Lives

A recent article in The Independent describes a fascinating study.  Researchers found that forcing surgeons in a London hospital to implement a single new procedure caused the death rate after surgery to fall 47 percent.  Complications likewise fell by 36 percent.

These are enormous numbers:

Donald Berwick, the president of the US Institute for Healthcare Improvement, said of the innovation: “I cannot recall a clinical care innovation in the past 30 years that has shown results of [this] magnitude.”

The change?  The surgeons were required to run through a simple checklist before and after every procedure.

Now, this checklist [pdf] is not lengthy.  In fact, it’s less than twenty questions.  Even more strikingly, the questions are not complicated: before administering anesthesia, has the patient confirmed his identity?  Does the patient have any known allergies?  Have we confirmed that this is the right foot?  I’d wager that the most common reaction of people seeing this checklist is “you mean these things weren’t being asked already?”

The thing is, these questions were already being asked.  The difference is that they weren’t being asked in systematic way.  Members of a surgical team assume that these obvious things have been checked because they’re so obvious.  “Of course this is the right patient.”  “Of course we asked if he’s allergic to the anesthetic.”  “Of course we marked the right limb for amputation.”

The problem is, these things are sometimes done incorrectly, or missed entirely.  Not often, not usually; just sometimes.  Forcing doctors and nurses to use the checklist caught these “sometimes” problems, and dramatically improved patient safety in the process.

From this study, we can draw the following conclusions:

1. Even obvious errors are easy to commit, and once committed, can have terrible consequences.
2. Those who overlook these problems can be highly-trained and intelligent people.  It appears that you cannot out-train or out-brain our human tendency to err from time to time.
3. Instituting a rigorous system to check for errors lowers their occurrence dramatically, compared to existing processes that check for them informally.

Laid out this way, these statements are probably starting to look familiar, because they are exactly like the arguments for using unit tests in programming.  As a software developer, I look at this list and think we knew this stuff already!  We knew it, and we came up with a way of mitigating the problems caused by our own flawed humanity.

The errors that this checklist catches are the kind that kill people.  Thousands of people, every year.  If we, as an industry, had somehow conveyed the efficacy of systemic testing with the medical field, this checklist might have been implemented sooner, and lives could have been saved.

The fact that ideas like this don’t flow across fields is not surprising.  Industry-specific myopia reflects our human tendency to form into groups and distrust outsiders.  Despite this, I do believe it’s worth struggling against.  Perhaps the field of architecture could provide useful insights into the construction of software (perhaps it already has).  Maybe the next big idea in architecture will be inpsired by chemistry, or biology.  Maybe the next time I open my editor I should think “first, do no harm.”

It certainly couldn’t hurt.

 

via http://codeulate.com/?p=23