Copyright 1994, Jack G. Ganssle

Abstract

Slobs pervade this industry. Hey, there's nothing wrong with a mess... unless it costs valuable time.

Published in EDN in October, 1994.

There he sits... the organization's engineering guru, respected but somewhat feared because of his arcane knowledge. His desk is a foot deep in paper, the lab bench a mess of old food containers and smoldering solder dripping. Tools and resistor clippings threaten to short out any test system carelessly placed on the bench. Wires crisscross every square inch of tabletop - scope probes, clip leads, RS-232 cables - all going somewhere... though perhaps no one really knows their destination.

Ask the guru for a piece of paper and be prepared to wait. He burrows frantically through the mess. Usually the paper never comes to light. It's lost. Don't worry, though - he'll recreate it for you as soon as he has a chance. Probably the PAL equations he'll come up with will be about right, but if they're not - no problem! He's already debugged that circuit twice, so is quite the expert.

To many managers tolerate this level of chaos. Me, I'm a reformed lab pig. My twelve step recovery program revolved around living in tiny places - a VW microbus, many boats - which force you to be organized simply to deal with the complete lack of space. There's no room to be a slob on a small sailboat! Fortunately my personal quest for organization rolled over into the lab when I discovered just how much time I saved by putting things where they belong.

Mess and clutter quite simply decrease productivity. Those few minutes a day spent putting things away save hours of searching. Sweep the solder drippings and wire segments off the bench once in a while and your incidence of catastrophic failures will plunge dramatically. I write this just minutes after hearing of one of our instruments that was vaporized by a carelessly misplaced high voltage line.

An organized lab promotes correctness. How many times have you seen engineering changes that never quite made it into production because someone forgot to write them down? Or, was the notation made on the corner of a napkin that accidentally was used to wipe up a spill before it was disposed of? The Basics

When starting to debug a new project remove everything from the bench and sweep it clean. A quick wipe with a damp cloth removes those accumulated coffee stains. Then, put everything not absolutely needed back on the shelves. This is the unique chance we get once in a while to remove the clutter, so be relentless.

Any embedded project will require at least a computer and a scope. Decide what test equipment you'll use continuously, and which will be used only on an as-needed basis. All too often even a simple embedded system has some sort of communications link requiring an extra computer as a source of data. I like to use a laptop for this as it requires little bench space.

Be sure you can easily reach the computer's frequently-used connectors. If two different devices must share an RS- 232 port buy a switch box ($10 from Jameco at 800-831-4242) and reduce the wear and tear on connectors... and your back. That two second switch flip will also save a few minutes of cable fiddling.

Don't work with unacceptable power distributions. Too many of us spend half our lives swapping power plugs. Buy an outlet strip - Techni- Tool (215-941-2400) sells strips of 15 for only $64.

Miles and Beryl Smeaton sailed their aging boat around Cape Horn many years ago with expert boatbuilder John Guzzwell as crew. When the boat flipped in 30 foot seas and the hull cracked open, Guzzwell was shocked to discover that all of the Smeaton's tools were rusty and dull. As water poured in he carefully sharpened and cleaned the tools before undertaking the repairs that eventually saved their lives.

The moral is to buy good tools and take care of them. You'll live with those dikes and needle nose pliers for weeks on end. Buy cheap stuff and your blood pressure will skyrocket every time you can't clip a lead close to the board. Keep them organized - get a little toolbox to keep them from falling onto the floor and getting lost.

Some engineers complain that others swing by when their lab space is unattended and steal things. Tools have a terrible habit of mysteriously disappearing. One solution is to make each engineer and technician purchase their own tools. This might be a bit draconian, though. I have had pretty good luck by keeping my space neat, and making it clear that things will not disappear.

This works in a small company, but I'm afraid that in a large organization such tactics may be in vain. One fellow I know complains that in his mid-sized company each morning vital computer parts are missing or mysteriously swapped. That super VGA controller now seems limited to CGA resolution, or the nice keyboard turns into a coffee-stained intermittent beast. I wish I knew how to deal with such pettiness, but perhaps the sheer size of some organizations is accompanied with similar problems from big cities.

How is your soldering equipment? A vacuum desolderer is great for making large scale changes, but during prototyping I find it's often easier just to hack away at the board, mounting chips on top of chips and using plenty of blue wire.

During the first few days (or weeks) of bringing up a new embedded system I often find myself making lots of little modifications to the system. A hot iron always at hand is critical. After things start to more or less work I start testing I/O interfaces by writing low level drivers and exercising the code, making software and hardware changes in parallel as needed. The code changes much faster than the wiring, so it seems wasteful to keep an iron hot all the time. JDR (800-538-5005) sells a neat $30 cordless soldering iron that heats in seconds, the ideal thing for those infrequent modifications.

Being an immensely stupid person I require vast quantities of clip leads. Most of my ideas are wrong, so I save a tons of time by using a clip lead to try a design change and see what happens.

Clip leads have a very short lifetime in a development lab. Accidentally connect Vcc to ground and the plastic tip melts horribly. I hate it when that happens. We used to send a runner to Radio Shack occasionally to replenish our supply but found that "the Shack" can't keep up with our needs.

Now we buy 100 clips at a time and have a high school kid solder up 50 leads. It gives us an infinite supply for a while, and may help a fledgling engineer find his true vocation. (Bring a part-timer in from your local high school to help maintain the lab. The cost is minuscule, the lab will be better off for it, and you'll show one more kid that there are alternatives to slinging burgers).

Beware of clip leads in high speed digital. Connecting a switching signal to ground with even a short lead often just does not work. The lead looks like a transmission line and presents a rather high impedance at the frequency of your circuit; so high that it may have no effect. Warm up that portable iron and solder a short shunt from the signal to ground. Software

An embedded system's hardware is never far from the software. Personally, I believe that at the very least the person responsible for designing the hardware should provide low level software drivers to the software team. It's unfair to dump a barely-documented complex system onto a firmware engineer's desk and say "now it's your problem".

Be sure your lab area is set up to insure that you can do serious software development! Clearly, your computer must include the properly installed compilers and assemblers needed for the project. Just as important as quality hand tools are the debuggers, make utilities, and other software resources needed to quickly and painlessly write compile, and test the code.

Set up the environment with a Make utility so you can compile/assemble without twiddling compiler switches. Only amateurs develop by tediously hand-assembling instructions into the debugger. Never make patches in the debugger and upload these to a file for later use. Change a source file, so you can alter the comments as required to write drivers that are beautiful to behold and a joy to use.

Develop your code with the environment the firmware folks use so they can immediately integrate your test routines. It drives me crazy to see the lousy planning so endemic in this industry. All too often laziness or stupidly reign, so one engineer does his work under a particularly obsolete version of a compiler, while the rest of the team must painfully massage the code into something compatible with the version they use. Why work so hard?

Hardware design requires as much software support as does the firmware. PALs, PLDs, and FPGAs let you create much of the hardware design late in the game and so are a wonderful thing. Be sure your bench is set up with all of the tools you need to edit and compile these.

Unfortunately, too many of the tools are not happy running under a standard DOS or Windows environment. You may need a special boot sequence to free memory. Our PAL compiler is very unhappy with Microsoft's EMM386 for some reason; our PC layout software will not run from a Windows DOS shell, and the FPGA compiler requires every resource each system has... and then some. One solution is to use a boot scheme that asks which software you'll be running, and then invokes the proper AUTOEXEC.BAT and CONFIG.SYS. Another is to wait for all of this stuff to be reasonably available under Windows.

All too often I speak with a sad engineer whose PC is a 286 running Windows in 1 Mb of RAM. Decent computers are so cheap, and modern tools demand so many resources, that it's simply false economy to make do with yesterday's technology. And yet, no company can toss out all of the computers every 8-12 months to follow the latest offering. Clearly, the minimum engineering machine today is a 486 with 8 Mb of RAM and plenty of disk space.

If you are saddled with an antique machine measure the time spent waiting for compiles. Make a persuasive argument to the boss based on your salary plus overhead. Let him know just how much money he'll save with a $2000 - $3000 investment. Documentation

I wrote about setting up a drawing system in September. The drawing system is only as good as the information that goes into it. All too often the frenetic pace of debugging hardware tempts us to be less than careful about writing down changes.

Resist this temptation. Your company is paying you to debug a prototype for one reason only: so it can be turned into a working production system. If you carelessly forget to document modifications the company will need at least one additional PCB revision, which you'll have to debug all over again. This is a terrible waste of money. A wise manager of such a documentation-free engineer will either retrain or fire the individual.

Avoid taking notes on scraps of paper. The best solution is a meticulously maintained engineering notebook. Write everything down, clearly and concisely. The good sisters of my grammar school all but committed suicide over their failed attempts to teach me penmanship, so such clarity is a particular headache for me. I've learned to slow down and print, since most of the time I can't read my own script.

Some engineers document directly into a computer file. If your environment is so perfect that you can always seamlessly switch to the editor perhaps this works - if you keep backups. In most cases, though, being stuck in a program you can't exit you are forced to make notes on paper.

Use one set of schematics to record changes. This is your master development drawing set. Staple them together and clearly label them as your masters.

In the movie Jurassic Park one brilliant slob controls a sea of workstations. When he disappears in his quest for the big score we see his desk covered in filth, with papers and books scattered helter-skelter, old food remnants decomposing into new forms of life, and McDonald's wrappers blocking computer ventilation ports. Why is this image such an accepted icon of our industry?