More Programming Pearls book

The Origins of Method R

This thing called “Method R” is a method for optimizing the performance of a system. The reason Method R is special is that it is itself optimized—it is designed to keep you from wasting your time. This article is about where Method R came from.

In the Beginning…

In the 1990s, the world was a boomtown for Oracle. A new age of relatively inexpensive minicomputers democratized IT all over the world. Instead of having to buy time on another department’s computer, you could just buy one of these new “open systems” from a vendor like DEC, Sun, Sequent, or Pyramid and run your accounting system on it yourself. No more chargebacks and scheduling and boring 1980s-era accounting packages.

People all over the world were doing it. But all the exciting new hardware and software had—guess what—a lot of growing pains. There were all kinds of problems, of course, but the ones that interested me in particular were the performance ones. There were a lot of performance problems. And there was a lot of money available for fixing those problems. This of course attracted consultants like moths to a flame.

Or flies to a trash can.

It was great being a consultant, though.

One perverse feature of the era was that being incompetent could actually increase your earnings. Through the magic of a time and materials (T&M) contract, the longer you took to solve a problem, the more you could get paid.

Normally, you would expect that healthy competition would shut down this particular perversion, but two important cogs in the gear-train weren’t ready yet:

  • The supply of expertise couldn’t match demand. This provided no incentive for consultants to improve, or for prices to fall.
  • Not many businesspeople were capable of understanding whether their consultants were doing a good job or not. This, also, provided no incentive for consultants to improve, or for prices to fall.

My motive was different. I wasn’t trying to camp out at a customer and pump revenue out of them. I was trying to make as many customers happy with my employer’s software (Oracle‘s software) as I could. The folks who were exploiting my employer’s customers on the more-revenue-for-incompetence plan were my enemy.

We Need a Method

By the late 1990s, I was in charge of nearly a hundred people at Oracle Corporation, solving performance problems at customers. I had good, smart people working for me, and we waged war against the poachers. My problem was, I couldn’t train people fast enough. It wasn’t for a lack of good teachers. It was for the lack of a good method to teach.

We did teach tips and tricks, but actually improving a system still required a fully tipped and tricked consultant to use a lot of un-distilled intuition and experience. My measurements at the time indicated that it took about 7 years for a smart college graduate to become a great performance optimizer. But by then, a consultant would be burned out from too much travel.

It was a treadmill. It led nowhere.

So We Made a Method

In late 1999, I left Oracle to start a business. Our mission was to create whatever it would take to make two things happen:

  1. Create a repeatable, teachable, optimized method—one that doesn’t waste anyone’s time—for optimizing a system.
  2. Create the software tools required to make that method actually usable in Real Life.
  3. Teach our customers everything we know, so that they can make well-informed decisions about performance.
Optimizing Oracle Performance book

Method-building was my focus from 1999 until 2003 when we published our first O’Reilly book, Optimizing Oracle Performance by Cary Millsap with Jeff Holt. The whole time we were developing it, we were consulting and teaching, exposing everything we knew to whatever audiences we could attract.

Our Influences

All the time we were developing Method R, we were reading the works of other scientists, trying to learn how people in non-Oracle domains had solved problems like the ones we were facing. Here are some of our major influences.

Dave Pelz

Dave Pelz's Putting Bible book

Dave Pelz is a former NASA scientist who became a golf researcher and coach. He teaches using the scientific method and real data. A single sentence from this book about golf perfectly describes the problem that many companies with performance problems faced throughout the 1990s:

There’s nothing worse than working hard on the wrong thing, expecting improvement from it, then ending up with nothing.

Dave Pelz, Dave Pelz’s Putting Bible, p215

The structure of his work—measuring the right thing, to ensure that you’re always focused on the right problem—inspired us and helped us explain our mission to others.

Jon Bentley

More Programming Pearls book

The very first paragraph of Jon Bentley’s More Programming Pearls says:

The stethoscope is a simple tool that revolutionized the practice of medicine: it gave physicians an effective way to monitor the human body. A profiler can do the same thing for your programs.

Jon Bentley, More Programming Pearls, p3

He then goes on to show the output of a profiler: a simple table that explains the response time—as a user would experience it!—of a computer program. That table profoundly changed my expectation for what kind of performance data I insist upon seeing. It was the genesis of the Method R receipt for response time™ idea.

Anjo Kolk

YAPP paper

So, can you create a profile for an Oracle program in the form that Jon Bentley describes? It turns out that the answer is yes. Our friend Anjo Kolk, an Oracle employee at the time, taught Jeff and me that we could create precise, accurate profiles using Oracle trace data.

This opened the door to creating software that could provide performance information in the format that a business needs to make well-informed decisions.

Eli Goldratt

The Goal book

Eli Goldratt’s The Goal transformed me. It was one huge, continuous a-ha. It solidified in my mind how to define the relationship between business and technology. It identified our real goal in a fog of millions of things we could designate as our goal:

Your goal is to simultaneously increase these metrics for the business: net profit, return on investment, and cash flow.

Eli Goldratt, The Goal, p48

Goldratt’s goal is our goal. His problems are our problems, just with different nouns. For example, Goldratt asserts that cost accounting motivates behavior that is contradictory to manufacturing optimization. Change just two nouns, and you arrive at our assertion that hit-rate accounting motivates behavior that is contradictory to Oracle optimization. The beauty of this awareness is that Goldratt’s analysis of his assertion informs the analysis of our assertion, all the way into the details.

Kent Beck

Extreme Programming Explained book

I loved Kent Beck’s Extreme Programming Explained every bit as much as I loved Goldratt’s The Goal. Beck’s book—harmonious with Goldratt’s—is about setting the right targets and using the right measurements to get there.

Make sure what you are doing has business value, meets business goals, and serves business needs. For example, solving the highest priority business need first maximizes the value of the project.

Kent Beck, Extreme Programming Explained, p25

Wrap-Up

We stand on the shoulders of giants.

If you’re interested in how all these thoughts hang together, you might enjoy my second O’Reilly project, called How to Make Things Faster.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *