My Move Out of Index Funds

I have struggled for years with the logic behind both asset allocation models and indexing, two tenets of modern investing, ideas I’ve used most of my investing life. Unfortunately.          

I recently posted my arguments against asset allocations. My frustration has been that asset allocation eventually got me to invest almost half of my wife’s and my life savings in bonds that historically barely keep up with inflation, and then in 2022 lost over 13% of their value. I’m already moving away from that model.

My next move is away from index funds. The very reasonable argument for index funds is that they may be your best bet for investing in equities rather than trying to “beat the market” by owning individual stocks. I agree that the easiest way to win in the stock market is owning low-cost index funds. A reasonable person can establish an allocation formula for their equities, for example, 70% of stocks in the US and 30% international, balance back to this allocation every year or two, and do little else for decades. It works and I highly recommend it.

But I have a major issue with index funds. By their design, when you buy an index fund, you are overallocating to overpriced stocks, and underallocating to underpriced stocks.  What is overpriced and what is underpriced is subject to great debate, a debate index funds settle by declaring nothing is ever over or underpriced, a tenet of indexing, and that the best estimate of what a stock is worth today is what it is selling for today.

I disagree with this. Stocks often get overvalued and undervalued by almost any valuation measures. Great examples are the Dot-Com Bubble of 2000 and the Great Recession of 2008. But what the intrinsic value of a stock is at any given time is an endless and emotional discussion, a little like religion, and it won’t get settled here.

Let’s just say that I don’t buy this argument that stocks always trade at a price that best represents the value of that company at that time. A live example (and there are endless more) is Tesla. It recently had more market value than the market value of all other automobile manufacturers in the world combined.

Read that last sentence again slowly. No rocket science or deep financial analysis is required to recognize that this is either an overpriced stock in the long term, or all other automobile manufacturers in the world are collectively underpriced.

During 2022, from high to low, Tesla lost over 70% of its value without any significant change in the company. That’s how much variation there can exist in a stock that supposedly is trading at its fair value. And my bet is that at its low, it’s still way overpriced.

The most basic investing maxim that I learned in high school is that you should buy low and sell high. But indexes, by their design, buy high and sell low. It’s not intentional but it is part of their design, a design that is simple and easy, but causes other problems. An index fund mimics the composition and performance of a financial market index, such as the S&P 500 index.

So if in early 2022 you bought an index fund that tracts the S&P 500 index, you are overallocating your money into many overpriced stocks like Tesla, and underallocating your money away from many underpriced stocks, the opposite of what one should do.

There are other advantages to owning individual stocks besides this issue of index funds overbuying overpriced stocks. Although it is easy to find low-cost, low-turn index stock funds, stocks have even lower fees. Many brokerage firms have zero fees for their accounts and zero fees for buying and selling stocks.

So by owning stocks you can reduce any fees to zero sans spreads. If you then hold your stocks for long periods, real fees approach zero. You can have your entire life savings maintained online for years, with full access to the firm’s tools and services, and never pay a fee, directly or indirectly. How these firms make a profit is another discussion.

The arguments against owning individual stocks are many and valid. It takes a lot of time. It requires research and knowledge, although not necessarily any special cognitive skills. The best argument against it comes from Daniel Kahneman, the author of “Thinking, Fast and Slow.” He convincingly suggests that people have too much confidence in human judgment, specifically with forecasting.

His research on stock picking, a form of forecasting, shows overwhelming evidence that professional stock pickers have awful track records doing what they’re purportedly paid to do. It supports his claim that human are terrible forecasters, whether it’s stocks or anything else requiring a long-term view.

Finally, one more strong argument against owning individual stocks is that most money in the stock market is made in very few stocks. Therefore, regardless of one’s skills, the chances of hitting on these few stocks is slim, no matter how good one might be at their trade, and success mostly comes down to luck. And the best way to counter luck is to own all the stocks in a low-cost index fund.

Acknowledging these challenges, people have endless ways to try to make money in the market. I’ve already mentioned buying underpriced stocks and selling them when they return to their intrinsic value or higher. There are strategies for out-smarting other bad investors, such as the ones who bought Tesla in the last several years. These approaches are sometimes known as the “bigger fool,” assuming there will always be people willing to pay yet more for an already overpriced item.

Timing and technical exercises are endless, including the Santa rally and the "sell in May and go away" theory that tries to beat the historical underperformance of stocks from May-October. Apparently, there’s an argument for buying and selling anything based only on the day of the week or even the time of day. And technical analyses are endless, a little like reading tea leaves.

Most have some merit and for sure, someone has always made money doing all of them, which countless articles give undue attention to.

I bought my first stock while in college, Niagara Mohawk Power, a utility that still operates, now owned by a British utility. My next purchase was Manville Corporation, the manufacturer of asbestos-containing building products. That ended badly. I bought some stocks during the Dot-Com Bubble. They ended like most of the market. Ditto the Great Recession.

I mostly learned what’s often stated: It’s hard to beat indexes. I purchased stocks with full confidence they would rise quickly. But it was based on a lot of emotion and little information.

I never quit, probably for the reasons many buy stocks. It’s a little like gambling but with a far better chance of success. I’ve continued to entertain myself, randomly buying stocks based on various valuation theories, and then holding them as long as I can stand it. Since the Great Recession, I’ve usually had about a third of our equities in individual stocks, about as much as I could tolerate. I’ve had some disasters (e.g., Peabody Coal), some close calls and some wins, most notably Ford Motor that I bought during the Great Recession fire sale.

How have I done with these stocks? It’s hard to say because it’s hard to know what I would have owned if I hadn’t purchased them. And for various reasons, it’s hard to make an apples to apples comparison with comparable index funds I’ve had. For example, my 401(k)s have normally been funded by bimonthly purchases from my salary, commonly known as dollar-cost averaging. My stocks are purchased randomly. Comparing collective returns is difficult.

But as part of my look through the carnage of 2022, I noticed that my individual stocks dropped about half of what the S&P 500 did. Again, comparisons are hard because I own some international stocks and funds.

So I looked a little further and found that over the past many years, my individual stocks have performed nearly twice as well as my index funds have. Further, it held true for most of these calendar years. A look further back has more mixed results but in total, since the peak before the Great Recession to today, my individual stocks have performed significantly better than my index funds.

I considered whether I got lucky. So I looked at returns over several periods excluding the best and worst stocks. The middle ground held about the same as the extremes.

I’ve searched returns for a single year across the entire S&P 500 and contrary to what I’ve read, I don’t find it true that the vast majority of returns come from a very small handful of stocks. These winners make a lot of press, but that’s not the only place money is made. And it’s also where a lot of money is lost.

I did a backtest of what would have happened the past ten years if I had had all of our money in only the same stocks I owned at the time. That is, what would have had happened if I had not owned any stock funds or bonds. Again, it’s a mixed picture – higher volatility but the returns are significantly higher than what I experienced, and in only one year would our total portfolio value have dropped lower than what we actually experienced with our mix of bonds, index funds and stocks.

Another common mistake investors make is focusing on market extremes, the rises or falls from recent highs and lows, rather than long-term returns. For example, the S&P 500 dropped 56% high to low in the Great Recession. But if you limit your analysis to 12-month or other longer term views, the extremes are much less. So when I look through returns, I normally only look by calendar year, or other long periods. This removes so much of the noise that causes such emotion. These short-term variations don’t have nearly the actual impact to your portfolio that your emotions respond to.

Which brings me back to the same place. There is a high price for higher stability, and if one can ignore the noise, there is significantly more money to be made owning individual stocks than there is in either stock index funds or in bonds. As an aside, this is one of the reasons (along with paying their own salaries) that professionals struggle to do well – their clients respond quickly to this noise forcing them to respond accordingly, too, which often includes selling underpriced stocks.

This is where I’ve been stuck for years. I don’t trust indexes and I don’t trust professionals, but I’m never confident that I or anyone can pick stocks well. But there’s little evidence that my stock picking is any worse than index funds. And in fact, I’m reasonably confident that beating the indexes isn’t that hard, which has always been my theory.

To use stocks as long-term investments, I generally rely on three well-known factors: what the stock valuation is using some common metrics; the financial health of the company; and finally, some cautious consideration for what analysts might say about a company. I mostly ignore sentiment, inertia or any technical analysis, other common factors considered in selecting stocks.

Please note that I don’t trust analysts either. They have a Lake Wobegon tendency to rate most stocks above average. But they can highlight some issues. I trust their poor reviews more than their good reviews.

As for valuation, I try not to put much into a stock’s most recent numbers, and instead look closely at its last several years. In addition to profits, I also look at free cash flows and dividends. I also consider their book and enterprise values.

For a company’s financial health, I look at its return on equity, its margins and its liquidity, and very closely at its debt. It’s hard for a debt-free company to go bankrupt.

There’s no secrets to what I’m doing. This information is widely available online. And although it’s time-consuming, it’s not nearly as complicated as it sounds. My usual procedure for buying a stock is to first use various ways to come up with a large number of stocks that I may be interested in, whether something I hear about or see on a list.

However I get there, if I like what I’m seeing, I chase down several pieces of information on the company, and then score it across a half-dozen measures. The scoring is to help contain my emotions, which like most humans are terrible at predicting where a stock price may go. If it all adds up to a good score, then I’m serious about buying it, which I still may or may not do.

Most of my life, whether investing or anything else that requires time and work, I’ve leaned heavily towards the law of parsimony, a principle that recommends searching for explanations constructed with the smallest set of elements. Although my stock analysis can seem a little overwhelming, I suspect one could probably come to the same decisions with half of the information I use. Maybe debt, margins, dividends, and price to cash and earnings, with a look at any warnings from analysts.

I painfully hold to stocks I own whether they’re doing well or poorly. I consider selling when the same analysis doesn’t add up to buying a stock. But even then, I still give it some time. And time is one of many tricks I use to keeping my emotions out of these decisions. My default is to do nothing.

Yes, I miss out on Google and Apple and Netflix. But I also missed out on Tesla last year. Cummins, Emcor, Ryder, Amgen and Teck aren’t nearly as interesting, but I trust them a lot more. Index funds trust overpriced stocks much more than I do.

How far will I stray from index funds? I don’t know. I don’t make fast moves. After combing through scores of stocks over several weeks, I recently made a couple of new stock purchases. As I normally do, they’re a broad mix across sectors, capitalizations, and value and growth.

And that may be all I do for a while as I wait, watching what happens with the recession “everyone” is predicting. If I do beat the market, doing little or nothing will get me closer to where I want to be. If I’m wrong, I’ll find myself making some changes. And confessing my mistakes to you.

My Goodbye to Asset Allocation

For as long as I can remember, I’ve read endless variations on two basics of investing. One is that you should maintain an asset allocation between stocks (equities) and bonds (fixed income), moving to a larger percentage of bonds as you near retirement. The second is that you shouldn’t own individual stocks, and instead you should buy them in indexed funds.

There are valid arguments for both claims. But there are quieter places that challenge some of these assumptions. I’ve challenged both for years and now I’m slowly abandoning both.

Today, I will discuss my move away from the standard asset allocation formulas so widely recommended. Later in another post I will walk through my concurrent move away from any stock funds, including indexed funds.

Last year’s markets blew up the asset allocation model between stocks and bonds. In 2022, for a variety of reasons, stocks (S&P 500 including dividends) lost over 18%. But U.S. bonds didn’t do much better, losing over 13%. The theory for owning bonds is that they will neutralize some of the stock losses in a market downturn. That theory failed miserably this past year.

I acknowledge that bonds did perform five percentage points better than stocks. And I acknowledge that this is the worst bond performance in forever. But this is small consolation to the fact that over decades, bonds generally return about one percent more than inflation, and stocks return at least six percent more than inflation. In inflation-adjusted value, stocks have returned about six times more than bonds. Six times. That’s hardly reason to invest up to half of your life savings in bonds.

For years, Warren Buffett has repeatedly said that stocks are a better and an even safer investment for long-term investors. Yes, in the short-term, stocks can be painful. But in the long term, after inflation return on stocks is much, much better than bonds. Six times better, in fact.

But surprisingly, even in the short-term, bonds can be poor performers. Although 2022 is an outlier, bonds can lose money. And the short-term bonds that are almost certainly not going to drop significantly in value normally return less than inflation.

For my first 15 years investing in a 401(k) plan, I kept almost 100% of my wife’s and my retirement money in stock funds. They did fine. Turning forty, I bought into the asset allocation formula to a degree, and moved us to about 20% bonds and 80% stocks. As we aged, I did the "responsible" thing – something I have always struggled with – and increased our allocation of bonds. I entered the Great Recession of 2008 with nearly 30% of our money in bonds.

When stocks dropped 56%, high to low, in the Great Recession, I admitted defeat and slowly moved us to nearly 50% bonds. Now we were safe. Or so I thought.

Then 2022 came. After having up to half of our life’s savings in bonds during one of the greatest bull markets ever, I saw our bonds trashed. I expected it with stocks that had doubled and tripled and more. But bonds?

Yes, over the past ten years, our bonds have not even kept up with inflation. Safe? Secure? Hardly. And they were mostly investment grade.

I did a backtest of our investments the past ten years, testing my results if I had held no fixed income and instead had all of our money in the mix of equities I did own. Although returns are more volatile, the returns are notably higher than what I experienced. And it was interesting that the test never had us with less invested than we had with the asset allocation models I was using.

This reminded me again of other ideas I’ve heard periodically, probably from Jeremy Siegel or Jonathan Clements. The counter idea to asset allocation is to keep 5-7 years of money you may need for an emergency or a market correction in cash and short-term bonds. In the event of a strong market correction, you then have five years to wait it out, using this emergency fund that is virtually guaranteed to hold its value in any market.

(It is noted that as in life, nothing is guaranteed in the financial world. But for our discussion, let’s assume we trust the dollar to hold a reasonable value and for the U.S. government to honor all of its debts.)

For the rest of the money, if you can handle the turbulence, the best returns will be in stocks. However, if you can’t sleep at night, then adjust stocks downward to the point you can sleep. That’s different than keeping a set percentage of your life’s savings in bonds.

It's important to know yourself and how you handle volatility. If you know you're the sort of person who may panic sell when the market drops 25% in a short period, bonds might be safer for you. But if you can keep a long view and not make impulsive orders, over five years you are virtually guaranteed to be in a far better position with your money in stocks rather than bonds.

For me, I can’t sleep thinking what a mistake asset allocation has been. As we speak, financial analysts and journalists are busy both justifying and readjusting the asset allocation models they’ve pushed for decades. As is normally the case, they’re telling us to now go where we wished we had been a year ago.

As is true for most forecasting, there’s a heavy bias from the most recent history, a human tendency referred to kindly as “recency bias.” It mostly reminds us of how bad humans are at forecasting anything.

But I’ve already moved on. Been there, done that. I’m not interested in a new asset allocation model that may have worked in the past but probably won’t work in the future any better than past models.

I’m moving ahead with a 5-year emergency fund that will always have all the money we may need for five years sitting in a stable place that earns little – but doesn’t drop in value. That will be mostly I-bonds and short-term Treasuries. I can ladder Treasuries in over 5 years at over 4% interest. That should probably keep up with inflation. But they won’t drop. Both investments guarantee the return of our principal plus about 4% interest.

For the rest, I’m mostly walking away from other fixed income.

How far will I go? I don’t know. I don’t make fast moves. I started by ending the reinvestment of dividends and gains from bond funds. I’m looking for opportunities to sell longer-term bonds and replace them with either short-term bonds or stocks.

Then it’s a waiting game, watching what happens, monitoring everything we have. I hate selling anything, almost to a fault. And if markets go by their historical averages – which they rarely do – our allocation to bonds will continue to drop.

If I’m right that bonds are a lousy place to be, I’ll get to where I want to be without a lot more work. If I’m wrong, I’ll find myself writing a mea culpa – and trying something else.

But since I mostly don’t sell low, my opportunities to return to a higher bond allocation would only come when stocks are doing well. And then it’s hard to justify any such move since my theory is then working. That is, I may just be stuck in my own investment Catch-22.

My next step is to move away from indexing as my primary means of investing in stocks, and instead move to owning more individual stocks than I have in the past. But that’s another discussion.

Christmas 2022

I’m sitting alone in an empty house during the blizzard of the year noting that there have been a few changes the past couple of years. No worries - we’re still married and we’re still in the same house in Duluth. But Ann’s in St. Paul helping with a seven-week old grandchild. Let me catch you up.

Not that long ago, Jason was living in Austin, Texas, and David and Ben were both in San Francisco. Since then, Jason and his wife, Nicky, moved back to St. Paul, bought a house in Dayton's Bluff, and now have a son, Jyri. It’s the Finnish spelling of the slightly more common Russian name, Yuri.

Due to COVID, David moved to Redmond, Washington, and then earlier this year moved back to St. Paul where he owns a house not far from Jason. Ann and I helped with the move, driving a car and U-Haul over 1600 miles in 52 hours. It was our second ride across the Rockies that winter, and both were beautiful!

Ben got married to Xin in 2021, had a daughter, Astrid (“Azzie”) last summer and has now purchased his first home in Issaquah, Washington, where he’s lived for a couple of years. I’m sure a dog will be coming soon to keep up with his brothers.

Jason continues to work for the St. Paul firm that hired him out of college. Ben left his original employer, Thumbtack, and now works at Airbnb. David, who also worked at Thumbtack, now works for Instacart. The past several years have been great for technology, but some clouds have arrived on the horizon. I’m sure they’ll all do fine, though.

For us, it’s been a very busy and wonderful family time. We get to St. Paul often and the kids get to Duluth. But Ann has also taken on a new job. Both new sets of parents asked her if she’d be willing to help with the new babies.

Ann was thrilled and spent a month this fall in Issaquah caring for Astrid, and is now in St. Paul caring for Jyri. It’s worked well, and the kids also appreciate the cleaning and cooking that she does!

Since she intends to continue helping with the grandkids for at least several more months, Ann finally stepped down as leader of Celebrate Recovery at our church. Ann started the organization 14 years ago next month and has been critical in building it up. Although CR struggled during COVID, like the rest of the world we’re getting back to normal (whatever that is). She is still heavily involved as a leader but she’s no longer its director.

Not to be outdone, fall of 2021 I was laid off from my long-term employer, Saturn Systems. It was part of the sale and consolidation of the company with another software engineering firm from the Twin Cities. I then set myself up as a self-employed consultant, something I’ve loosely talked about doing for years.

 

Although the work hasn’t been steady, which is mostly OK with me, I did have a good contract last spring. I’m still ambivalent on letting myself be permanently unemployed but I am enjoying testing what life might be like not working.

We’ve had a lot of fun with the kids. We’ve spent a lot of time at both houses in St. Paul. Ann and I like to walk, and their homes are in great walking locations along the Mississippi River.

When Ann was in Issaquah this fall, Ben rented an Airbnb in the North Cascades and invited me to join them. The five of us spent a week together hiking and hanging out with Astrid. It was a great time, beautiful scenery and great accommodation, a whole house and yard in a rural neighborhood.

Last fall while Ann was in Washington, I did a road trip to see all three of my sisters who live at least parts of the year in northern Wisconsin and the Upper Peninsula of Michigan. It was a great ride to my favorite country, riding quiet two-lane roads just as the fall leaves are appearing.

As we’ve done a couple of times before, Louise and I spent two days in my truck off-roading. I always liked doing this as a kid but only recently found that Louise likes it, too. We explored roads, rivers and rustic campgrounds. It’s one of those unexpected grand times I have vacationing!

I spent a similar day with Ben during our time in Redmond helping David move. Ben and I took a day-long road trip adventure north of Seattle along the coast, stopping at deserted parks to walk, grabbing food as needed. We even took a ferry. It was just a wonderful time to spend with my son.

Merry and blessed Christmas to you.

Jon & Ann

Hiring, Fast and Slow

It’s no secret that when picking stocks, most people, including professionals, can barely keep up with a monkey. What’s less known, though, is that human beings in general are poor forecasters, and it impacts us in many ways beyond stocks. Further, there is little evidence that people can be trained to be better forecasters, and in fact, we struggle to even acknowledge this serious shortcoming.

One such place where good forecasting could be very helpful is in hiring. Hiring is largely a forecasting exercise, is highly subjective and has poor results. But I have learned some ways to improve this process in spite of our forecasting limitations, primarily by removing our instincts from the process. Let me explain.

I’ve worked in information technology all of my professional life, and have spent a significant portion of this time hiring technical people, primarily software engineers. Early in my career, I learned how much differently a person may perform at a job compared with the impression I had from their interview. I soon realized that job applicants are presenting their best side and will work hard to conceal their shortcomings.

I also learned that almost anyone can look good on a resume or a reference. One of the first hiring tips I received was to read through a candidate’s resume but to then set it aside, giving little value to its content. It’s not that people lie but rather that facts are fungible. The same is true with references. They are as good as candidates at projecting an overly optimistic picture.

I also noticed that several different people interviewing the same candidate can have widely different opinions, and all sides can be widely wrong. And their opinions can change for no apparent reason.

Not surprising, for many years our hiring success wasn’t much better than the monkeys picking stocks.

I did learn several other things, though. I learned that the best indicator of future performance is past performance, and the best approach is to try to figure out what a candidate has done in the past. Not what they have wanted to do, not some spongy analysis of how amazing they have been, but what specific actions have they actually done.

I learned to avoid discussions on dreams and goals. I learned that even if people could evaluate themselves well, you’re probably not going to get a reliable answer. I rarely asked “what-if” questions. It’s another self-evaluation that is of little help in evaluating their past performance.

I learned to collect as many facts as possible based primarily on my questions, not their rehearsed summaries of their activities. I focused on specific examples, drilling into the smallest details. It is harder for a candidate to eulogize on their behavior once they are asked to describe a specific interaction, such as a certain topic discussed with a peer, or the smallest details of a program they wrote.

I learned that high GPAs, awards and fancy colleges may not mean much. They may, but it depends. Finally, I learned that one of the best sources of reliable information, even a candidate with professional experience, is a careful review of their college transcripts.

But the best help I ever got was from the book “Thinking, Fast and Slow” by Daniel Kahneman, one of the more fascinating books I’ve ever read. It describes the two modes of thought people work with. One is “fast,” instinctive and emotional, and the other is “slow,” more deliberative and more logical.

His lifetime of research strongly suggests that people have too much confidence in human judgment, specifically with forecasting, probably because forecasts tend to use your fast brain. The problem with your fast brain is although it’s great dealing with a current situation such as an immediate danger, it is not very good at looking ahead years.

In an especially interesting section, he goes through his research on both stock picking and hiring. Both tasks require an ability to forecast the future. He highlights the overwhelming evidence that professional stock pickers have awful track records doing what they’re purportedly paid to do. And the same is often true for hiring, even when done by professionals. The results can be just as bad.

This is seemingly true for any forecasting. Most forecasts are just slightly better than guesses. Economists are a great example of horrible financial forecasts. And even when presented with this information, we will continue to feel and act as if each of our specific predictions was valid. Our emotional minds are so confident in our views that we are mostly unable to hear evidence to the contrary.

Mr. Kahneman provides a psychological argument for why this is true. Probably for efficiency, the human mind has a strong “halo effect” which “inclines us to match our view of all qualities of a person to our judgment of one particularly significant one.”

For example, if we think a baseball pitcher is handsome and athletic, we are likely to rate him better at throwing the ball, too. Our halo effect tends to have us hiring people based on factors largely unrelated to success, such as personality and physical appearances, and then applying them to other important factors with less regard for the facts.

I vividly remember the glowing interview an employee had with a candidate. The evaluation was heavily focused on the candidate’s charming personality, humor and the fact that right during the interview the candidate fixed a problem the individual had with their PC. But most software engineers and many amateur techies probably could have fixed the same problem. It had questionable relevance to the position but had a notable impact on the interviewer.

Kahneman’s evidence suggests that a hiring formula can be developed for a specific position, industry or company, and this formula will outperform the professionals. His theory is that if you remove much of the broad judgment interviewers make, and instead focus on a handful of clear indicators that the slow brain sees as predictive of success, you can significantly improve your hiring. The trick is to push aside the instinctive and emotional fast brain normally used in hiring.

I was intrigued by his argument. Using this information and some of my own experiences, I radically changed the way we hired, with great results.

Instead of trying to intuit what a candidate would do at a job, I started basing our hiring decisions primarily on a simple formula using specific quantifiable data, data that seemed predictive of good employees. Here is how I did it.

From a large pool of current and former employees, I and my managers ranked each employee into approximately 4 quartiles, top to bottom. We based their rankings on several measures that we were able to quantify.

Then based on this ranking I took the top 25% of our current and past employees and compared them with the bottom 25%, looking for notable differences. I only looked at the differences in what we knew about them before they were hired and did not consider items we learned later but could not have reasonably known when evaluating the candidates.

My idea was to see if there was any information we could use in future hiring that might help us select people that would perform more like our top 25% and less like our bottom 25%.

The results were astounding! Please remember that these results are for software engineers working in our organization. Your results may be much different for your needs.

We were able to find significant differences in success based on their level of education, the specific school they attended, their degree and the GPA they received. This was true of both entry level and experienced candidates. Amazingly, we were even able to find a strong correlation with their letter grades in a couple of classes normally taken as part of standard software engineering training. It was also notable that we found no discernible difference between a bachelor's and a master’s degree.

We found several significant indicators for job retention, including their past positions and where the jobs were. We found correlations with their previous experience, too, specifically with what basic responsibilities they had in each position and how long they stayed at that position.

Although I considered many factors, I was able to reduce this information to 4-5 key indicators that summarized a candidate. These indicators fit on the back of an envelope.

This is one of the other secrets of formulaic hiring and forecasting: Don’t use excessive analysis and detail. There are probably less than a half dozen discernible indicators available that will get you the same or better results. Further, too much detail and too much latitude opens the process up to finding a way to let your intuition abscond with your hiring, rather than using a few clear facts.

I didn’t try to understand why a certain indicator predicted certain behaviors but rather just accepted the facts. For example, we had one highly rated college with strong graduates but on the job, they had a negative correction with successful hires. It’s counterintuitive but we accepted it. Anything that allows your fast brain back into the process is not good.

The process succeeded very well. We still had misses and surely we passed on some good people. But confidence in hiring improved significantly.

I was now able to quickly filter out candidates. Once a candidate passed our key screening, we then did several interviews, letting various people take a try at deciphering the candidate, primarily along our key indicators.

We did our best to ignore dress, communication, culture fit, commitment, strategic value and motivation, all words often bandied about as critical to an organization’s success. But our analysis suggested otherwise.

There is one serious problem with this approach, though. People, especially the professionals and managers, will normally discard any thought of formulaic hiring, responding with hostility. Any notion that you can mostly ignore your intuition when making hiring decisions, and instead rely primarily on a handful of facts is seen as nonsense.

Apparently, it upsets people to think that their assumed ability to read people is largely an illusion. After several tries over a long period of time, and in spite of an obvious improvement in our hiring success, I gave up trying to explain our methodology.

Based on Kahneman’s book, we also eschewed onsite testing. It’s paraded around as a must-do but there’s little evidence that a quick test tells you much about one’s ability to work well in a position.

To repeat, he instead highly recommends that you look at what the individual has already done in the field and don’t try to assess those abilities, whether by testing, intuition or any other means that has shown little ability to forecast the future behavior of an individual performing in your organization. You do not want to look into their eyes and find that impression you’ve already decided on, normally based on other often irrelevant factors.

My suggestion is similar to Mr. Kahleman’s. However you do it, identify some key indicators regarding the candidate’s past performance that should be good indicators of success and can be normally elicited as part of your screening process. These can be based on your past hiring experience, as I did, or your own ideas of what seems to be reasonable factors in accessing an applicant for a position. Finally, don’t overcomplicate your process.

Have each interviewer rate the candidate 1-5 on each indicator, add up the results and hire the candidate with the highest score. You’ll make mistakes but a lot fewer of them.

A New Look at Clean – and Life

A couple of years ago I heard a young guy, James Hamblin, on National Public Radio talking about his new book, Clean: The New Science of Skin. He was challenging our modern norms on cleanliness. In his uncontentious style, he explained how important it was to wash our hands a lot. But he also questioned how necessary it was to suds up our forearms every day in a hot shower. Or do we need a daily shower at all?

I was intrigued by his comments and did a quick internet search on him. He's an M.D. who changed careers into journalism. He looks so young that he is still compared to Doogie Howser, the teenage physician from the sitcom by the same name.

I have heard similar things before. A friend of mine told me several years ago that his dermatologist told him he didn't need to wash his feet, that any water and soap from a shower is sufficient cleaning. I haven't deliberately washed my feet since, and they look about the same today as they always have.

I heard that antibacterial soap is not good for your body and regardless, doesn't clean any better than Ivory. So I quit using antibacterial soap whenever possible.

Years ago when one of my sons was in college, he told me that he hadn't washed his hair in over a month. He claimed it was as clean as it had always been. I tried a lite version of what he was selling. I started washing my hair every other day, using as little shampoo as possible.

I quickly noticed that I no longer had to add gel to my hair. My natural oils were apparently providing the same effect as some fancy French product. My hair feels clean but now it isn't nearly as dried out as it was.

Later, I asked the woman who cut my hair for years if she noticed anything different. She kept looking at my hair, ran her hands through it and, mystified, said it looked like it always did. I explained my new hair treatment and she told me that she mostly does the same. She said that shampoo is hard on your hair and shouldn’t be used daily.

I remembered back to my Saturday night baths as a kid. I loved baths, but like most kids, they were once a week. Between baths, I did what most kids in rural Northern Michigan did. I played outside when possible, including a lot of time in the grass and dirt. I don't remember feeling dirty.

By high school, I was into the daily shower, including a heavy lathering of my hair, a routine I maintained for decades, except for an occasional soap brand switch.

There was one notable exception. For three summers over an extended college experience, I lived canoeing and camping in the Northwoods. It was a life I loved, and in my dreams I would spend the rest of my life in a cabin.

My friend's wife found this interesting. She shared with me her longing for the American dream, a house with a white picket fence. I had no such dream. A one-room cabin with a water pump, wood stove and outhouse was all I wanted.

While living in the woods those summers, the daily shower wasn't happening. But I did spend a lot of time in the water. Other than regularly washing my hands, I cleaned up twice a week, mostly in a lake, using a small bar of soap and a bit of dish soap. No gel. No deodorant. No hairdryer.

Modern cleanliness is an attractant for flies, and the longer I was in the woods and the less I washed, the fewer flies I had to fight with. Eventually, I felt like I was part of the nature that surrounded me. I rarely felt dirty. Quite the contrary, it was the best my skin has ever felt, tanned and weathered, and a bit leathery.

My friend's wife got her white picket fence. Like her, I and most of my friends did the same. I have to admit that it is a comfortable life. I like my pick-up. I like being warm and comfortable. But in some alternative universe, a side of me could still spend at least a portion of my life living much closer to nature.

So when I hear someone challenge the value of a daily shower, I'm curious. After listening to Mr. Hamblin's simple and clear message that modern society has probably overdone what it means to be clean, possibly to our detriment, I started on a new journey, cautiously following the path of less may be more when it comes to clean. My wife notices nothing different. And the only difference for me is my morning bathroom drill is faster than ever.

I finally read Mr. Hamblin's book. It is quite intriguing. It is not a scorched earth assault on modern life and he never suggests that the world follow his lead. What he does, though, is present a short history of cleanliness and the factors that may have pushed our society too far from its roots. He is suggesting a more moderate approach to health and cleanliness, one that may appear more radical than it actually is.

Mr. Hamblin has done extensive research exploring how we got here, examining both the science and the culture of clean. He talks to various medical doctors, scientists, marketeers, capitalists and many others exploring the meaning of clean and its effect on our bodies. He finds that contrary to public perception, the concept of cleanliness is tied up more with class and beauty than with heath and germs.

It is said that we all come from nature and we all eventually return to nature, so staying close to nature may be a good plan for your time alive. This has been my experience.

The currently accepted theory for many diseases is that they come from germs. Modern life leans heavily on removing these pathogens from our environments, which leads neatly into keeping ourselves and our environments clean.

However, contrary to what many want to believe, our bodies do not exist alone. Rather, they are part of a microbiome, the microbes such as bacteria, fungi, viruses, and their genes that naturally live on our bodies, the people we connect with and all of nature that surrounds us.

Humans cannot live apart from their microbiome. Each of us has literally trillions of microorganisms that live inside us and on our skin. We and these microorganisms have evolved together.

In our efforts to rid ourselves of “germs,” Mr. Hamblin presents strong evidence that we have introduced new medical problems. These problems could come from daily showers, over-processed food, sanitizers, separation from nature and the outdoors, and even antibiotics.

Mr. Hamblin’s suggestion is that we move closer to our microbiome that we are inexorably connected with, that we slow down this senseless fight and that we instead embrace the complexities of the nature that we are intricately part of, that is itself an extension of ourselves. His claim is that our obsession with being clean is harming the microbiome that keeps us healthy. That is, cleanliness is probably bad for our health.

So how did we get here? Here is what I have learned.

Soap is a product that has been available for a very long time, and works about the same today as it always has. The basic process is trivial and most soaps are nearly identical chemically. The rest is sales and marketing. Surprise, surprise that capitalism and our obsessions with ourselves can get us so far off track.

Soap became a household product once it could be produced economically, and then it went from being a luxury to a requirement for living in modern society. Now to even speak of not showering is, as it has been put to Mr. Hamblin, "not really dinner conversation."

Cleanliness is about more than just the broad availability of soap. In past eras, many more people died of infections and injuries than from chronic diseases. Because of advancement in healthcare, including our ability to fight infections, our life expectancy has risen and chronic disease has now become our leading cause of death.

Eczema, acne, psoriasis, multiple sclerosis, diabetes, Crohn’s disease, asthma, allergies and many other autoimmune diseases are all on the rise, some as much as three times what they were just a few generations ago. For example, hay fever was once almost exclusive to isolated, well-off individuals, while farmers who were regularly exposed to higher levels of pollen almost never got it. Today, it is widespread. And this rise in autoimmune diseases is in spite of advances in skincare and modern medicine.

For example, Mr. Hamblin provides evidence that adolescent skin problems are probably worst today than ever possibly because of our efforts to keep unnecessarily clean. He suggests that it is at least worth a try at dropping the common skin remedies, and instead to start washing our faces far less often.

Some of this rise in chronic conditions is because so many people now live longer than in generations past. But other factors include our move from a rural society to an urban society, and our obsession with cleanliness, at least partially because our brains still disproportionately fear infections.

Excessive cleanliness and urban life both further remove us from our microbiome, which may be a big part of our problem with rising chronic illnesses. He notes how children who grow up on farms tend to have lower rates of asthma, hay fever and allergies compared to city kids. Microbiologists have found that Amish people, who spend most of their lives working on farms, have very low rates of autoimmune conditions and associated inflammation.

There is strong evidence in reverse, that low rates of asthma, allergies, eczema and skin problems, are strongly correlated with low rates of skin cleaning.

Science is coming to understand that there is a complicated relationship between our bodies and microbes. This includes their important role in developing our immune systems. Most skin microbes are harmless. They create antimicrobial substances that protect us from pathogens by competing with them for space and resources, lessening the likelihood of autoimmune conditions.

If we scrub off these microbes and the natural oils on which they feed, we lose some of our exposure to the microbes that surround us, microbes that are mostly needed by our bodies, and this may contribute to the increase in autoimmune diseases.

This “biodiversity hypothesis” doesn’t propose that hygiene is bad but that the loss of different kinds of microbes is bad—that modern inflammatory and autoimmune diseases are linked to us being deprived of exposure to the microbes we evolved to be exposed to, including pathogens, as well as beneficial and neutral microbes. And we aren’t just deprived of them by washing and using antibacterial products, but in all the ways we are today isolated and sterile, living in a world that may be too clean.

When people now spend most of their lives indoors, scrubbing all the microorganism they can, they are lacking the wealth of bacterial particles that used to temper our immune systems.

The best advice right now is to think of hygiene as similar to medicine—extremely important in some scenarios, and also very possible to overdo. The same goes for exposure to microbes. Historically, exposure has been a much bigger danger than over-cleaning. Now, in much of the world, it is the reverse.

Which brings us back to our morning shower, sudsing ourselves with our favorite antibacterial deodorant soap, making sure our forearms are clean enough for modern society. I’ve known people who carry the daily shower over to other areas, daily washing every towel and every piece of clothing worn by the family, including the kids’ pajamas.

I spent some time researching this topic, and found little written on it. Most of the mainstream websites – Mayo Clinic, WebMD, CDC, Cleveland Clinic – sell the same message as soap manufacturers. They even note that a daily shower may be needed for your mental health.

Wikipedia has an entry on the “hygiene hypothesis.” It notes that studies have shown that various immunological and autoimmune diseases are much less common in the developing world than the industrialized world, and that immigrants to the industrialized world from the developing world increasingly develop immunological disorders.

I found suggestions that kids shouldn’t be protected from dirt, that eating some dirt is probably good for your system and its ability to fight off infections.

About the only consensus, though, is that you should wash your hands a lot, brush your teeth every day and avoid antibacterial soap. The rest is up for discussion.

Mr. Hamblin suggests a measured, more minimalist approach. As for showers, the consensus is to use them “conservatively." Mr. Hamblin once went five years without a shower.

I quit washing my feet years ago, I quit showering every day and my hair gets a small fraction of the shampoo it once did. I brush, floss and shave daily. After that, it varies. My wife notices nothing different, and I’m sure no one else notices anything different, either. My hair isn’t nearly so dry. I use a lot less water.

I’m being far more cautious than Mr. Hamblin, but I intend to keep pushing the parameters, taking a measured approach. I’ve always maintained that fighting Mother Nature is a bad plan. For all our Western independence, we’re far more locked into the living world we inhabit that any of us want to believe.