Why "product" evaluations don't work.

“Product” evaluations don’t work, for the same reason Wall Street “stock pickers” don’t work. More recently, the film Moneyball aptly showed why baseball "talent scouts" never really worked, either. Why? Nobody has a crystal ball. They might have a few lucky picks and a few good hits for a while, but they inevitably fail. Sometimes miserably. Predicting the performance of an individual stock (or baseball player) is simply impossible. There are too many variables to compute. In the case of the stock market, it moves on sentiment, not just rational factors, as well as "externalities" that affect price (up or down) regardless of the inherent value of the stock. That’s why “beating the market" is a myth. (See this great article on how the best of the best money managers still can’t "outperform").

The same applies to consumer products. The performance of an individual product in the marketplace is a mix of rational and irrational factors, plus "externalities". It is therefore unpredictable. Certain experts will try to make you believe that they alone have solved the multivariate mystery of product success. But that’s simply impossible. The only person qualified to judge a consumer product is the consumer! We know what we like when we see it. Not before it is launched.

Before product launch, every opinion on an “idea” is subjective guesswork. Pre-launch product evaluations or idea assessments get distracted by irrelevant factors, such as vague market trends or allegedly insightful numbers that pose as science, or the votes of a few statistically unrepresentative participants in an online community. True product success in the marketplace is less about the raw conceptual "idea” than it is about good timing, great marketing, flawless manufacturing, distribution excellence, and often, no small amount of luck. So, a "silly" idea can succeed and a "great idea" can still fail. Ideas alone cannot be valuated, because they have no inherent value. Success is purely determined by execution and timing.

Here's another way of looking at it: product evaluations are like high-school popularity contests. They have no predictive power of ultimate success in life. Subjective value judgments of raw “ideas” are simply too early in the invention process. After all, when looking at a patent, drawing or even a finely tuned prototype, everybody has a different vision of how the final product might actually appear on the market, or the customer it might appeal to. Subjectivity means that people project their personal feelings and hopes (or skepticism) onto an idea. Thus, product evaluations and contests must rely on one person’s imperfect opinion. Sometimes, they attempt to blend many people’s subjective options, hoping this somehow reduces the chance that an entire "herd" of experts would miss the obvious. Yet they do. All the time. (I should say, "we" since I am often a product judge in invention contests).

The only person who does a good job of predicting the success of new products is Jordan Pine. He publishes the widely respected SciMark Report about DRTV campaigns. Mind you, these are fully developed items, and fully launched infomercial campaigns by some of country’s most experienced TV producers and manufacturers. So Jordan is doing the predicting after the product launches. Yet even his track record is admittedly spotty. Jordan never fails to have a good laugh every time a “silly” product turns out to be a hit, and a “great idea” fails. Real experts, like Jordan, know their own scientific limitations in front of picky US shoppers: the largest consumer market in the world with notoriously shifting tastes, needs and expectations.

Product evaluations don’t work for the inventor, either. In the early stages of the inventor's journey, the idea is fragile, often subjected to incredulity, or even ridicule. Inventors don’t need some outsider to pooh-pooh their idea, nor do we need somebody to stroke our ego in the other direction. We just need the truth. We need to know our strengths and weaknesses.

To make matters even worse, most “product” evaluations have a hidden vested interest to exaggerate one way or the other – wildly positive or overly negative -- because they sell invention development services. Such a glaring conflict of interest is what prompted “Operation Mousetrap” by the FTC in the late 90’s. Inventor scams would pump (with a positive evaluation) and dump (after selling cookie-cutter, expensive services with astronomically low chances of success). Industry-wide regulation may once again be needed, and soon ... because some companies now offer "free" evaluations. Remember, in a mousetrap, the cheese is always free.

Our company sells no “development” or “licensing” services whatsoever. We are unique in that regard, whereby professional evaluations is all we do. We charge a fair price for an objective, scientific, rigorous, professional service.

How do we do it? Think about a new invention. Can you predict success just based on the idea itself? Of course not. There are too many factors at play. Companies know this. In fact, working with them, we have determined that there are 5 “pillars” that support every good invention submission. These 5 pillars increase an inventor’s likelihood of landing a good deal with a company or investor. Like building-blocks, the 5 pillars add up to a single numeric score. But we do not attempt to predict market success. We simply give the inventor the tools to have a better chance of landing a good deal based on the factors that companies and investors truly care about. It is much like knowing your credit score before applying for a loan. If you know your score is good, it improves your chances of getting approved and getting good terms. If you know your score is low, you can work to shore up your score, thus avoiding an undue rejection, or worse ... getting stuck with bad terms and bad rates.

When companies and investors consider a new invention, the “product” may be important (whether it “fits” or not in their product lines and brand strategy) but they don’t judge solely on the product. They go beyond subjective opinions or gut feelings. In deciding whether to invest or not, they do proper diligence on the prototype, on legal protection, on the overall business potential. Licensing negotiations and investment decisions are serious commitments of money. They consider many factors. Not just the product.

So beware of “product” evaluations. They’re not reliable predictors of success. Instead, we suggest you work with us to evaluate your “process” and how much invention equity you have to negotiate with. This will ensure you have maximum chances of landing a good deal with investors or a manufacturing company.