Overcoming Obstacles to Early Product Validation

I’ve made the case that many enterprise organizations undervalue or ignore the importance of early product validation. On one hand, this seems silly: especially on the web, it’s extremely easy to validate utility with actual users before investing millions into a product.

However, it’s also quite understandable. Well-intending product teams face significant challenges when they propose early product validation in a large, established organization.

Forthcoming from O’Reilly, Lean Enterprise tackles a similar concern, but many of the book’s recommendations rely on action from key business decision makers who have the authority to drive change within a company. For the product manager, designer, or developer working within an enterprise box that cannot be easily redrawn, I’ve observed a few key barriers (and some solutions) to implementing a culture of experimentation.

Brand

Whether business-to-business or business-to-consumer, an established organization needs to protect its brand equity. Releasing a product that feels incomplete for the sake of validation may raise eyebrows among marketing groups. Like an artist protecting his or her incomplete work, brand champions may fear that customers will pass judgement on the brand as a whole if they’re exposed to an experiment or minimum viable product, or suffer the fallout from a failed idea.

Potential Solutions:

  • Leave the product unbranded and focus on early adopters. It is completely possible to validate both the demand for and usage potential of a service all while dissociating it from your brand. Early adopters are also more forgiving when it comes to imperfect products.
  • Be honest with your customers. Frustration often derives from misunderstanding, and a brand advocate can become dissident if a product appears to be changing for the worse. Find ways to give context to users on the benefits of your incremental releases and validation experiments.
  • Take the time to do a few things really well. If you’re validating a product idea with a set of core features, polish those up nicely. Spending extra time on unproven features may seem counterintuitive in a Lean context, but you’ll avoid the false negatives that can come from technical, usability or aesthetic issues.

Technical Limitations

In an organization poorly prepared for iterative testing or continuous delivery, teams are often faced with the spectre of technical infrastructure blocks—especially when their products integrate complex legacy systems.

Potential Solutions:

  • Rely on smoke & mirrors and bypass the system. In place of robust technical underpinnings, a human (or monkey, if you can train it) can collect user inputs, manually process the data, and serve up interface feedback to the user. In The Lean Startup, Eric Ries calls this type of simulation a “concierge MVP.” It might resemble a real product, but is nothing but an interface.
  • Develop a testing platform. If experimentation is likely to be ongoing but never a part of the core product, invest some time and energy into a set of tools that support rapid prototyping and deployment on a parallel system.
  • If the box doesn’t fit, get inside of it and push on the edges. Champion continuous delivery practices within your organization at a grassroots level. If at all possible, make the case to upgrade or update the problematic infrastructure.

Budget Cycles

Annual budget cycles and centralized approvals make it difficult to change product direction—and in fact, can deter change altogether—in many enterprises. How can a Lean(ish) project team operate inside the constraints of a budget-driven organization?

Potential Solutions:

  • Get ahead of the curve. Allot funds in this year for product validation that will define the next budget request. This could mean focusing November’s effort on validation of product ideas in anticipation of a new budget cycle beginning in January.
  • Use guerrilla tactics and circumvent the budget by bolting the work on to another project. Early validation is appealing because of its efficiency. Capitalize on this efficiency further by utilizing homemade or jerry-rigged testing mechanisms and methods to avoid large overhead costs.
  • Allay fears of endless spending. Define clear periods for exploration and experimentation. Even if you can’t run a perfect, conclusive experiment, derive at least some insight before full-blown production begins. This constraint will also encourage you to conduct input research and modeling to aim your experiments appropriately before the timer starts ticking, which will net further insights.

Intolerance for Learning

In early product validation, teams prioritize efforts that support experimentation and learning above all else. While a project stakeholder may have a pet feature, if that feature is not core to the experiment, it is not a priority. Moreover, in this type of exploratory work, a month of activity may produce only the answer, “we shouldn’t build that.” This can be hard for the C-suite to stomach, especially if they measure success by lines of code. Remember grade school science fair? A well-executed experiment that disproved a theory was still worth an A+. But for some reason it still seemed less valuable to have a failed hypothesis.

Potential Solutions:

An intolerance for learning among stakeholders can be very challenging, since it calls into question the very premise of experimentation and often requires a paradigm shift to rectify. Fortunately, we can learn a lot from other organizational change models, including Agile transformation and Kotter change management.

  • Demonstrate to decision makers the value of learning about a product’s utility. Provide examples (the Airbnb story is a great one) of other organizations, the blunders they’ve avoided, and the customer experiences they support. Examine your own past product failures. Was utility part of the problem? Illustrate how early product validation might have helped. If at all possible, develop a champion within the executive leadership to drive urgency and compel other decision makers to support early product validation.
  • Again, if the box doesn’t fit, get inside of it and push on the edges. Rally your fellow advocates to promote the benefits of the approach (develop what Kotter calls a “guiding coalition”). Employ marketing tactics where possible: hang posters, distribute literature, get the organization thinking about the topic.
  • Remember that it doesn’t need to be perfect. Strike a compromise between learnings and working product. Commit to delivering tangible outcomes (like a platform or basic UI pattern library) that can work even if the product direction shifts significantly based on your findings.

Understanding your product’s utility is an important aspect of product development. Yes, certain products succeed by chance—especially when unanticipated social practices reveal unexpected utility (Bell called, they want their business communication device back). But relying on chance in a highly competitive environment is risky. And while you can certainly apply UCD testing tools in a controlled environment, the capacity to validate in the wild will lead to greater confidence.

Unfortunately, the enterprise world is rarely as amenable to early validation as the volumes of “Lean” literature might suggest. But that’s okay. By staying scrappy, team members working within more traditional organizations can employ tactics that help mitigate risk and reduce waste. If even the smallest degree of learning is happening, better products will result. Of course, this list of blockers and enablers is purely anecdotal and only the beginning.

As a proponent of useful products, what challenges have you encountered? What workarounds have you employed?

Sign up for our newsletter