Startup Structure and Information Flow

The following is excerpted and adapted from my private notes during my time at Predikto (2014-2018), exploring how information flows through startups and its impact on company structure. This is part of a series on startup organization and methodology, including posts on Science vs. Engineering in Startups and Decision Making Under Uncertainty.

Process, Structure and Information

Startup development has its own process and development cycle. There is a defined process of identifying a real problem, validating it with real customers, building a minimum viable product, getting it in front of the customers, rinse and repeat. But does this always make sense? What makes this a uniquely startup process? Why does it work, or does it always work?

In this post we will examine the impact of corporate structure on the flow of information through the company, and how that in itself can be a significant indicator of later stage positioning in this certain class of startups.

The Four Core Roles

A company, regardless of size (even one person), can be broken into four primary roles:

  1. Engineering
  2. Sales
  3. Management
  4. Product

In brief: product determines what to make, engineering makes it, sales sells it, and management facilitates the flow of information and resources between the three groups. Other common roles like marketing, research and development, services, and IT all serve to support one of these primary roles (marketing supports sales, research supports engineering or product, etc.).

The Critical Role of Management

Management’s role in facilitating communication is absolutely critical for one reason: the natural state of every other group is to overstep their bounds. Engineering will tend to overreach into product’s realm. Product will tend to overreach into engineering’s. Sales will tend to overreach into product and strategy.

Good leaders in each of those fields will, given the opportunity, spill over into their neighbors. These micro-aggressions are an anti-pattern in that they seem, at first, to be a helpful mixing of ideas and experience, but as they begin to undermine the authority of the encroached-upon group, they degrade that group’s ability to perform.

So management’s ability to keep the groups’ roles well understood and respected, and to control the flow of information and the process of accountability is absolutely critical; it is in fact, the most important thing management does. If management does nothing other than this, and if the groups themselves perform well, the company will be successful.

This begs the question: what does it mean for the other groups to perform well? Fundamentally, the job of Product is to determine what is to be built. The startup development process embodies this role (as often the founder is the first head of product), in that it engages in a collaborative conversation with those who are hopefully going to pay for the end product. The goal is constant iteration and the agility to keep up with changing markets and new opportunities. Many startups have built themselves into empires using this methodology, and continue to today.

The practical role of a product manager is to be a conduit and a filter for information. In the startup development context, information flows from the customer or potential customer through the product manager where it is consolidated into features (desired end-results), and prioritized. Engineering can then estimate the scope of the tasks and schedule work based on that scope, the feature’s priority and the available resources. Through this iterative process, the changing demands of the market can be met and the product can stay on the leading edge.

This works out into a quite simple and neat workflow that can be thought of as a conservation problem. Effort can be allotted some measure (like points), and given a certain engineering team, a certain number of points are available to be completed per week or month. Given that rate, the product manager can assign priorities such that the most important things can get completed and those less important can be scraped.

But where does this fail? The end of the pipe, as it always is in business, is capital. If a business has infinite ability to spend, it can simply keep hiring engineers and eventually will be able to meet the requirements of product and therefore the customers. Barring that, at some point there will be no more money to hire engineers, so capacity to complete points will be limited, which means capacity to prioritize points will be limited, which means requests from the customer will be limited. Intuitively, this is obvious; one can’t be everything to everyone. But the implications of this limitation are magnified in certain cases.

The Two Types of Technology Companies

In perhaps an over-generalization, there are two types of Technology Company:

  1. Intellectual Property (IP)
  2. Service

An IP company is in the business of creating defensible technologies of value. This includes software, processes and physical widgets that have yet to be created. Service companies in the technology context are companies that are aggregating, assembling or otherwise utilizing existing technologies (often via IP companies) to provide some service. In simple terms: Qualcomm has IP, HortonWorks is a service. This is not to say that the service companies don’t produce new and extremely complex technologies, but that the flow of information through the product manager goes in a certain direction (especially at first, but more on that later).

We can continue to think of the flow of information through the product manager as a pipe. The information can flow left to right, starting with some new discovery from the engineers that gets packaged, positioned and sold out into the world, or right to left, where the world says it wants something, product identifies and consolidates that need, and engineering meets the requirements of product. In early stage companies, this flow tends to correlate with whether the company is an IP company, or a service company. IP companies tend to develop ideas and features internally, while service companies, oft by necessity, solicit their customers and leads.

Many companies tend to live somewhere in the middle, or be exceptions to this rule, but one area where the rule is most reliable is in the nascent market. A nascent market is one young enough to not have standard practices, or traditional methods, but also promising enough to have name recognition. People know they want it, which is perfect for a startup, but they aren’t quite sure what “it” is, or what it might look like. In a nascent market, the flow of information through a company dictates the categorization of the technology startup:

  • If the information flows from sales to engineering, the startup will tend towards services
  • If the information flows from engineering to sales, the startup will tend towards IP

But why?

The Four Core Realities of the Nascent Market

A startup in a nascent market can be categorized by four core realities:

  1. The customers won’t know exactly what they want in a product. In fact, that which they say they want, they may not actually want once they have it.
  2. Different customers won’t agree with each other on what they want.
  3. Everything will take a relatively long and uncertain amount of time to build (due to lack of industry standards, tribal knowledge or existing solutions)
  4. Stability, trustworthiness and adoption rates of the product will win out over minor technical differences.

The first thing you should notice is that realities 1 and 2 are at odds with 3 and 4. Due to 1 and 2 a startup in a nascent market can be certain to get inconsistent requests and feedback from leads and customers. If the market is in fact nascent (i.e. promising), there ought to also be a high volume of requests and feedback, along with marketing and analyst feedback/hype. This is all fine and good and should enter into the company, sift through the pipeline, be consolidated into features and prioritized for engineering’s consumption.

But remember our limitations. The core of the startup development model depends on fast and granular iteration. The reality of the technical and nascent market is that engineering time is slower and more uncertain. Poor scoping is common, and priorities are estimated poorly. Problems are amplified by lack of industry standard to fall back on. Inundated with heterogeneous inputs, uncertain market direction and complex engineering, the IP company does what any company would, it assembles existing technologies to help keep up with requirements. And with that simple decision, likely made without question, the startup has begun its inevitable transition from an IP company to a services company.

It is extremely difficult for a startup in a technical and nascent market to remain a pure IP company while following the traditional startup development process. What’s the caveat? Remember the example where the company has an infinite supply of money to keep hiring engineering resources? That’s where the venture capitalists (VCs) come into the picture.

How to Win the Nascent Market

The only thing more expensive than educating a nascent market is building one, and VCs are itching at the chance to do it. The market is, as previously stated, won based on the stability, trustworthiness, and adoption of the product, above all else. This is for a few reasons:

  • Poorly understood markets and solutions cannot be reliably judged based on merit, and no clear winner rises out of chance.
  • In the absence of long track records or clear and well understood metrics, buying decisions are made off of perceived value metrics:
    • Brand recognition
    • 3rd party referrals
    • Previous experience with the product in another job or use-case
    • Broader adoption within the buyer’s vertical
    • Attractiveness/User Experience
    • Pure salesmanship
  • Broad adoption, after the first sales, is based off again, perceived value:
    • Being “good enough”
    • Being usable
    • Being attractive

VCs, at least the good ones, know this to be true. This fact is self evident in how and who they invest in. An oft-touted phrase in investing is to “bet on the jockey not the horse”. This phrase alone exemplifies the understanding that in a nascent and technical market, and given a bottomless pit of money, a bright entrepreneur and product leader can identify customer needs, filter those into product features, and pay for engineering to build it. Given time, sales, and some luck, this model ought to win the market, which can be monetized regardless of profit metrics at that stage (through IPO, acquisition or new business models).

Service companies are hit or miss in this space. Many struggle due to lack of existing technologies to leverage in their new space, which means that often, service companies in the technology context do best in the disruption or changing of existing markets. In previously unknown markets and those which require extremely advanced technologies, IP companies win out (another tendency reflected in the investments of the best VCs). The service based nascent market companies that do well tend to ride along open source projects, implementing, extending or offering enterprise support for them (i.e. Cloudera, HortonWorks, DataBricks, etc.).

When there are huge investments that seem to not make sense in a currently tiny market, someone is trying to execute on this model. A lot of the time it works. But not everyone has or wants to take on VC investment, and not every IP Company in a nascent market is a Silicon Valley darling. So how else can an IP company keep from drowning in a sea of uncertainty?

Another Option

Recall that there are two options for flow of information through a company. When IP companies go from right to left in nascent markets, they tend to be forced into the service space, but what happens if they originate information from the left side. Here’s where research comes in. Outside of startups, in academia, larger companies, and widget companies, there is the concept of a researcher, who will without prompt or input from a potential customer searches for new and interesting “things”.

The roles of players change slightly when information flows from left to right. Researchers and engineers ideate, discover, and produce new technologies and features. Product positions and commercializes the new technologies, and strategy finds the market to sell the product into. Sales and marketing do what they always do.

In fact, sales and marketing have a bit of a simpler job. While they don’t get to play as much of a consultative role, and their hands are a bit more tied with regard to changing the product, information can still trickle backwards. Further, the product being sold is more stable and more unified than a constantly changing one in the previous model. Where the IP Company struggles in this case is in research, and in strategy. Researchers and engineers are great at coming up with interesting things, but there is a reason startups validate ideas with customers: engineers are often wrong. Engineers like interesting things, customers like helpful things. But this isn’t a deal breaker, it is simply the same problem that you have when the information flows right to left: engineers cannot be left to decide product direction alone, just as customers cannot. It is the role of the product manager to select the most marketable and sellable technologies that come out of engineering to pursue, and it is the role of strategy to position the product and select its market.

Building a Technology-Driven Company

So how can a company be technology driven:

  1. Curate a culture of research curiosity, by encouraging some degree of development without direct customer support or solicitation. This can be done by forming parallel tracks of research and technology driven development and product driven development within the company. The time of engineering is owned by head of research and head of product, but that negotiation is moderated by the head of engineering to ensure resources aren’t over-allocated. This split may be 5% research and 95% product, but having some degree of internal driven ideation helps to damp external uncertainty. For more on this balance, see my post on Role Separation in Startups.
  2. Balance requests from sales leads with engineer/research originated ideas, internalize the difference between what could be done, and what a customer might want. Both have uncertainty, but different kinds of uncertainty, together they paint a more informed picture.
  3. Feed back requests from customers through the PM but filter them heavily against the 4 core realities of the nascent market.
  4. Focus on technology, then productize and position it, rather than picking a position and working back to the technology: product-market fit vs. market-product fit.
  5. When scale allows, develop a dedicated research/science group which works outside the SDLC for more open-ended tasks and research questions; curate a culture of discovery not implementation.
  6. Build an outward facing technical leadership, to develop the perception of expertise in the broader market. By the time a salesperson is in front of the customer it is too late to convince them of expertise, they need to have internalized it before that stage of a complex sell.
  7. Value above all else the usability and stability of products built on defensible technologies. Build and earn trust. Within the team and within the market.
  8. Respect the flow of information through the company, and keep in mind its potential pitfalls.

Making new things is hard. There are a lot of different ways to go about creation, but as a company grows in number of people involved, new complications arise. Many roles are by definition at odds with each other, and so a careful balance and due consideration to structure are critical to making sure the whole group can find success. The key is finding the right balance between innovation and execution, between research and product development, and between long-term vision and short-term needs.