CREATING BRINGING THE RESEARCH
TO THE BUILDERS
How designing automated research infrastructure for an AI startup illustrates the process of systemic design applied to human systems.
I was working with an early-stage AI startup, building systems that would help them stay oriented to market signal without the cognitive burden of constant research. This challenge has two layers: first, building sensors that surface useful signal; second, making that signal flow to operators in a digestible form at the right time.
THE PROBLEM SPACE // 01
THE CONTEXT
Most startup and business operators know their success depends on the ability to learn and adapt. In other words: how well feedback loops internalize information and convert it into effective action. There are many frameworks, methodologies, and best practices that build on this idea in various flavors, but the pattern is universal: learn and adapt.
Getting it wrong creates ambiguity and inefficacy; getting it right provides focus and the sustained ability to create something valuable.
Rarely is the problem that the operators are unaware of this idea. The problem is in the inherent difficulty in building an instance of an effective feedback loop and fitting it to how people actually work.
That last part is an important nuance. Startups are ultimately organizations of people; and these people often get into startups because they want to make something based on a vision of what could be, and generally not to submit themselves to the tenets of commercial cybernetics.
There's always a tension between the impulse to build and the need to stay focused on the market.
FRAMING THE WORK // 02
THE CHALLENGE
The top priority was to give the team the information they need to make good, signal-based decisions with the lowest burden of context-switching, remedial research, or uncertainty. They needed to stay close to market signal without stopping to conduct episodic research on opportunity cost, user behavior, market signal, etc.
This is where systemic design is clearly a great tool for the job: seeing the existing flows, understanding what components to connect, and building just enough to align a team's existing abilities.
The design thesis I ended up using: As a systemic designer, what can I do to remove as much burden as possible from operators so they integrate more effective entrepreneurial research in their daily decision making?
Notice that the definition is not something like: "Make dashboards so anyone can see the information". Maybe dashboards will work, but only if it actually removes burden and leads to more research integration. Unless you have a ton of organizational goodwill and willpower to burn through, you can't just demand that people start behaving like perfect information processors. To expect good results you need to understand what you are working with, and that starts with discovery.
THE PROCESS // 03
THE APPROACH
Discovery
The process starts with mapping. I draw a basic loop diagram and annotate where operators make decisions. The map becomes messy pretty quickly depending on the complexity of the system, but this helps me understand:
- Who needs what information, for what decisions, and when?
- What's the right "slicing of information" appropriate for a given set of choices?
- What information is tactical versus strategic?
I ensure that I understand the people involved:
- What do they already understand and value?
- What is their role as an operator?
- Where is their attention already going?
- Where do they engage in conversation about data and signal? How do we make the right information available in those moments?
Design
After discovery, I move to design: defining goals, requirements, success criteria, and scope before I identify which patterns might be useful in the given scenario.
There are many patterns from many disciplines you can use in systemic design. Here are the ones that stood out for this project:
Attenuators: From cybernetics. Mechanisms that filter and refine information so humans can actually act on it. Simple in concept, but nuanced to create well. Information must be grounded in reality and arrive in a form that fits the receiver's local needs.
Feeds: A familiar UX/UI concept. Activity feeds have a strong relationship with attention. They're easy to scan and are unobtrusive to bring things to attention.
Formal vs. informal flow: From systems thinking. Every system has a stated and unstated form. If I design only for the stated flow, I miss the unstated: the cloistered conversations, 30 second intuition checks, long-term intuition built up over consistent exposure.
Accumulation: From gamification. Make it visible that daily data is building into something. Make it fresh daily to create anticipation, predictive mentalities, theories about what and why.
Modularity: A basic engineering principle. Keep the infrastructure such that we can add, remove, and change components quickly.
Automation: A basic process principle. Gathering data daily is labor intensive. Automating it also creates consistency.
Implementation
Once you know what patterns you need, existing infrastructure often serves the role clearly. Most of the automation was designed greenfield.
The final design that emerged was an automated reporting system that creates and publishes tiny, digestible reports (app usage, web analytics, market taps, etc), and distributes them daily to channels on a chat platform where operators already spent their time. The chat channels provided context for the reports, places for discussion, and allowed operators to engage with minimal context switching.
Additionally, this could be built within a few weeks and operated on a monthly budget well under $200.
Facilitation
Usually there is a need for facilitation at this stage to "light the spark" of a new system. However, there wasn't a huge need for that on this project. As soon as the channels started producing daily stats, curiosity drew operators in.
IN PRACTICE // 04
SHOWCASE: COMMERCIAL
INTENT MARKET TAP
One of the original attenuation reports was a market tap designed to let us react to commercial interest as quickly as possible. Here's how it worked:
- Filter custom domain emails from signups
- An AI agent does a simple search on the company
- That same agent classifies the company
- Store the info, create a cohort in PostHog, update a running count of commercial user categories
- Push a daily report of the running total of the top incoming commercial categories along with summaries and website links
Within a few weeks we had a legitimate and evolving new insight:
New Surprising Rankings: We saw a clear leader in commercial signups emerge. This was an ICP that had been considered, but not prioritized. Even more surprising was the second runner up, an ICP that had not been on the radar at all.
A real data asset to build off: We now have a list of interested, pre-validated firms to study, build relationships with, and conduct further research on, available immediately.
OUTCOME // 05
THE RESULT
The system is now a live, evolving piece of infrastructure. The sporadic, cognitively expensive data review is replaced by pre-qualified information that is surfaced consistently, provides proactive understanding, and may be referenced without operators leaving the tools they are already on.
REFLECTIONS // 06
THE LESSONS
LESSON // 01: Startups are organizations of people. Designing systems that ignore how people actually work is designing for failure. Meet operators where they are, not where you wish they'd be.
LESSON // 02: This article isn't about how to make reports, use AI, or a list of lean principles. It's meant to illustrate the process of clearing obstacles for operators to do what they do best.
LESSON // 03: The tools that made this possible were: the ability to see and map systems, a grasp of how to adapt systemic patterns to a specific situation (especially human systems), and enough technical skill to build and connect the components.
LESSON // 04: The context is specific; the approach transfers.