Since Evidence for Impact launched, a couple of organisations have got in touch with roughly the same question: we're thinking about building an AI tool for our org – how did you go about building yours?
It's a familiar question and it's almost exactly what the Centre for Homelessness Impact (CHI) asked me last year. They thought AI could help make their evidence more readable and accessible. Was there a product here? Could we run an experiment – a sprint – and try to build something for them?
This is the story of what happened next, and what I'd tell anyone asking the same question today.
Eight weeks: is it possible, and can we trust it?
The answer to CHI's question was the eight-week AI product sprint. Not a strategy document, not a long discovery phase – a short, focused piece of work with one job: find out whether there's a real product here by building and experiencing it, not guessing.
By the end, we had answered two questions. Technically, it was entirely possible: we could ingest CHI's evidence and have AI read and learn from it. And, really importantly, we found a way to guardrail the AI and prevent it from making things up. It came back with correct answers, grounded in real evidence.
What CHI walked away with was an internal alpha: a tool that let their team search the evidence, ask questions and get answers, and add whole batches of new evidence for the AI to read.
The unlock was obvious and it was enough for CHI to see the potential of putting it in front of many more people. So they commissioned the next phase: turning the alpha into a proper platform that could launch to users.
Eight months: from alpha to launch
Going from "it works for us" to "it works for everyone" is where most of the work lives. Four strands ran side by side.
Hardening the engineering
An alpha proves a point; a platform has to stand up in public. That meant hardening the code and building in all the things people rarely notice (and are quite boring) until they're missing – authentication, security, caching, privacy by design and many more.
Deepening the product
We built on what the alpha had proved. Search became more effective, quicker, broader and more filtered at the same time. Chat became a proper back-and-forth conversation, with a citation engine and sessions you can come back to. And we added collaboration features, so people can save the evidence they find and share it with colleagues.
Testing with real users
We started testing with our first cohort of users in April, and their feedback shaped much of what followed. It was encouraging to see such positive feedback, but also valuable to learn what changes we need to make. Some of these were visible – a button here, a small feature there. Many were almost invisible: the language and tone of voice, how responses are generated, the small behavioural tweaks that make a tool feel right rather than merely work.
One of the biggest changes came straight from users. They wanted to look beyond the evidence base to the wider web, without losing the rigour they’d come to rely on. So we designed web mode (above) with all the rigour built in from the start. It favours authoritative sources and checks that every page it cites actually exists. What it can’t do is vouch for what a page says, which is exactly why the interface turns red when you step outside the evidence base.
Design as a trust layer
It's tempting to think of design as visual polish added at the end. But here, the polish is part of the product.
A lot of the platform's trust signals live in its design language and affordances. One primary blue runs through the brand, citations, search and chat, so grounded evidence always looks the same. Tags show at a glance how robust a piece of evidence is. And when search steps out to the web, the interface turns red – a clear signal you’ve left the verified evidence behind. The tone of voice is calm and authoritative, but never overreaches: it sets out what the evidence says and leaves the judgement to the reader. And when the platform doesn’t know something, it says so rather than guessing.
None of that is decoration, and much of it isn’t even visual – it’s behaviour. It’s how people know what they can rely on.
How we worked
How we built it mattered as much as what we built. A few principles we always lean on at Creative Crow:
A small, senior, autonomous team. The core team was four people, including a client lead on the CHI side. Everyone was senior, driven and given autonomy to make decisions – which is what lets you move really fast.
Ship something every week. Every week we shipped something – a new feature, an update, a bug fix, and most weeks several. Every Friday we reviewed what we'd done with the client, and sometimes with users, and planned the week ahead. It kept us accountable: weekly cycles rather than months of nothing and then something.
Keep it conversational. We worked together in Slack rather than over email, so small questions got answered in minutes, not days. We favour quick huddles only when needed, over a ton of prescheduled meetings.
Keep the experts in the loop. We brought in subject-matter experts at CHI, such as their head of evidence, at key points – to check not just the outputs, but how the AI makes its decisions.
Search ranking is a good example. Every result is ranked on two things: how closely it matches what you searched for, and how strong the evidence behind it is. Getting that balance right isn't really an engineering decision – it's an evidence decision. So we tuned it together with the people who know the evidence best, and kept checking it as we went.
So, is there a product here?
Back to those couple of emails. If you're wondering whether there's an AI product hiding in your organisation's knowledge, here's what I'd say:
Start small and scrappy, and find out fast. Eight weeks is enough to know whether it's possible, desirable and whether you can trust it.
Treat trust as a design and tech problem, not a marketing narrative.
Put it in front of real users and experts early, and keep listening.
Keep the team small, and ship every week.
Thinking about something similar?
I’m always happy to have an informal chat about whether there’s a product in your idea. Drop me a line, or have a look at how the AI product sprint works.
