Aktify

Teach the models, and scale the teaching

The charter: a product team, a vision, and a way to train models at scale.

I joined as roughly employee number sixteen, recruited by Jason Ferriante, a friend and chief data scientist I had worked with at Lendio. My job was to build the product team and decide where the company was going.

One thing stood out immediately: the process for training the conversational models did not scale. Accuracy sat at 71%, and every improvement was hand-built. The people who suffered for it were the clients’ customers, who talked to the agents, often when no human was awake.

• The problem a non-scalable process for curating and training models.
• The customers clients who deployed agents for sales and for service.
• The users people talking to those agents, any hour of the day.

Humans in the loop, linguists on the team

Two moves at once: build the tooling, and staff the teaching.

First, I chartered an internal system to automate and manage curating, monitoring, building, and deploying models, and hired Seth Christensen as Director of User Experience to research and design it.

Second, I assembled a team of affordable experts. I recruited “NLU analysts” from the linguistics departments at BYU and the University of Utah: graduates in a niche field who were not sure how they would use their degrees. The model flagged conversations it handled poorly, aggregated them, and sent them to the linguists to say how the conversation should have gone.

Accuracy up, team built, profitable

A human-in-the-loop system, an organization, and a business that paid for itself.

Model accuracy rose from 71% to 88%, and training became efficient and repeatable. The product organization grew from 2 to 20 people in 12 months, spanning AI product managers, NLU analysts, analytics, and UX, with Ashley Wilson (PhD) and Cassandra Darley as PMs and analytics engineer Austin Adair, whom I promoted and trained into product.

I also created a vision and strategy for a consumer-centric conversational AI product, which attracted $150 million in VC funding that we chose to decline, and ran the P&L of a $20 million business at a 24% EBITDA margin.

• Conversational AI training HITL model training; accuracy 71% to 88%.
• Product organization AI PM, NLU, analytics, and UX teams; 2 to 20 in 12 months.
• Vision and strategy consumer-centric AI; attracted and declined $150M in VC funding.
• P&L $20M business at 24% EBITDA ($4.8M annualized).

The most rewarding outcome

What the work meant, and why I was ready to move.

My favorite result came from a nationwide chain of rehabilitation clinics with an agent on its website. Analysts saw people reach out in the middle of the night, some in real crisis. Trained for that context, the agent could stay with them through the night and hand them to a human who could help.

With the training infrastructure, the human-in-the-loop process, and the strategy in place, and the business profitable, I had delivered what I came for. Trent Miskin and Brock Blake had been trying to bring me back to Lendio, and the timing finally worked.