Automating Revenue So Graph Go Up

Railway just crossed a significant revenue milestone. We can’t disclose how much we now make, but it’s a pretty big deal for us and our customers.

We officially started “doing sales” about a year ago, but after reading “pulp business,” none of the old rules applied to us. The expectations of developers have changed — all the advice about sending emails doesn’t work, most off-the-shelf revenue enrichment software is broken, and learning golf didn’t help.

We’re an engineering company, so we looked to first principles to build ourselves a unique revenue function.

To get there, we hit our head against the revenue wall very hard, repeatedly, until we built a sustainable business — the one we have today.

Let’s get into how it works — including a detailed look at the regression model we wrote to figure out if a user is going to upgrade or churn.

Let’s begin with a cold hard fact. Developers are highly impatient and want to use the product with as little friction as possible.

No contact form.

This is contrary to the aims of the average salesperson, who would like to know about the customer’s intent before they use the product.

This is the natural tension that emerges when salespeople are rewarded for selling and developers are rewarded for completing tickets. Those aims are often opposed. So, some companies swung towards something new and shiny called PLG.

The true essence of PLG or “product-led growth” (the part that we agree with) is to create a product that is so valuable, its purchase becomes a natural choice — where payment is the default and all barriers to adoption are minimized.

Yet some companies have misinterpreted PLG to mean that you should hand out free samples of your product to gain market share and then use your market dominance to influence whether you use the product or not.

However, as we have seen with much of the dearth of the free tier nowadays, giving away the product with indefinite trials in hopes that your customer uses it isn’t a good plan. Many customers just never end up paying you what you are worth, especially when compute isn’t cheap. As such, many companies are now reverting to sales-led motions.

So how do you get paid what your product is worth while also getting out of the way for developers?

Railway charged on Day 1.

It was a comically good deal for users and a bad deal for us. $20 a month got you near-unlimited compute. Since we wanted to avoid the fate of some beloved products but business failures, we’ve always believed in charging for things to prove business viability.

As more companies wanted to adopt Railway for their business, they came to us with questions about SLAs and NDAs. We got caught flat-footed. We knew we had to “do sales” at some point, but we weren’t sure how. The breaking point was when a major Fortune 500 CTO asked for a demo on Discord right after a message from DotNetHater420.

Should we Ramp ties and button-down shirts for the engineers that knew how to hold a conversation?

I read every sales book I could find to learn how to build a revenue function. Much of the content was focused on:

• Educating the reader on what is Sales (useful)
• Motion, which is how you sell the product and how customers get access to it (valuable)
• Technique, the etiquette and the questions you would ask a customer (useless)

However, how to sell to your audience was an exercise left for the reader.

We gave the advice an honest try by gating some of the product behind an Enterprise offering and sending boatloads of emails to an ideal customer.

We got a 0.3% response rate for customers already on the platform. Which is … not good. For reference, Ian, who is the best recruiter in the world (we’re hiring) has a response rate of 0.5% on his cold outbound emails. No one wanted to jump on a call.

Which makes sense. As a developer, I am in my editor, not my inbox … all these sales books and none of them offered good engineering solutions tailored for serving engineers!

For the calls with large prospects, we were insistent that we didn’t want to start slicing off or forking the codebase to appease one customer at the cost of our user base. Doing so would have built a tent where you have two nearly opposite points of tension. Not us.

We restored (nearly) all features to all plans and went back to the drawing board.

You know when customers actually did want to jump on a call with us?

When they had a major issue and were emailing us in all caps.

And after those calls, we noticed something interesting. Every customer who ended up taking a meeting with the team grew their usage … like, a lot.

We then realized that the existence of sales for us was to close the information asymmetry gap that our user faces. Instead of trying to move used cars, we should be more like a TA in college, helping the developer come to the correct answer for the problem that they had, even if it wasn’t Railway.

It looked like a Success function.

As Railway’s product team finally delivered expected features like Regions, more of our conversations revolved around existential themes like “Will you exist in 5 years?” or “Can I trust your company?”

Companies came to Railway for quick deployments and whether or not they stayed depended on if they could trust us with their business.

From this insight we then created our first Enterprise-lite offering (“Business Class”) which allowed us to designate increased compute limits and support SLOs for those who needed more attention (like paging us at 3 AM) to address their concerns.

However, selling egg protection insurance is no use if the egg is already cracked. This issue was especially prevalent when a customer was growing fast and we had no idea they were on the platform. We would then perform an upsell after an angry email… good for our revenue, but bad for our conscience.

Despite the occasional dropped egg, this approach of jumping on a call after a steamed email was helpful. We learned a lot from those experiences. But the number of dropped customers increased as more developers jumped on the platform. Worse still, we had no idea who they were until their stuff broke.

That’s when we started thinking about scaling.

Nine months into our Success practice, we started noticing more patterns around the type of customer that needed a helping hand.

But first, we checked if there was a revenue intelligence vendor that would help. Not for lack of trying, most products just outright didn’t work or were aggressively demo-gated. Trust us, we didn’t want to have to build this out ourselves.

So we started aggregating parameters that would generate for a time series. We decided on privacy-respecting data points like number of successful builds, failed builds, regions configured, number of support requests, and feature adoption.

We then aimed to boil these parameters into a single score to determine customer temperature and ranking the customers.

All of the nights laddering on Starcraft 2 were now going to be put to use making our own ladder. To get there we did a bunch of regression modeling to determine which parameters were significant to make something similar to an ELO score.

Using the customer’s spend as the control, we tested to see which parameters were significant in a customer’s spend by first getting p-values via multiple linear regression. The p-values helped us check if the data points were a significant influence on platform spend. Closer to 0 meant it had nearly no effect while anything too large meant that we should be very suspicious of over-fitting to those values. And we couldn’t just eyeball these numbers — we needed to test those values before settling on a final model.

We proceeded to do multiple tests using a stepwise regression and a LASSO test. It showed us some obvious-in-hindsight significant correlations like that more disk usage would lead to a stickier product experience. (The DB business is a good one I suppose.)

We then worked to remove colinear factors in scoring. An example here is Memory or CPU usage, which is highly correlated to spend. Thats not entirely useful for us since that doesn’t generate any insight, however failed deployments had an extreme determination on if a customer was successful.

We then looked deeper into their journey. If analytics showed us a spate of failed deployments, that usually signaled frustration brewing on the platform. If a customer failed to overcome their metamorphosis, they’d either send us a strongly worded email after they deleted everything, or grow massively.

After getting acceptable weights for our parameters we generated a score that showed the temperature of a customer. We then grouped them to different tiers of adoption with affordances for both promotion and relegation.(Jury is still out if we ship Leagues to Railway.)

Anytime a customer was at risk of relegation? We would reach out.

We then used Hex to build a dashboard to track our customers’ growth over time. With the help of our friends at Chartmogul, we also automated opportunity creation within our CRM. (Shoutout Nick and team for delivering the API.) With that, we have a internal dashboard we can use to track company movement within the platform.

Now when we get a ping that a customer is growing, we start to pre-warm up resources that could be helpful to a cohort’s growth. As an example, a cohort at the time of writing is very concerned with O11y, so Melissa spun up a nifty OTEL resource for those customers. And it’s already been useful for the team in predicting who I will likely be talking to over the next week.

The journey in my career from reluctant DevOps person to SWE, to PM, to Support Eng, and now to Sales/Successperson taught me that the desires of the line engineer is the same as the line account executive — they both obsess over observability for the sake of their customers.

As engineers, we want to build out Railway’s Sales function where you don’t need to talk to us at all to alleviate your fears. But, we also want you to know that you always have our team’s expertise at your disposal if you do want a call.

When you do reach out, you should feel like the team is able to anticipate your concerns. As such, we have a few learnings that we plan to incorporate into our offerings. Such as:

• We have a feature discoverability problem on the platform. Much of our churned users often complain we don’t have a feature that already exists. We have Evan from the Product team now thinking deeply about this problem. We promise not to hire multiple PMs to play out their political battles on our product.
• We’re working to revamp our offerings to reflect our internal league model. Your level of support on the platform should just be a matter of your expected tenure and committed spend that you’ve agreed to.
• We’ve started generating insights cross correlating data on off-product events such as outages and support response times to determine if they affected a customer’s propensity to stay on the platform. (An immediate insight we got was that periods of repeated outages would increase churn to a great extent.)

When we set out to answer the question: “Do we get paid what our product is worth?” we needed to know what our customers see value in. The answer for us is speed and trust.

Instead of having to wait 30 days to see how we did last month, we can now track a good amount of realtime gross movement on the platform.

The best part is we now have a sort-of internal stock market that we can use to help us allocate our time and resources to invest in our users who are growing quickly. Instead of spamming the funnel with credits, we can target interventions such as offering a Slack channel to fast-growing customers.

Much, much more is planned on this front.

Meanwhile, we also now have Pro workspaces reporting deeper usage — in part because we opened the lines of communication with Pro users. Over the last year, I booked 120+ demos (keep in mind I still ship code) and 4x’ed the number of companies on the platform.

This is all done so that people can trust us with their business, so that we can exist in perpetuity to serve them, which is the highest honor.

What else do you want to hear about? Let us know on Twitter.