What We Learned From Launching Our Beta To 400+ People

Beta Testing | What We Learned From Launching Our Beta To 400+ People

Back in October when we were gearing up to launch our beta, we had a newsletter of around 250 people as well as another list of 200 or so people who wanted to know when it would be ready.

My goal for beta was to get around 20 people using the software out of that list of 250. I thought 20 would be a realistic number that would be enough to get some good data, but also not so much as to completely overwhelm our three-man team.

I made an offer to the list to see who was serious about testing it out and to my surprise 50 people requested to have an account created for them.

Although we had only wanted 20, we decided to allow these 50 people in and from there we continued to add more and more.

The beta lasted around 2 1/2 months and we created over 400 accounts.

These are some of our learnings.

Creation Does Not Equal Activation

My main worry when we created 50 accounts was that we would be overwhelmed, however soon I realized that 50 accounts being created does not mean that 50 people will actually activate the software.

In fact the number of people who actually activated from that list was around 20.

There are many reasons why a person may not activate an account.


Other times people are just not comfortable installing a desktop application.

IMPORTANT: On May 1st, NinjaOutreach launched it’s Web App so it is no longer just a desktop application

And even if everything else seems to be okay, sometimes people just forget.

Over the course of this entire beta, while we created around 450 accounts, only just over 100 actually activated the software – around 25%.

That said, now that we are commercial and our free trial is up, we are seeing that the activation rate is a lot higher. This is probably because they organically decided that they want to test the software and they also committed payment details.

Some Customers Don’t Want To Be Developed

Customer development involves meeting with customers to get an understanding of the usage case – essentially how they like the software, how they’re using it, and how they think it could be improved.

What startup enthusiasts don’t tell you is that your customer does not always want to speak with you.

In many cases people want to use a software, but have no interest in having a meeting with you.

We could have made it mandatory as part of the beta that people had to meet with us, but we decided this would only limit the number of users.

Instead we tried various methods to get customers on the phone, but above all we just tried to make ourselves very available and included our call scheduler in the signature of every email.

I would guess that we met with around one third of the beta users.

This was certainly valuable in understanding the different segments of our target market, as well as what features our uses wanted to prioritize.

I certainly don’t regret the time spent on meetings and only wish we could have had more.

Customer Support = Easier Said Than Done

One thing that always baffles me is when companies have awful customer support.

To me it seems like nothing will irk a customer more than having poor support, like long waiting times on phones.

Despite wanting to make customer support a priority, we also struggled while in beta. Mostly due to the fact that we only have one developer and most customer support instances were technical by nature.

Although I was perfectly willing to talk with any customers who are having problems, there were only a handful of instances where I was able to solve the problem 100% myself.

Because this is a software application a lot of the technical issues are very specific to the user’s computer. Let’s say an inquiry comes in the process probably looks like this.

  • First I decided I can handle it myself if not reported to the developer.
  • Next to developer decide if he can answer it just email or tell me what to do.
  • If not the developer has to schedule a meeting with the customer in all likelihood they live in different countries.
  • While meeting with the customer the developer has to look at the error report and see firsthand what the problem is.
  • Then the developer had to solve it

Needless to say, sometimes customer support just fell through the cracks.

Now that we’re accepting payment, we’re certainly going to do a better job, but during beta when nobody was paying it seemed that the best thing to do was to improve the product for the benefit of everybody instead of spending hours and hours on a single customer support issue.

As much as it pains me to say some people just did not get chance uses software.

Twitter FTW!

One of our biggest lead generation wins was Twitter, and it was almost entirely automated.

All that was needed was

  1. A list of Twitter handles for our target market
  2. A bulk social media management platform for sending out tweets
  3. A method for responding to the tweets

Step One: A list Of Twitter Handles

We used Moz’s tool FollowerWonk to capture thousands of Twitter handles for your target keywords in seconds.

Here’s what it looks like when I type in digital marketing agency.


On the paid plan, these results can be exported.

Step Two/Three: Bulk Twitter Management

The next to two tasks were accomplished in one tool – HootSuite.

This works for Twitter management and allows you to bulk upload tweets and manage any replies and mentions you receive.

As a result we were able to upload hundreds of tweets every week and respond to the conversations, which led to over 100 interested agencies in our target market who wanted to participate in our beta.

Believe it or not we got a better response from Twitter than email.


We finally exited beta in mid-January after nearly a month spent working on payment processing.

We’re almost done with the backend work and will be moving ahead with product development.


David Schneider

Dave is an author at Ninja Outreach and has a passion for digital marketing and travel. You can find him at @ninjaoutreach and [email protected]