Now that you have launched your product, you do expect some traction. So you go pally with media/bloggers/PR guys – and within a few months, you have a decent traffic to your site.
And you launch more features to keep your users engaged..and more. But, after a while you realize that users aren’t consuming what you want them to.
Suddenly, your bounce rate has gone up – so what do you do?
Usability Test
Well, this is one of a typical issue faced with any new product launch and if you are a startup, you have a bigger constraint – i.e. how much of ca$h can you spend on usability tests?
How do you conduct a usability test?
- Hire a usability consultant?
- A:B/Split testing?
Yes to all of the above, if you are very very well funded and have enough ca$h to spend!
Challenges with the above approach
Typically, usability consultants recruit people (based on certain demographic profile that matches your product) and conduct a closed focus group study in the lab – the only challenge is that they tend to train users (not knowingly, but it just happens) to use the product.
Nothing wrong with the approach, just that you will end up talking to users who will NOT at all complain about your product, but at the same time, will not even use it in the future.
In short, they are undergoing the usability test not because of your product, but because they have been recruited by the consultant.
And mind you, you are paying a hefty amount for all this!
A typical usability lab study
Go for it, if you have enough ca$h and can afford a sample size big enough to build a pattern.
Same applies to A/B testing - if you have enough time (and can spare some effort in building different landing pages), go for it.
If not, read on!
Usability options for a Cash-Strapped Startup
Do it yourself.
Yes. What I mean by DIY is that you should be conducting your own usability tests.
Where do I get the users?
Start off with your friends – take a random sample of 5 power users (assuming you have friends in IT sector who are poking and Orkutting each other every day).
Then, go beyond the power users.
Where to?
Cybercafes.
Indian Cyber Cafe - Up in the Clouds
Randomly, pick up a casual user, tell him that you are going to pay for their 30 minutes of Internet access (costs Rs. 10-20), show them your product and ask them following:
Before you start:
- Tell them about the product – what it does etc etc.
- Keep certain goals in mind - for e.g. user should write a review, should click on ‘share’ button etc etc.
Start the test
Warm Up
Let them play with the product, get familiar with the site for the first 5 minutes.
Ask them:
- What is the product all about?
- What’s their understanding of the product?
- What is it that they believe, they can do with the product?
Let them play with the product for another 10 minutes (get them signed-in if needed)
Get Intensive
- Build a story - for e.g., if you are a local search player, tell them that ‘you have to find the best pub in MG Road, Bangalore – as you are supposed to meet your blind date there‘
- Based on the story, ask them to perform right actions.
- Watch their steps - where are they clicking? If you want to test out a logged-in feature, were they able to understand that they need to signin before using the feature?
- At any given time, do not go beyond the 2 stories - user is likely to get bored and scream at you.
Lessons
- Once the user is done, ask them what’s their overall feel of the product/feedbacks etc.
- Will they come back to the site?
- Something they just couldn’t understand? Something that they simply hated (for e.g. sign-in process?)?
Go back and keep a tab on these users – how many of them use the product again? Based on your success ratio, optimize on the questions and scenarios that you want to use.
These are I’d say, a startup’s way to keep their feet on the ground and get a first-hand experience of user behavior.
What’s your opinion?
Hi
Nice article.I am going to try each option/approach.
Thanks,
Rajeev
http://www.brandsindiaonline.com
Hi Ashish
Have to say this is one of the most useful articles I have read lately. I especially loved the advice on hitting the cyber cafes to do some usability testing. I would try something like this out and let you know how this goes
Keep up the good work
Prateek
awesome stuff!! definitely worth spreading thru!! WoM – will fwd it to as many startups as I know.
Hey Ashish,
Very useful and subtle approach!
I would add one thing here that many times the team itself is not using the product on a regular basis and they expect the end users to do so. The reality is that you will be able to solve many usability issues once you start using it yourself on a regular basis.
My 2 cents
Pingback: How Should Startups conduct Usability Testing? « Himanshu Khurana
The first approach seems the best.If you are convincing enough and the idea exciting enough,you would find many to do so.
Personally this week I am beta testing bumptop
btw ashish,how about maintaining a list of people willing to test or a list of startups that need this kind of jobs to be done?
Ashish!
Let me congratulate you primarily for posting a thought-provoking topic. As someone who makes a living in this domain, I would like to give my 2 cents
, hope it helps.
Usability testing post launch is not the right thing:
I am sure a lot of people will disagree with me (more so in this part of the world), but then, UT is not something that is just a line-item in a Product Managers “to do” list and doing it at the far end of a product / service’s launch will not be of much help. On the contrary, UT’s fused into dev cycles on an ongoing basis have shown tremendous results as opposed to the ones conducted post-launch or in beta stages.
UT consultants are expensive is more of a myth than anything else. While it is true that some “specialized” state-of-the-art-and-what-not entities do claim lots and fill a lofty number in the price column, the plain fact is UT’s are and can be as flexi to be attuned to wants and needs of the start-up in terms of time, cost, objective etc. One major problem is with the Usability/User eXperience fraternity – Not much work and outreach is happening to inform and educate people about the field, its methodologies et al so that prospective clientele can take “informed” decisions. There is no one-size-fits-all solution. I would like to quote my own experiences where I have helped start-up’s across the globe to reach their “optimal experience” conforming to their offerings, demographics, markets et al. One fundamental (carved in stone) statement I insist on before signing the SoW or contract is ” …….Continual involvement of the UX team from the beginning of the product/service’s development cycle…….” this is because, a lot of customers treat Usability as a “can do it later, nice to have” commodity and eventually the Usability consultant / firm gets stuck with an insane timeline, low response times and lots of pressure cooker situations
Not EVERYTHING that the users say is necessarily true:
This was known to Henry Ford when he said “If I had asked people what they wanted, they would have said faster horses”, aspects that depict success like Contexts, criteria, interaction et al keep changing continually and it requires a trained eye to observe, capture, act-upon and fix problem areas.
I am more than happy to help start-up’s in their UX strategies and would be glad to answer questions.
I have to agree with the above comment. Cost effective user input is done before you have a site to test. Doing the testing yourself will save you a bit of money, but nowhere near as much as finding faults early in development.
I disagree – most of the times, you would like your users to take the usability tests on a working site (I mean, as a startup – how much of all this can you do pre-launch? Its all about doing and learning!)
Richard, I completely agree with your “it is all about doing and learning”,however start-up’s just don’t become entities overnight. In as much as learning is concerned, the ability to fail fast and move on much faster is far more important than most of the other constraints that a start-up will face.
You are more than welcome to disagree, however, i would like to point out that you do not need to have a “WORKING” as in designed, developed and hosted site/application/product or service to actually conduct a test. There are multiple, cost-effective and more importantly usable testing methodologies that one can embrace even before moving on to much mature stages of UT cycles.
Cheers
UdHaY
http://www.linkedin.com/in/uxfirst
Richard, I’m not saying that you shouldn’t test the working site. I’m just saying that you should also test before you have a working site. This isn’t some far out theory of mine. It’s accepted best practice in the field of usability.
However to further make the point, I have tested a couple start-up websites. Neither had carried out testing before building their lovely whizz bang websites. Neither exist any more.
Pingback: Weekly Recap of Interesting articles |Technology and Business Startups in India
This is something all the companies has to do. A usablity tester must some1 who is entirely new to the system. Get some real user to the company and ask them to play around..
This is what exactly I was having it in my mind.. [:)]
V
Again a great post!!
Only point i would like to make is when you are picking the users, make sure that he is our targeted users. but in anycase feedback of a dumb user is always important.
Hi Ashish
Nice post. Liked it coz I am a big proponent of UT since its part of my job as a Product Manager
However I feel Usability is a highly relative field. It changes from person to person. When you are making a product it really becomes important that you know the persona who is going to use the product. e.g. while working for a financial giant, all my users were high on IT quotient. For them the content and information were important. but while working on SCM domain, i figured out that navigation and guided behavior of an application was really important to keep a user to use to application
Usability also changes from the kinda of solution you are building. It has a different approach when you solution is a Enterprise application and its different if you are a web portal
Also by following certain set standards we can keep layouts very simple and make it more usable
Also I personally feel doing UT of a product when it is built, is rather a bad approach. For me UT starts when as a Product Manager, I design a concept in a wire frame or ppt, which then gets refined by a Ux expert (in case you have one). UT has to be planned in every iteration which you are delivering
I dont know why, but its a rather simple field which is been made to look complicated if we do it at the last moment
Regards
Vishy
These pages sure do seem inexpensive. Thanks telling us.