Home     RSSRSS

Archives: Process

Mobile Performance Testing- Requirements Gathering Process

November 30, 2011 by kiranbadi1991 | Comments Off on Mobile Performance Testing- Requirements Gathering Process | Filed in Performance Engineering, Process, Project Management

As per the research done by Morgan Stanley, mobile internet will soon be overtaking fixed internet and we can see and feel the same trend now given that smartphones and tablets sales are overtaking PC sales,so all these changes compels us to go with a market demand.



So I thought let me share some of my thoughts based on my exposure to mobile world.I did poke around with WAP/Openwave emulator back in 2006/07 with both Silk Performer and LoadRunner.I do remember they have a protocol bundle for testing mobile traffic back then.I faintly remember it was more of an open source emulator support which these tools were supporting.Now the scenario is different, WAP is dead and now plain HTTP rules the mobile world.So we have some old rules which I believe still applies to mobile world and then there are also some rules which are totally new when we think in terms of Performance Engineering.

The strategy for designing for the mobile web site differs drastically compare to the desktop based web application given that applications works in the mobile context with limited amount of resources like screen size,network capacity,Hardware resources etc.The site needs to be lean in design and be accessible on the smallest mobile device with the exact functionality as offered on desktop based web applications.So the end to end strategy for testing mobile based application also differs compare to the desktop based web applications at times throwing unique challenges to the testing team.

It is also important to know and understand the capabilities of the various types of phones available in the market so that we have some idea as what is happening in the client side of the device.Some phones have Cursor based navigation, some phones have focus based navigations and some have touch and some multi touch based navigation.Low end devices normally have cursor or focus based navigation,so users using this focus or cursor based type of device will not normally prefer to have navigation depth of more than 3 links while browsing the site.The same user might browse a little longer in your site with either the tablet of smartphone.So it is fair to expect that page depth of the browsing in case of smartphones and tablets will be somewhat higher compare to Low end devices.This will help us in designing appropriate work load modelling for performance testing.

In this writing, I will try to summarize as what all things needs to be known and collected as part of requirement gathering for performance testing of the mobile web applications.

  • Know your site along with its business domain and benefits it brings to users who are using your site.
  • Know your users and how they use your application in mobile environment.
  • Know the type of mobile device or handset your users uses to access the application.
  • Know the network bandwidth your users uses in their device to access your application.
  • Know the carrier your user uses to access your application.
  • Know how long your users browses your site in the mobile space and collect the page depth of their browsing experiences.
  • Know the type of the mobile browser your users use to access your application in the mobile environment.
  • Know the user’s device’s screen size.
  • Do not trust the web server logs to design the scenario as user behavior is not the same as compare to desktop users.If your business grows ,users multiply exponentially compare to desktop users given that users will be using your site on the move.
  • There is very little or absolutely zero think time concept in Mobile space.Users are not going to wait to click the links unless they are verifying the some information before doing submit.

Sometimes it might happen that we do not have information on these points,in that I would suggest follow the mobile web standards and test taking those standards as your requirements.

Tags: ,

When are you really done with Performance testing

September 6, 2011 by kiranbadi1991 | Comments Off on When are you really done with Performance testing | Filed in Performance Engineering, Process, Project Management

Quite often we are taught that testing is said to be done when we are out of schedule/budget or when we have tested all the business process as outlined in the performance test plan.However my experience is telling me that they are all incorrect way of representing ”done”.I feel these are all more of the constraints imposed on the team for various reasons rather than measuring parameters or milestones.

Performance testing is done only when we have communicated our finding to all stakeholders which includes all our good and bad results.If we have some failures during execution,it makes more sense to access the impact of those failures in production and communicate these findings to all relevant stakeholders and get their signoff on these findings.If for any reasons,performance team is not aware of the impact, then they should ask the impact to the relevant concerned stakeholder before providing signoff.

A very small number of errors cannot be measuring criteria for closing the execution cycle.Years back I did use to accept less than 5% error count as acceptable criteria for closing the execution cycle however as I gained some more exposure I realized there were lot of other factors which also needs to be accounted for in addition to less than 5% errors.So today I feel  criteria for done is to access the impact of any leftovers unresolved errors and communicate these finding to all and get their signoff.

Tags: , , ,

Some Expectations from Software Tool Vendors

August 21, 2011 by kiranbadi1991 | Comments Off on Some Expectations from Software Tool Vendors | Filed in Performance Test Tools, Process, Small Businesses

As  a consultant ,what do you expect from the various tool vendors who are into the business of producing and selling tools ,

Below here are some of the expectations I have seen people talking or wants that every tool vendor should know or try to attempt,

1. Get close to your Users.Hear them, Feel them and be empathetic to them.This is universal rule of marketing.However definition of Users varies depending on the product involved and audience to whom you are selling.

2. Try to involve them in some kind of discussion, it could be community related or technical or something related to the industry overall.It really shows that you are committed to their profession and professional growth.If someone in the community speaks negative about your tool,stay calm and investigate rather than reacting.There always exists a section of users who are always demanding.If you don’t meet such users, for any reasons,then you can safely say you have miles to travel in your sales professional.

3. A good sales man is one who never ever says that his tool is best and can solve all problems by the click of the button.Its just that with growth of technology in recent years, we have seen exponential growth of  problems.Some problems have solutions, some remains unresolved and some requires workarounds.So investigate  and confirm before you speak else it should go as opinion rather than solution.

4.In my profession of Performance Engineering which requires a lot of technical expertise, I have seen some of best tools failing to capture reasonable market in spite of having relevant technical competency  for reasons that there wasn’t any kind of commitment from persons assigned to capture the market share for their tool.So if you are in business of selling tools, get involved with users by interacting in their professional forums,conferences or by writing white papers etc.I really appreciate the sellers who participate in  professional forums and help the users to see the complete picture.Yes this idea might not sound good to some folks as they might believe it creates some kind of bias in the mind of readers,but again we need to remember that we live in era of technology where information whether correct or incorrect is available at the click of button.I normally assume that bad experience is hard to hide now a days.

5.Teach the users as how to use the tool correctly and in his context, Fix the context in case you feel the context is incorrect.Tell him why is his context is wrong and how much it costs him for proceeding his way rather than your suggested way.People normally appreciate this kind of information rather than scripted  talk.Educating the users and then selling him gives you a lot more business than selling him as uneducated.

6.Finally if you are competing with other vendor whose areas of business is similar to yours, compete head on with merits and demerits of the case,but never leave the thread incomplete to guess,it leaves lot of questions unanswered in the minds of the readers and this impacts your reputation as a sellers, so for any reasons if you cannot reply back in the internet, use backdoor policy and try come to some kind of conclusion which looks good to all.The reasons I am writing this is in past 7 years of being part of my professional forums, I have just seen to so many vendors expecting change and good results after a few days of launching their tools.To capture a market, it requires a consistent effort in the right direction and targeting right users.You cannot be a overnight leader in your area unless you have mighty god standing behind you.

7. Your sales mantra should be” lets grow together” rather than just focusing on the growth of your own tool.Mantra here is if your user’s are growing only than profession will grow and so will your tool and your company.

Tags: , , , ,