Mobile Testing Part I: Twelve Challenges of Mobile

Just over a decade ago, the first iPhone was released.  Now we live in an age where smartphones are ubiquitous.  Our smartphones are like our Swiss Army knives- they are our maps, our address books, our calendars, our cameras, our music players, and of course our communication devices.  Testing software would not be complete without testing on mobile.  There are so many considerations when testing on mobile that I've decided to make this into a three part series: this week I'll discuss the challenges of mobile testing, next week I'll discuss mobile manual testing, and the following week I'll talk about mobile automated testing. 

First, the challenges!  Below are twelve reasons why testing on mobile is difficult.  I thought it would be fun to illustrate just what can go wrong with mobile software by describing a bug I've found in each area.  Some of these bugs were found in the course of my testing career, and some were found on my personal device as an end user.

1. Carriers: Mobile application performance can vary depending on what carrier the device is using.  In the US, the two major carriers are Verizon and AT&T, and we also have smaller carriers like Sprint and T-Mobile.  In Europe some of the major carriers are Deutche Telekom, Telefonica, Vodaphone, and Orange; in Asia some of the carriers are China Mobile, Airtel, NTT, and Softbank.  When testing software on mobile, it's important to consider what carriers your end users will be using, and test with those carriers.

Example bug: I once tested a mapping function within an application, and discovered that while the map would update based on my location when I was using one carrier, it did not update when I was using a different carrier.  This had something to do with the way the location was cached after a cell tower ping.  

2. Network or Wifi: Device users have the choice of using their applications while connected to the carrier's network, or while on wifi.  They can even make a choice to change how they are connecting in the middle of using the application; or their connection can be cut completely if they go out of network range.  It's important to test an application when connected to a network and when connected to wifi, and to see what happens when the connection changes or is lost completely.

Example bug: I have a wifi extender in my house. When I switch my phone's wifi connection to use the extender's IP, Spotify thinks I am offline.  I have to force-close the app and reopen it in order for Spotify to recognize that I am online.

3. Application Type: Mobile applications can be Web-based, native, or a hybrid of the two (developed like a Web app, but installed like a native app).  Some of your end users will choose not to use a native or hybrid app and will prefer to interact with your application in their phone's browser.  There are also a variety of mobile browsers that could be used, such as Safari, Chrome, or Opera.  So it's important to make sure that your web application works well on a variety of mobile browsers.

Example bug: There have been many times where I've gone to a mobile website and their "mobile-optimized" site doesn't have the functionality I need.  I've had to choose to go to the full site, where all of the text is tiny and navigation is difficult.  

4. Operating System: Mobile applications will function differently depending on the operating system.  The two biggest operating systems are iOS and Android, and there are others, such as Windows Mobile and Blackberry.  It's important to test on whatever operating systems your end users will be using, to make sure that all of the features in the application are supported in all systems.

Example bug: This is not a bug, but a key difference between Android and iOS: Android devices have a back button, while iOS devices do not.  Applications written for iOS need to have a back button included on each page so users will have the option to move back to the previous page.

5. Version: Every OS updates their version periodically, with new features designed to entice users to upgrade.  But not every user will upgrade their phone to the latest and greatest version.  It's important to use analytics to determine which versions your users are most likely to have, and make sure that you are testing on those versions.  Also, every version update has the potential to create bugs in your application that weren't there before.  

Example bug: Often when the version is updated on my phone, I can no longer use the speaker function when making phone calls.  I can hear the voice on the other end, but the caller can't hear me.

6. Make: While all iOS devices are manufactured by Apple, Android devices are not so simple.  Samsung is one of the major Android device manufacturers, but there are many others, such as Huawei, Motorola, Asus, and LG.  It's important to note that not every Android user will be using a Samsung device, and test on other Android devices as well.

Example bug: I once tested a tablet application where the keyboard function worked fine on some makes but not others. The keyboard simply wouldn't pop up on those devices, so I wasn't able to type in any form fields.

7. Model: Similar to versioning, new models of devices are introduced annually.  While some users will upgrade every year or two to the latest device, others will not.  Moreover, some devices will not be able to upgrade to the latest version of the OS, so they will be out-of-date in two ways.  Again, it's important to find out what models your end users are using so you can make decisions about which models to test on and to support.

Example bug: This is not a bug, but it was an important consideration: when Apple released a new model of the iPad that would allow a signature control for users to sign their name in a form, the software I was testing included this feature.  But older versions of the iPad weren't able to support this, so the application needed to account for this and not ask users on older versions to sign a document.

8. Tablet or Smartphone:  Many of your end users will be interacting with your application on a tablet rather than a smartphone.  Native applications will often have different app versions depending on whether they are designed for tablet or phone.  An application designed for smartphone can often be downloaded to a tablet, but an application designed for a tablet cannot be installed on a smartphone.  If a web app is being used, it's important to remember that tablets and smartphones sometimes have different features.  Test your application on both tablets and phones.

Example bug: I have tested applications that worked fine on a smartphone and simply gave me a blank screen when I tried to test them on a tablet.  

9. Screen Size:  Mobile devices come in many, many different sizes.  While iOS devices fit into a few sizing standards, Android devices have dozens of sizes. Although it's impossible to test every screen size, it's important to test small, medium, large, and extra large sizes to make sure that your application draws correctly in every resolution.  

Example bug: I have tested applications on small phones where the page elements were overlapping each other, making it difficult to see text fields or click on buttons.

10. Portrait or Landscape: When testing on smartphones, it's easy to forget to test in landscape mode, because we often hold our phones in a portrait position.  But sometimes smartphone users will want to view an application in landscape mode, and this is even more true for tablet users.  It's important to not only test your application in portrait and landscape modes, but also to be sure to switch back and forth between modes while using the application.  

Example bug: I tested an application once that looked great in a tablet when it was in portrait mode, but all of the fields disappeared when I moved to landscape mode.  

11. In-App Integration: One of the great things about mobile applications is that they can integrate with other features of the device, such as the microphone or camera.  They can also link to other applications, such as Facebook or Twitter.  Whatever integrations the application supports, be sure to test them thoroughly.  

Example bug:  I tested an application where it was possible for a user to take a picture of an appliance in their home and add it to their home's inventory.  When I chose to take a picture, I was taken to the camera app correctly, and I was able to take the picture, but after I took the picture I wasn't returned to the application.  

12. Outside of App Integration: Even if your application isn't designed to work with any other apps or features, it's still possible that there are bugs in this area.  What happens if the user gets a phone call, a text, or a low battery warning while they are using your app?  It's important to find out.

Example bug:  For a while, there was a bug on my phone where if my device timer went off while I was in a phone call, as soon as I got off the phone, the timer sounded and wouldn't stop.  

I hope that the above descriptions and examples have shown just how difficult it is to test mobile applications!  It may seem overwhelming at first, but in my next two blog posts, I'll discuss ways to make testing simpler.  Next week, we'll be taking a look at writing mobile test plans and assembling a portfolio of physical devices to test on.  

Cross-Browser Testing

In today's Agile world, with two-week sprints and frequent releases, it's tough to keep on top of testing.  We often have our hands full with testing the stories from the sprint, and we rely on automation for any regression testing.  But there is a key component of testing that is often overlooked, and that is cross-browser testing.

Browser parity is much better than it was just a few years ago, but every now and then you will still encounter differences with how your application performs in different browsers.  Here are just a few examples of discrepancies I've encountered over the years:

• A page that scrolls just fine in one browser doesn't scroll at all in another, or the scrolling component doesn't appear
• A button that works correctly in one browser doesn't work in another
• An image that displays in one browser doesn't display in another
• A page that automatically refreshes in one browser doesn't do so in another, leaving the user feeling as if their data hasn't been updated

Here are some helpful hints to make sure that your application is tested in multiple browsers:

Know which browser is most popular with your users

Several years ago I was testing a business-to-business CRM-style application.  Our team's developers tended to use Chrome for checking their work, and because of this I primarily tested in Chrome as well.  Then I found out that over 90% of our end users were using our application in Internet Explorer 9.  This definitely changed the focus of my testing!  From then on, I made sure that every new feature was tested in IE 9, and that a full regression pass was run in IE 9 whenever we had a release.  

Find out which browsers are the most popular with your users and be sure to test every feature with them.  This doesn't mean that you have to do the bulk of your testing there; but with every new feature and every new release you should be sure to validate all of the UI components in the most popular browsers.

Resize your browsers

Sometimes a browser issue isn't readily apparent because it only appears when the browser is using a smaller window.  As professional testers, we are often fortunate to be issued large monitors on which to test.  This is great, because it allows us to have multiple windows open and view one whole webpage at a time, but it often means that we miss bugs that end users will see.  

End users are likely not using a big monitor when they are using our software.  Issues can crop up such as: a vertical or horizontal scrollbar not appearing, or not functioning properly; text not resizing, so that it goes off the page and is not visible; or images not appearing or taking too much space on the page.  

Be sure to build page resizing into every test plan for every new feature, and build it into a regression suite as well.  Find out what the minimum supported window size should be, and test all the way down to that level, with variations in both horizontal and vertical sizes.  

Assign each browser to a different tester

When doing manual regression testing, an easy way to make sure that all browsers you want to test are covered is by assigning each tester a different browser.  For example, if you have three testers on your team (including yourself), you could run your regression suite in Chrome and Safari, another tester could run the suite in Firefox, and a third tester could run the suite in Internet Explorer and Edge.  The next time the suite is run, you can swap browsers, so that each browser will have a fresh set of eyes on it.  

Watch for changes after browser updates

It's possible that something that worked great in a browser suddenly stops working correctly when a new version of the browser is released.  It's also possible that a feature that looks great in the latest version of the browser doesn't work in an older version.  Many browsers like Chrome and Firefox are set to automatically update themselves with every release, but some end users may have turned this feature off, so you can't assume that everyone is using the latest version.  It's often helpful if you have a spare testing machine to keep browsers installed with the next-to-last release.  That way you can identify any discrepancies that may appear between the old browser version and the new.  

Use visual validation in your automated tests

Generally automated UI tests focus on the presence of elements on a web page.  This is great for functional testing, but the presence of an element doesn't tell you whether or not it is appearing correctly on the page.  This is where a visual testing tool like Applitools comes in.  Applitools coordinates with UI test tools such as Selenium to add a visual component to the test validation.  In the first test run, Applitools is "taught" what image to expect on a page.  Then in all subsequent runs, it will take a screenshot of the image and compare with the correct image that it has saved.  If the image fails to load or is displaying incorrectly, the UI test will fail.  Applitools is great for cross-browser testing, because you can train it to expect different results for each browser type, version, and screen size.  

Browser differences are something that can greatly impact the user experience!  If you build in manual and automated systems to check for discrepancies, you can easily ensure a better user experience with a minimum of extra work.  

You may have noticed that I didn't discuss the mobile experience at all in this post.  That's because next week, I'll be focusing solely on the many challenges of mobile testing!

A Gentle Introduction to Session Hijacking

We all know that Session Hijacking is bad, and that we should protect ourselves and our applications against it.  But it's difficult to get easy-to-understand information about what it is, and how to test for it.  In this post, I'll first describe the different types of session hijacking, and then I'll provide a walkthrough on how to test for session hijacking using the OWASP Juice Shop and Burp Suite.

Session Hijacking refers to when a malicious user gets access to authentication information, and uses it to impersonate another user or gain access to information they should not have.  There are several types of Session Hijacking:

Predictable Session Token- this happens when the access tokens being generated by an application follow some kind of pattern.  For example, if a token granted for a login for a user was "APP123", and a token granted for a second user was "APP124", a malicious user could assume that the next token granted would be "APP125".  This is a pretty obvious vulnerability, and there are many tools in use today that create non-sequential tokens, so it's not a very common attack.

Session Sniffing- this takes place when a malicious user finds a way to examine the web traffic that is being sent between a user and a web server, and copies the token for their own use.  This is classified as a passive attack, because the malicious user is not interfering with the operation of the application or with the request.

Client-Side Attack- in this scenario, the malicious user is using XSS to cause an application to display a user's token.  Then they copy the token and use it.

Man-in-the-Middle Attack- this attack is similar to Session Sniffing in that the malicious user gains access to web traffic.  But this is an active attack, because the malicious user uses a tool such as Burp Suite or Fiddler to intercept the request and then manipulate it for their purposes.

Man-in-the-Browser Attack- this takes place when a malicious user has managed to get code into another user's computer.  The implanted code will then send all request information to the malicious user.

Now we are going to learn how test for Session Hijacking by using a Man-in-the-Middle Attack!  But first, it's important to note that we are going to be intercepting requests by using the same computer where we are making the requests.  In a real Man-in-the-Middle Attack, the malicious user would be using some sort of packet-sniffing tool to gain access to requests that someone was making on a different computer.

The instructions provided here will be for Firefox.  Since I generally do my browsing (and blog-writing) on Chrome, I like to use a different browser for intercepting requests.  But Burp Suite will work with Chrome and Internet Explorer as well.

First, we'll need to download and install Burp Suite.  You can install the free version, which is the Community Edition.  Don't launch it yet.

Next, we'll set up proxy settings in Firefox.  Click on the hamburger menu (a button with three horizontal lines) in the top right corner of the browser.  Choose "Preferences" and then scroll down to the bottom of the page and click on the "Settings" button.  Select the "Manual Proxy Configuration" radio button, and in the HTTP Proxy field, type "127.0.0.1".  In the Port field, type "8080".  Click the checkbox that says "Use this proxy server for all protocols".  If there is anything in the "No proxy for" window, delete it.  Then click "OK".

Now it's time to start Burp Suite.  Start the application, then click "Next", then click "Start Burp".

Next we'll set up the certificate that will give Firefox the permission to allow requests to be intercepted.  In Firefox, navigate to http://burp.  You'll see a page appear that says "Burp Suite Community Edition".  Click on the link in the upper right that says "CA Certificate".  A popup window will appear; choose "Save File".  The certificate will download, probably into your Downloads folder.

Click on the hamburger menu again, and choose "Preferences".  In the menu on the left, choose "Privacy and Security".  Scroll to the bottom of the page, and click the "View Certificates" button.  A popup window with a list of certificates will appear.  Click on the "Import" button, navigate to the "Downloads" folder, choose the recently downloaded certificate, and click the "Open" button.  A popup window will appear; check the "Trust this CA to identify websites" checkbox, and then click "OK".  The certificate should now be installed.  Restart Firefox to ensure that the new settings are picked up.

Next, return to Burp Suite and turn the intercept function off.  We're doing this so that we're not intercepting web requests until we're ready.  To turn off the intercept function, click on the Proxy tab in the top row of tabs, then click on the Intercept tab in the second row of tabs, then click on the "Intercept is on" button.  The button should now read "Intercept is off".

Navigate to the Juice Shop, and create an account.  Once your account has been created, you'll be prompted to log in.  Before you do so, go back to Burp Suite and turn the intercept function back on, using that "Intercept is off" button.

Now that Burp Suite is set to intercept your requests, log into the Juice Shop.  You will see nothing happen initially in your browser; this is because your request has gone to Burp Suite.  In Burp Suite, click on the Forward button; this will forward the intercepted request on to the server.  Continue to click the Forward button until the Search page of the Juice Shop has loaded completely.

In Burp Suite, click the HTTP History tab, in the second row of tabs.  You will see all the HTTP requests that were made when the Search page was loaded.  Scroll down until you see a POST request with the /rest/user/login endpoint.  Click on this request.  In the lower panel of Burp Suite, you will see the details of your request.  Notice that your username and password are displayed in clear text!  This means that if anyone were to intercept your login request, they would obtain your login credentials, and would then be able to log in as you at any time.

Next, return to the Juice Shop, and click on the cart symbol for the first juice listed in order to add the juice to your shopping cart.  Return to Burp Suite and click on the Intercept tab, and then click the Forward button to forward the request.  Continue to click the Forward button until no more requests are intercepted.

Return to the HTTP History tab in Burp Suite, and scroll down through the list of requests until you see a POST request with the api/BasketItems endpoint.  Right-click on this request and choose "Send to Repeater".  This will send the request to the Repeater module where we can manipulate the request and send it again.  Return to the Intercept tab and turn the Intercept off.

Click on the Repeater tab, which is in the top row of tabs.  The request we intercepted when we added a juice to the shopping cart is there.  Let's try sending the request again, by clicking on the Go button.  In the right panel of the page, we get a 400 response, with the message that the Product Id must be unique.  So, let's return to the request in the left panel.  We can see that the body of the request is {"ProductId":1,"BasketId":"16" (this number will vary by your basket id),"quantity":1}.  Let's change the ProductId from 1 to 2, and send the request again by clicking the Go button.  We can see in the response section that the request was successful.

Let's return to the Juice Shop and see if we were really able to add an item to the cart by sending the request in Burp Suite.  Click on the Your Basket link.  You should see two juices in your cart.  This means that if someone were to intercept your request to add an item to your cart, they could manipulate the request and use it to add any other item they wanted to your cart.

How else might we manipulate this request?  Return to the Repeater tab in Burp Suite.  The request we intercepted is still there.  This time, let's change the BasketId to a different number, such as 1.  Click Go to send the request again.  The response was successful, which means that we have just added a juice to someone else's cart!

So, we can see that if a malicious user would be able to intercept a request to add an item to a shopping cart, they might be able to manipulate that request in all kinds of ways, to add unwanted items to the carts of any number of users.  They are able to do this because the request they intercepted has the Authorization needed to make more successful requests.  When you set up Burp Suite to intercept requests in your own application, you will be able to test for Session Hijacking vulnerabilities like this one.

This concludes my series of posts on Security Testing, although I will probably write new ones at some point in the future.  In the next few weeks, we'll take a look at browser testing, mobile testing, and visual validation!

Hidden in Plain Sight- Using Dev Tools to Find Security Flaws

A common misconception is that all security testing is complicated.  While some testing certainly requires learning new skills and understanding things like networks, IP addresses, and domain names, other testing is extremely simple.  Today we're going to talk about three security flaws you can find in an application by simply using your browser's developer tools.  These flaws could be exploited by an average user of your application, not just a well-trained black-hat hacker.  

Editing Disabled Buttons: 

When you are on a web page that has a button that is disabled and only enables when certain criteria are met, such as filling out all the fields in a form, it may be possible to enable it.  I've included instructions on how to do this in my blog post on testing buttons.  

A user of your application might use this flaw to get around submitting a form where there are required fields that she doesn't want to fill in.  Or a user might enable an Edit button and submit edits to data that he shouldn't be able to edit.  

Viewing Hidden Data:

Recently someone showed me a security flaw in an application that was listing contact details for various members of the site.  Depending on the user's access rules, there were certain fields for the members, such as their personal address, that were not displayed on the page. But when the developer tools were opened, all of the hidden fields were displayed in the Elements section!  

Any user of this application could open up the developer tools and search through them to find personal data for any of the members of the site.

Finding Hidden Pages:

It's possible to find links that are not displayed on a web page by looking in the Elements section of the developer tools.  Let's try this out using the OWASP Juice Shop.  I'll be providing instructions using Chrome, but you can also do this with Firefox and Internet Explorer.  Navigate to the Juice Shop, then click on the three dots in the top right corner of the browser.  Select "More Tools" and then "Developer Tools". This will open the Dev Tools section.  Click on the Elements tab if it is not already selected by default.  We're going to take a look through the HTML of the page to see if we can find links that are not displayed in the browser.  Let's get our bearings by clicking on a displayed element in the page.  On the Juice Shop main page, there is a nav bar on the top of the page, with links such as "About Us" and "Contact Us".  Right-click on one of the elements in the nav bar, and choose "Inspect".  Notice that this highlights the corresponding section of the HTML in Dev Tools.  You'll see a number of items all tagged with the <li> tag; these are the items in the nav bar.  Open up each one by clicking on the carat on the left side, and you'll find one that looks like this:

The status of this list item is hidden, as you can see by the "ng-hide" attribute.  The 'ng-show="isLoggedIn"' attribute tells us that this nav bar item should only display when the user is logged in.  Finally, we see the link that the nav bar item would take us to if we were to click on it when it was displayed: "#/recycle".  Let's try navigating to this link by changing the page's URL from https://juice-shop.herokuapp.com/#/search to https://juice-shop.herokuapp.com/#/recycle.  Click Enter, and you will be taken to the recycle page.  You have successfully navigated to a page that you should only have access to if you have logged in!

This is why it is important to do authorization checks when a user navigates to a page.  It's not enough to simply hide a link, because it's so easy to find hidden links by looking in Dev Tools.  Any user could find hidden links in your application and navigate to them, which could give them access to an admin page or a page with data on other users.  

As you can see, testing for these types of security flaws is quick and easy!  I recommend checking for these flaws whenever you test a web page.  And here's a bonus vulnerability; if you create an account and log into the Juice Shop, and look in the Network section of Dev Tools, you will see your username and password in plain text!  We'll talk more about this next week, when we take on Session Hijacking.