Automation Index Formula – A checklist to help in identifying the tests that are feasible to automate
“Just because a test is automatable it does not mean it should be automated” Elfriede Dustin Automation testing begins with an analysis of what is feasible to automate, taking into account the budget, resources, schedule, and available expertise. Given limited resources and tight deadlines, we first need to prioritize what is to be automated. The effort required can be measured with the help of the Automation Index. Automation Index Formula The automation Index is the ratio of the number of test cases that are feasible to be automated against the total number of test cases. AI = TFA / TC where AI = Automation Index TFA = Tests feasible to be automated TC = Total number of Test Cases A checklist to help in identifying the tests that are feasible to automate: Tests that are yes for the above are good candidates for automation. Factors that are to be considered in addition to the above are: Have questions? Contact the software testing experts at InApp to learn more.
Mobile Application Testing
Introduction: Handheld devices are evolving and becoming increasingly complex with the continuous addition of features and functionalities. Testing is challenging in the handheld, wireless world because problems are new, or they show up in new ways. This paper is aimed to highlight certain crucial areas the tester needs to concentrate on while testing mobile applications. The 4 main areas to consider: Understanding the behavior of the device UI & Usability Testing. External Constraints Stress Testing Understanding the behavior of the device If you are new to a device the first thing you should do is to get familiar with how the common device functionalities work – such as its Phone, Camera, Contacts, Calendar, Program, etc. Things to note while exploring inbuilt applications: Overall color scheme/theme of the device. Style and color of icons Progress indicators when pages are loading. Menus – How they are invoked and typical items they contain. Overall responsiveness of applications on the device. UI & Usability Testing: The unique features of mobile devices pose a number of significant challenges for examining the usability of mobile applications, including screen orientation, multi-modality, small screen size, different display resolutions, soft keyboards, and touch screens. Screen Resolution If your application is supported on various devices that have different screen resolutions, make sure you test with the device that has the smallest screen and the application still looks good on larger screen sizes as well. Screen orientation (Landscape/Portrait modes) If your device supports screen orientation changes be sure to include lots of testing where you rotate the device from portrait to landscape display, and vice versa, on all of the pages within your application. It is also important to test input reactions when the screen orientation is changed. Try using the soft keyboard while changing the orientation repeatedly. Attempt this repeatedly and quickly to see if the rapid changes in orientation have a negative effect on the application. Touch Screens: Make sure that the application supports multi-touch (eg: pinch, two-finger tap, two-finger scroll, spread, two-hand spread, etc), single touch – eg: tap, double tap, scroll, etc, touch based on the requirement. The application should be tested for long touch and soft touch behavior. Soft keyboards – Points to consider Does the soft keyboard appears automatically Does the first layer of the soft keyboard include shortcuts related to highlights? Does a long touch on a soft character key bring up several different character choices? Can the soft keyboard be dismissed and re-displayed easily Can the soft and hard keyboards be used interchangeably (if the device has both) Do soft keyboard characters entered in password fields only show up as **** Multi-modality: Multi-modality combines voice and touch (via a keypad or stylus) as input with relevant spoken output (e.g., users are able to hear synthesized, prerecorded streaming or live instructions, sounds, and music on their mobile devices) and onscreen visual displays in order to enhance the mobile user experience and expand network operator service offerings. Make sure that the application supports the functionality based on the requirement. External Factors Affecting Mobile Application Testing Network Connections: App going to be used on devices in various locations with various network connection speeds, it is important to plan testing coverage for the following scenarios: Only Wi-Fi connection Only a 3G/2G connection With no SIM card in the device In Airplane mode (or all connections disabled) Using the network through a USB connection to a PC Test intermittent network scenarios that a user might encounter in the real world: Phone calls: The tester has to check the application behavior during incoming and outgoing calls. Make sure that the application works fine during the following phone calls. The application is interrupted by an incoming call, originator hangs up the call The application is interrupted by an incoming call, terminator hangs up the call The application is interrupted by placing an outgoing call, originator hangs up the call The application is interrupted by placing an outgoing call, terminator hangs up the call. Other Interruptions: The tester has to consider the below interrupts that could have an impact on the functionality or overall responsiveness of your application. Text messages Voicemail notifications Calendar events Social media notifications (Facebook, Twitter, etc) Alarm clocks Low battery notifications Device Settings Explore your device’s options, and change settings such as the following to see how they affect your application: Sound profiles – Does your application respect the device’s sound settings? Device password/unlock pattern – Does your application still install correctly when prompted for a password/unlock pattern? Font – How does choosing a different font family, size, or style affect the appearance and usability of your application? Screen time out/Auto on, off- Is your application subject to screen dimming or automatically turning off even when it is actually busy? Screen orientation – Does your application respect this setting? Connections – How does enabling/disabling Bluetooth or other connection types affect your application’s behavior? Stress Testing Certain mobile applications consume more memory and CPU than desktop applications. Stress testing is a must to identify exceptions, situations with the application hang, and deadlocks that may go unnoticed during functional and user interface testing. Note the behavior of the application while testing with the following scenarios: Load your application with as much data as possible in an attempt to reach its breaking point. Perform the same operations over and over again, particularly those that load large amounts of data repeatedly. Perform the repeated operations at varying speeds – very quickly or very slowly. Leave your application running for a long period of time, both interacting with the device and just letting it sit idle, or perform some automatic task that takes a long time. Test multiple applications running on your device so you can switch between your application and other applications. After testing several functionality switched off and switch on the device. Have questions? Contact the software testing experts at InApp to learn more.
Everything About Single Sign On (SSO)
What do you mean by Single Sign On (SSO)? Single sign-on is authentication to access the different applications from a single environment by without giving multiple usernames or passwords. Single sign-on uses only one login and through this users access different applications. Single sign-off, reverse action of SSO that single action of signing out terminates access of multiple applications. SAML (Security Assertion Mark-up Language) SAML is an XML standard that allows secure web domains to exchange user authentication and authorization data. SSO authentication process only applies to the web-based application For Windows, usable passwords and synchronize those passwords with your internal user database using the Provisioning API Using SAML, an online service provider can contact a separate online identity provider to authenticate users who are trying to access secure content. How SAML works in SSO Let’s see how SAML works in SSO authentication. Look at the workflow of SSO authentication used in the Google application. Initially, before the whole process, the partner must provide Google with the URL for its SSO service as well as the public key that Google should use to verify SAML responses. The following steps are the workflow of the entire SSO authentication process. Each step number is plotted in the above figure: The user attempts to reach a hosted Google application, such as Gmail, Start Pages, or another Google service. Google generates a SAML authentication request. The SAML request is encoded and embedded into the URL for the partner’s SSO service. The RelayState parameter containing the encoded URL of the Google application that the user is trying to reach is also embedded in the SSO URL. This RelayState parameter is meant to be an opaque identifier that is passed back without any modification or inspection. Google sends a redirect to the user’s browser. The redirect URL includes the encoded SAML authentication request that should be submitted to the partner’s SSO service. The partner decodes the SAML request and extracts the URL for both Google’s ACS (Assertion Consumer Service) and the user’s destination URL (RelayState parameter). The partner then authenticates the user. Partners could authenticate users by either asking for valid login credentials or by checking for valid session cookies. The partner generates a SAML response that contains the authenticated user’s username. In accordance with the SAML 2.0 specification, this response is digitally signed with the partner’s public and private DSA/RSA keys. The partner encodes the SAML response and the RelayState parameter and returns that information to the user’s browser. The partner provides a mechanism so that the browser can forward that information to Google’s ACS. For example, the partner could embed the SAML response and destination URL in a form and provide a button that the user can click to submit the form to Google. The partner could also include JavaScript on the page that automatically submits the form to Google. Google’s ACS verifies the SAML response using the partner’s public key. If the response is successfully verified, ACS redirects the user to the destination URL. The user has been redirected to the destination URL and is logged in to Google Apps. These are the basic principle behind the working of SSO authentication. Advantages of SSO applications Reduces phishing success, because users are not trained to enter passwords everywhere without thinking. Reducing password fatigue from the different user names and password combinations Reducing time spent re-entering passwords for the same identity Can support conventional authentication such as Windows credentials (i.e., username/password) Reducing IT costs due to a lower number of IT help desk calls about passwords Security on all levels of entry/exit/access to systems without the inconvenience of re-promoting users Centralized reporting for compliance adherence. Criticisms SSO servers can introduce a single point of network failure. The SSO server and other host security must be hardened What is the area that needs to be tested in SSO authentication applications Check Browser Cache Check authenticated and non-authenticated users Check users have different privileges Check URL security Check server performances Basic Test Scenarios for SSO Login Authentication: 1. Valid users logged in to their own intranet application and accessed external host application Result:- Users should be able to access external host applications since they are valid sso users. 2. Invalid users logged in to their own intranet application and access the external host application Result:- External application login prompt should show up since they are invalid sso users. 3. Valid user logged in intranet and accessed the external host application and at the same time intranet session expires Result: The user should be able to work on an external host application even if the intranet session expires. 4. Valid user logged in to intranet, accessed an external application and logged out, and again access an external application Result:- The user should be logged in to the external application without a login prompt. 5. Valid user logged in to intranet, accessed external application, and then session out from innotas application Result:- The user should be able to continue working on the external application after expires since a session has already been made with the intranet application, so while the session is out user credentials will be automatically fetched from the browser cache and logged in. Have questions? Contact the technology experts at InApp to learn more.
Software Quality Control vs Software Quality Assurance (QC vs QA)
Difference between QA and QC This is one of the most frequently asked questions, with many different versions of the definition. What is Software Quality Control (SQC)? Software Quality Control (SQC) is the set of procedures used by an organization to ensure that a software product will meet its quality goals at the best value to the customer and to continually improve the organization’s ability to produce software products in the future. [Source: Wikipedia: http://en.wikipedia.org/wiki/Software_quality_control] What is Software Quality Assurance (SQA)? Software quality assurance (SQA) consists of a means of monitoring the software engineering processes and methods used to ensure quality. The methods by which this is accomplished are many and varied and may include ensuring conformance to one or more standards, such as ISO 9000, or a model such as CMMI. [Source; Wikipedia: http://en.wikipedia.org/wiki/Software_quality_assurance] “Product” and “Process” are the keywords that distinguish the differences between QA and QC. What is Quality Control? Quality Control is “Product-oriented”, it focuses on the “product” itself, whether it meets its quality goals and user requirements. For example, Testing and Review fall into this category. What is Quality Assurance? Quality Assurance is “Process-oriented”; focuses on whether the process in a project conforms to organizational standards and methodologies, as defined. Since QA focuses more on the whole project, hence it can have supervision on quality control.
Service Oriented Architecture
Service Oriented Architecture SOA is an evolution of distributed computing designed to allow the interaction of software components, called “services”, across a network. Applications are created from a composition of these services and the services can be shared among multiple applications. Need for Service-Oriented Architecture Systems today are bigger Systems need to be interconnected OO works for small-medium sized systems CO (Component Orientation) works for medium-large systems SOA is an evolution from OO and CO and fulfills the need for very large systems What is Service Oriented Architecture SOA is an approach that helps design, implement, and deploy large systems that are created from components that implement discrete business functions. These components are called “Services”, which are distributed across geography, across enterprises and can be reconfigured into new business processes as needed. What is a Service A Service is a program that can be interacted with through and well-defined message exchanges. Messages can be sent from one service to another without consideration of how the service handling those messages is implemented, thus providing interoperability and adaptability. Services must be designed for both availability and stability. Services are loosely coupled to help provide the most important benefit of SOA: Agility 4 Tenets of Service-Oriented Architecture 1. Service is Autonomous Services are entities that are independently deployed, versioned, and managed. 2. A Service has explicit boundaries Services interact through explicit message-passing over well-defined boundaries, published via WSDL. 3. A Service exposes schema & contract and not class or type Services communicate using XML, which is agnostic to both programming languages and platforms. The schema defines the structure and content of the messages, while the service’s contract defines the behavior of the service itself. 4. A Service allows or denies use based on policy A Service decides what message it should process. Example 1 A Human Resources Management application could be created from the following services: An Employee Administration Service to manage hiring, changes in status, and termination. A Salary and Review Administration Service to manage salaries and employee performance reviews according to corporate standards. An IT Security Service to manage the addition and removal of access rights for employees according to their role and employee status. A Payroll Service provided securely over the Internet by an outside provider. An HR Department Portal Service provides a web browser-based user interface for members of the HR department presenting the functions of the above services. Example 2 A travel application used on a mobile can use SOA to obtain all the below information from different service providers Hotel Rates Currency Exchange Rates Weather information A list of places to visit Have questions? Contact the technology experts at InApp to learn more.