API Testing: This solution will solve all your problems
We have heard of many people really struggle with API testing. Many of our fellow colleagues and some of the customers would talk about their problems while doing API testing. We tried to investigate the causes of these problems. We found that in large custom software development projects, API testing is a challenge. In large projects, generally, there are two teams working. One on the back end and one on the front end. The speeds of the two teams never match and each team is waiting for the other to test. The folks developing the front end would either wait for the back-end folks to be ready or would have to spend time creating mock objects for front-end testing. Then there were software development projects which had interactions with external servers. Generally, there is no control over an external server. Again the only way to test the API would be to create mock objects. This would consume a lot of programming time. Generic problems related to API testing Other than these specific cases, there were some generic problems related to API testing. These problems were all-pervasive, across all projects. For example: “testing for error response.” The test case that was often left untested was how the client behaves when the server sends an error response. How does the client behave when an HTTP status of 500 is returned or when a 4xx or 3xx is returned? The client must behave predictably and gracefully but testing this was extremely difficult. In order to solve this problem, we needed a tool that would insert itself between the Web-Client (Browser, Phones, etc.) and the Server (Private, Cloud, or Third Party) and mock the Server. Thus the client would feel that the server is available in all respects and the Client testing can proceed even if the server is not available. We didn’t find a tool that would provide this solution. Hence we went about developing this tool which we have now come to call “iBounce.” Some of the key features of iBounce are: 1. Support MockServices: It introduces itself between the Web-Client and Application-Server and Mocks the Server. The Web-Client can be anything that consumes backend RESTful services – web browsers (HTML/JS), Phones, or other devices. The Application-Server is a server exposing a REST API. 2. Allow Sharing of Test Cases across platforms: iBounce does not per-se do test cases but it defines each interaction between the server and the client. As we call it “The Contract” between the Client and the Server. The contracts between the client and server remain the same regardless of the Phone / Browser used and in this way defines Test-Cases can be shared between devices/web-browser. This supports test automation in many ways and is very useful when we do not have access to a third-party such as Google APIs, AWS Management APIs, Financial Services, or Data.gov APIs. 3. Support File Access: We could load the Contract Information from files. 4. Easy for Non-Technical Use: Designed primarily for non-technical use. 5. Easy to create and validate test cases: We have implemented API capture. This is useful to initially create the Contract / Test Cases between the Server (maybe external) and the client. This makes the operation easy. 6. Have the ability to run from a CI environment: In the CI environments, while executing test jobs for the Client (only), the server and APIs must have already been tested. In many instances, we do not have the server ready or the server is not available for testing as it is a third-party server. In these cases, we introduce iBounce to mock the server and run the test cases in the CI environment. iBounce is being developed in a DevOps mode and is under continuous development and we expect to add many more features soon. Need help? Contact us now!
Managed Vs Unmanaged Cloud – An Infographic
There are lots of numbers involved when migrating to the cloud. There needs to be an important choice to be taken between a managed cloud (where the vendor manages, patches, and troubleshoots your cloud surrounding) or an unmanaged cloud (where your IT department has to do that on its own). Here’s an infographic focusing on the key differences. Managed Vs Unmanaged Cloud Analyzing the total cost of ownership when migrating to the cloud. The Extra Edge A managed cloud enables businesses to spend time on strategic work instead of baseline operations like procuring and maintenance. Saving Time over Cost Deploy hardware faster Best-in-class functionality Virtualization and Automation Brings down costs in procuring, testing, and deploying Adaptability With the flexibility for internal IT staff to focus on more value-add products, the managed cloud allows businesses to quickly react to market shifts while preparing for future demands. Employee Comfort Developing new competencies is an important step in increasing employee satisfaction, as it allows employees to gain new skill sets. Need help? Contact us now!
Front End API Testing – A Neglected Area
How do you test if the incoming API Calls to back-ends are correct? How do you test if the front end behaves properly on APIs returning errors? How do you test the front end before the back end is ready? The New Emerging Architecture With the emergence of Smartphones and HTML5, the front end of application software has clearly separated from the back end. The MVC5 of Microsoft, Java, PHP, Android, and iOS allows a clear separation of the front-end and back-end. in a typical environment, the Back-end applications usually expose themselves as REST API to which the front-end calls using XML or lately JSON. The Gap While there are many tools to test the Back-End, the database access, the API testing, and back-end security, there are few tools to check the API Calls! The problem is aggravated when the APIs are third-party APIs to which you do not have access. What do we do? The only way would be to capture the API calls from our front end Bounce back the expected (contracted) reply from the server and check against that. Such a tool could also return HTTP status 200, 300, 404, 500, and other errors, enabling us to test the front end under different API return values! Test out HTTP and the new HTTP/2 protocols. iBounce – API Testing Tool At InApp we have developed an effective API Testing Tool to do precisely that. We call it iBounce! This API Testing Tool has increased our productivity tremendously by helping us develop the front-end of applications, which consume public APIs, and helped us in developing the Front-End and Back-End by two teams asynchronously. Thus the front-end and back-end teams are not waiting perpetually for each other; Solving a lot of project management deadlocks. Need help? Contact us now!
SaaS Testing Using Cloud: An Emerging Discipline
Introduction More than a decade ago, in the era when technology evolved to bring in virtualization in the mode of cloud computing, a funny reply came from the owner of an organization when one of his subordinates shared a thought on migrating their services to the cloud. The reply was, “I just don’t trust keeping our data in the cloud. Plus if we got rid of the server, where would we keep the coffee maker!!! ?”. Later, after a decade, we witnessed that the cloud and its services became a game-changer and the deciding factor in the existence of many software organizations. Organizations can meet business demands in a quicker way and by achieving scalable costs by using Software as a Service, thus helping them to focus on their core business. Software as a Service (SaaS) testing using the cloud minimizes the hiccups that arise in traditional testing thereby providing infinite effective storage and flexibility via distributed environment. What is SaaS Testing? SaaS is a software distribution model in which applications are hosted by a vendor or service provider and made available to the customers over a network, typically the Internet. SaaS Testing refers to the methods used to ensure that applications built using the software as a service model of development function as designed. It also ensures the performance, security, and reliability of an application by leveraging the cloud-based infrastructure and computing resources to perform for a regular on-premise platform. Objectives of SaaS Testing should be: SaaS deployments should take less time than on-premise software deployments. Software hosted on the cloud should be highly secure, scalable, and reliable. The extent to which the SaaS solution can be customized Why SaaS Testing? SaaS testing is intended to be a speedy and productive manner to get applications released to customers. In agile methodology testing and quality assurance (QA) has become more critical as applications are delivered more rapidly. Since the number of testing increases, more focus should be provided on the quality of the applications developed in a quick and efficient manner. A. Efficient Load Testing There will be times when we are unable to predict the number of users who use the application. So in order to perform efficient load testing, we can approach any cloud service where we can simulate users and easily scale up to the desired load to measure the performance of the application under a large number of concurrent users. B. Replicate real-world usage It also helps users’ data to be moved to large data centers, which are remotely located, with the user being able to access the same anytime needed. C. Reduces the direct price of the equipment It reduces the direct price of equipment maintenance and management and helps attain rapid ROI on application assets and brings about faster time to market. Types of SaaS Testing Functional Testing SaaS Functional testing is performed for both remote and local applications. It consists of testing all features of the system including hardware and software testing. The different types of functional testing are System, Integration, and User Acceptance Testing. Non-functional Testing This type of testing is done to ensure that a web application meets the specified requirements; the way an application operates, rather than the specific behaviors of that application. It includes Security testing, Stress testing, Load Testing, and Performance Testing. Ability Testing Techniques Ability Testing Techniques ensure that the user receives appropriate services from the cloud environment on demand. Under this category, Compatibility and Interoperability Testing, Disaster Recovery Testing, and Multi-Tenancy testing are performed. Testing as a Service in Clouds TaaS aka Testing as a Service is a subset of a SaaS offering, allowing groups to outsource their entire test effort. With this model, the manpower and expertise required to execute the test are provided. This type of cloud testing is an approach that can also be used for specialized types of tests, such as performance, mobility, or security testing. The execution can be performed either on the client site or remotely from the outsourced provider’s test lab/facilities. TaaS Architecture Points to successful SaaS testing Build your understanding of the cloud Before moving your project to the cloud the objective and the strategy should be clearly defined. The organization or the management team should do formal research about the business needs and the limitation of moving the project to the cloud. Formulate your testing strategy The type of test and the risk involved during the test should be analyzed by creating a good test strategy. The duration of the usage of any cloud tool should be predefined and well-documented. Select a service provider A service provider has an enormous amount of experience and stability is the prime factor when choosing the vendor. License, cost, set-up, and tear down of the environment should be thoroughly scrutinized. Execute the test Companies should plan for optimal utilization of test infrastructure since it is a critical phase where applications are tested according to the defined test strategy. Monitor and analyze test results Test results should be monitored in real-time to understand and react to capacity – or performance-related issues. Cloud Testing Tools Needless to say, the emerging popularity of cloud testing has given rise to a set of cloud-based testing tools in the market. The right choice of testing tools depends on multiple parameters including application architecture, context, and customer needs. Here are some of the highly popular cloud-based software testing tools. 1. SOASTA Cloud Test It enables four types of test automation on a single web platform – Mobile Functional & Performance testing and Web-based Functional & Performance testing. 2. LoadStorm It is a load-testing tool for web and mobile applications. It is easy to use and cost-effective. It is ideal to check performance under excessive traffic or usage. 3. BlazeMeter It is used for end-to-end performance and load testing of mobile apps, websites, and APIs. It is JMeter compatible and can simulate up to 1 million users. 4. Nessus It is a widely used vulnerability scanner that
What’s The Worst Advice we’ve ever heard about Test Automation Tool Comparison?
The worst advice you can hear on Automated Testing Tools comparison is “There isn’t any need for using the tools trial version.” In this article, we will explain why a trial is essential when doing an Automated Testing Tools comparison and how we can do this trial systematically. There are many things you need to do while you compare Automated Testing Tools. You do an analysis on the basis of your selection criteria. To know the Automated Testing Tools comparison criteria please read our blog: Automated Testing Tools Selection Headache Solved How to Choose Automated Testing Tools? But finally, just before you take the plunge with the tool you have shortlisted, it is very important to do a trial. Till the time you haven’t gotten your hands dirty with the tool itself, you can’t really tell how useful or how useless it would be for you. Of course, you can’t spend weeks doing a trial. In order to do the trial systematically let me present to you 4 point plan of action: Automate selected scenarios using the shortlisted tools. When doing a trial you can’t run the full set of test cases. You wouldn’t want to waste your time doing so when you aren’t fully convinced this is your tool of choice. Instead, you should choose the test case scenarios and run them on the trial version. Select a complex test case. So which are the test case scenarios, which would be run on the trial version? This is a critical decision. If you choose a scenario that is simplistic, you wouldn’t know if your complex scenarios will run. Therefore choose a scenario that is fairly complex and covers all the critical components of your software. This would ensure that the automation test coverage through this tool is high. It is also important to keep the customer in mind when you shortlist the scenarios. It is important to choose a scenario that is critical to your customer. This is after all the main aim of test automation. Find a Known Issue. Don’t forget the purpose of testing. The purpose of testing is to explore any bugs in your application. The real test of the effectiveness of a tool is in its ability to find a bug. But how do you know if the tool has this ability? Well, if the application has a known bug, automate it and showcase that the issue is identifiable using the script execution Generate and Analyze the Report. Lastly, it is very important to have a close look at the report produced by the tool. Is the report in the format that you need? Is it easily understandable? All these questions are important before you finally end your Test Automation tool comparison. Need help? Contact us now!
Using Implicit and Explicit Wait in Selenium
Explicit Implicit wait selenium is an important command. They are used while running automation scripts created using Selenium Web Driver. Implicit and Explicit Wait in Selenium is primarily used to handle the different load times of elements on the web browser. Using Implicit and Explicit Wait in Selenium Example Consider a situation in which you are testing the Facebook web application and you want to test a scenario in which you post a comment. When you automate this test scenario, there is a possibility it may fail, especially during low bandwidth and high usage. The reason for the automation failure, in such cases, is that the post button was yet to load. Elements on a web application load at different times. In order to handle this, Selenium has introduced the usage of “Explicit” and “Implicit” wait. Selenium documentation on Explicit and Implicit wait warns users not to use both together. Using Implicit Wait in Selenium The implicit wait is a quick and easy way to handle the problem of elements loading at different times on a web application. It is applicable at a global level and affects all the elements. It basically tells selenium to wait for a specified time before it throws a “No Such Element” Exception. Let’s dive into an Example: driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS) In the above example, we have instructed selenium to wait for the time frame of 10 seconds if the element is not present. It means that if the element is not located on the web page within that time frame, it will throw an exception. The script will keep polling the DOM to find if the element has shown up. The polling interval cannot be changed. If the element is found selenium continues with the execution. There are some apparent drawbacks to this: Applied globally and hence affects all elements It can be only used to find elements and not conditions The polling interval cannot be changed It will slow down the execution time Using Explicit Wait in Selenium The explicit wait is a remedy to some of the problems which occur in implicit wait. However, the flip side to explicit wait is the complexity and the number of lines of code. When we use Explicit wait, we tell the Selenium web driver to wait for a condition to occur. This is different from waiting for a specified period of time. Here is an example: WebDriverWait wait = new WebDriverWait(<expected condition>,10) Here selenium will wait for an expected condition or 10 seconds whichever is earlier. The list of expected conditions for Java can be found here. Some of these conditions are listed below: alertIsPresent() elementSelectionStateToBe() elementToBeClickable() elementToBeSelected() frameToBeAvaliableAndSwitchToIt() invisibilityOfTheElementLocated() invisibilityOfElementWithText() presenceOfAllElementsLocatedBy() presenceOfElementLocated() textToBePresentInElement() textToBePresentInElementLocated() textToBePresentInElementValue() titleIs() titleContains() visibilityOf() visibilityOfAllElements() visibilityOfAllElementsLocatedBy() visibilityOfElementLocated() A user on Trello explained the differences in using Explicit and Implicit very well. I am summarizing the differences between Implicit and Explicit Wait selenium here: Explicit: Is in part of selenium which is local Can handle most of the conditions Returns either success or timeout error A success condition can even be the absence of an element Polling time can be customized But the as long and complicated syntax Implicit: Simple usage Is in the part of Selenium that is Global (the part controlling the browser) Can only work with finding elements Returns element found or error Polling time cannot be customized Need help? Contact us now!
Make Your Testing Run Faster with Protractor
Protractor is an E2E test framework for AngularJS app testing. It is a Node.js program that works on top of WebDriverJS. Protractor was developed by the Angular team for testing Angular. It recognizes Angular’s processes and waits for them to complete. As a result, the browser “sleep” time is optimized and the tests run faster in Protractor testing. Protractor Vs Selenium If you visualize Selenium and Protractor diagrammatically, the protractor encapsulates all of the Selenium. Since Protractor is built on a Selenium base, all the functionalities of Selenium are also available for Protractor testing. As you can see in the diagram, large parts of the functionalities of Selenium and Protractor are similar. But Protractor is also different in many ways from Selenium. One of the bigger differences is that Selenium interacts with a web page in a different manner when compared with Protractor. In Selenium, because of the way it interacts with the webpage, you have to wait or sleep many times in running automation. Selenium is not able to optimize the wait time for elements to appear on the web pages. Selenium requires the setting up of waits (Implicit and Explicit) so that automation can be synchronized. In case of wait is not programmed into an automation code, selenium will throw exceptions. In the case you are using an Implicit wait with Selenium, you can wait for the element. In case you are using an Explicit wait with Selenium, you can customize the waits to correspond with some expected conditions (elements to be clickable, elements to be visible, etc). You end up with lots of waiting based on what “wait” you program into the automation. The protractor is very tightly coupled with AngularJS and therefore is able to optimize wait and reduce wait time. It also has commands like `browser.waitForAngular()` to wait until the cycle is done before continuing on. Because of these functionalities in Protractor, you can avoid waiting (Explicit or Implicit) for business logic to process. In fact, Protractor automatically applies this command `browser.waitForAngular()` before every WebDriver action. For details on this please visit this page. With Protractor testing, the test script runs using Node.js. The protractor runs one extra command before performing an action on the browser to make sure that the browser application which is being tested has stabilized. For example: element(by.CSS(‘button.happy’)).click(); This will result in three commands being sent to the Browser Driver. /session/:sessionId/execute_async – First, Protractor tells the browser to run a snippet of JavaScript. This is a command which asks Angular to send a response when the application is done with all timeouts & asynchronous requests, and ready for the test to start again. /session/:sessionId/element – Then, the command to search the element is sent. /session/:sessionId/element/:id/click – Finally the command to “click” is sent. Instruct the web driver to wait until Angular has finished rendering and has no outstanding $http or $timeout calls before continuing. Note that Protractor automatically applies this command before every WebDriver action. Please visit this page for further details. Need help? Contact us now!
3 Myths of Test Automation
Let’s be clear, “3 Myths of Test Automation” is not about bashing test automation. If you are a tester frustrated at having to do test automation (or generally unhappy!), reading “3 Myths of Test Automation” will not bring you any solace. However, if you are a CEO/ CTO/ Product manager or someone genuinely interested in finding a value proposition through test automation, you may find this a good read. Automation testing is a concept that is heavily marketed today. You may ask, WHY? Well, the simple answer is – “Clients dig Automation.” Let me list down your common problem and how you believe test automation is your panacea: While these beliefs aren’t entirely false, they hide some common misconceptions. It is, in fact, these incorrect assumptions that result in the failure of test automation projects. Let us enumerate the tasks in test automation and map these tasks to a human or to a machine (automation script): If you agree with the task distribution in the table above then you cannot deny the following misconceptions: “Test automation is a means to reduce cost,” is one of the 3 myths of test automation. To be honest, test automation cannot be looked at from the prism of profit/loss. The benefits of test automation: – reliability, scalability, reduced dependency, robustness, etc, are unquantifiable. You cannot put a number on them. Even the most notable critics of test automation like James Bach or Michael Bolton, would agree that test automation has unquantifiable and innumerable benefits. Just that you can’t measure all these benefits and you can’t map them to cost savings. “With Test automation, I can replace manual testing” Nothing can be farther from the truth. As the table above shows, test automation supplements a manual tester. It does not replace the manual tester. Let me quote Jonathan Kohl (famous software scientist) from his interview with DZone: Kohl: “Think of all the things people can do well that machines can’t do well. Machines can’t feel, they can’t have a hunch, they can’t be suspicious, they can’t investigate, and they can’t change their minds due to better information. I don’t see automated testing throwing manual testing aside. And I have seen a lot of efforts fail when people have tried to do that.” “Test automation will improve my product quality.” This may not be a complete misstatement, but it is a myth. A machine can find a problem if it is programmed to find it, and it can do it without any intervention, efficiently, and with reliability. A human will be inefficient and unreliable. But a human being can do what a machine can’t; it can ideate and anticipate. A human being can find a problem, which was not thought of in the product specification (which is actually the true purpose of testing). If you really care about product quality, give the tester all the automation tools and let him ideate and anticipate. This is the value proposition offered by Test Automation! Need help? Contact us now!
How to Select the Right Test Automation Tool
Test Automation tool selection is a headache. But fear not, we have the cure for your ailment. At the outset let me congratulate you. By choosing to go for Test Automation you have made a very judicious decision. But if you are still considering Test Automation, you may want to read about: The benefits of Test Automation And I recommend the below, for the confused lot: If they made the correct decision? Now that you have made up your mind on Test Automation, the next step is to decide on the tool. Test Automation tool selection criteria 1. Cost Cost for obvious reasons is the first concern. There are many tools in the market. But not all tools are within your budget. Small organizations are generally not able to afford costly tools like UFT (from HP). Hence it is important for you to make a budget and shortlist the tools that fit your budget. It also makes sense to do this exercise first. This will save you unnecessary effort in analyzing tools that are beyond your budget. 2. Community Support Support by a community of developers is an important Test Automation tool selection criterion. Essential for organizations that are just starting out with Automation. They would need support in setting up the tool, remedy for any glitches, patches for bugs, and training on usage. Also, an organization can get a lot of help with usage if the tool has a large user base. A large user base ensures forums and vibrant discussion on the tool. The other way to ensure support is to buy from a recognized & established vendor (for example HP). 3. Script Creation time The major overhead of automation is the time spent on writing the script. So it is important to choose a tool that doesn’t make your Automation ROI negative. How can that happen? Well, let me give you an example. Suppose it takes the manual testing team around 100 hours to complete the testing of your software. Now you introduce Automation testing with an expectation to cut down your manual testing time by 50%. So you have saved 50 hours. Congratulations! But hang on. Did you realize that you may actually have spent 60 hours writing the script for the automation? So let me recalculate your effort and make it clear to you: Time spent on testing before automation = 100 hours Time spent on testing post-implementing automation = 50 hours (manual) + 60 hours (scripting) = 110 hours. My friend, you have a negative ROI for Test Automation. So you go back and buy the tool (price $ 3000) which makes scripting a breeze for you. But did you take into account the cost of the tool itself? Let me recalculate it for you: Cost of testing before automation = $ 4000 (100 hours @ $ 40 per hour) Cost of testing after automation = $ 2000 (50 hours of manual @ $ 40 per hour) + $ 3000 (tool). Net cost $ 5000. Once again your ROI is negative. It is therefore essential to consider both the script creation time and tool cost in conjunction before you purchase your test automation tool. 4. Scripting Language Well, I hate to break it to you. You can’t buy a tool and expect to script in the programming language of your choice. So if you aren’t proficient in all the programming languages, (.NET, Java, Python, Ruby, etc) which is highly likely (to this date I haven’t met a programmer proficient in all languages), I suggest you pick the tool which allows scripting in the language of your choice. 5. Handling UI Changes If you are testing the UI of your application, the ability of the automation tool to handle UI change is an important criterion for selection. Not all tools are equally capable of handling UI changes. It would make sense to analyze the complexity of your software’s UI. For example: if you are developing an analytics dashboard. You are likely to have many widgets. It would, therefore, make sense for you to choose an automation tool that can handle complex UI changes. Have questions about Test Automation? Contact the software testing experts at InApp to learn more.
Test Automation ROI Calculator
Test Automation ROI calculation is difficult. There are so many unquantifiable benefits to test automation. It is almost impossible to map these benefits to direct dollar savings. However, in order to make a business decision, it is important to know the test automation ROI. ROI is calculated by taking the benefits from the business decision and then dividing them by the cost of implementing that decision. Test Automation ROI can similarly be considered as a ratio between the benefits of testing and the cost of testing. The cost of testing is inversely correlated with the extent of automation. But this cost-saving happens over the years. In the interim, automation actually adds to the cost. The Test Automation ROI calculation formula is: (Saving on manual testing + Benefits from Test Automation) / (Cost of implementing Automation) Saving on manual testing can be calculated by using: The volume of Test Cases– The number of manual test cases that can be moved to automation has a direct impact on the time saved by automation. This also has a direct impact on test coverage. Number of Releases per year – Each time a product version is released, there is a substantial manual effort in testing the product. This can be cut down by using automation. Regression cycle per year– Regression tests are the ideal candidates for automation and are vital in calculating ROI. Number of browsers, devices (mobile, tablets), and configurations. Cost of manual tester. The cost of Implementing Automation can be calculated from Tool cost– Not all tools are free. Some like Selenium, Protractor, etc are open source, but a tool like Test Complete, and UFT adds substantially to the cost of implementing automation. The initial setup cost includes time spent on developing the automation frameworks, reusable components, and batch scripts. Cost of Automation Tester The benefits from Test Automation are the most unquantifiable. In order to calculate them the following should be kept in mind: To really understand the benefits of test automation, we need to see how and where all the automated tests can be utilized. It could be part of the verification process used by developers to ensure they have not introduced regression issues; integrated with the build tool as a post-build job to trigger the automation test runs on every build; besides obviously being used by the QA team as part of their testing for every release. Understanding the focus of automation with respect to maximizing test coverage, reduction in testing cycle time, reaching the market faster, etc The strategic direction of the organization To calculate your Test Automation ROI. Click on the link below: Give us your contact details here and we will collect the required data from you and calculate the ROI of Test Automation!