This is an integration testing involving both Top-Down and Bottom-Up techniques together. This is also called Hybrid Testing. This gives a comprehensive coverage in the integration testing.
It is a quick and brief evaluation of major functional elements of a piece of software to determine if it is basically operational and software environment as a whole is stable enough to proceed with extensive testing.
Scalability Testing is a non-functional test intended to test one of the software quality attributes i.e. “Scalability”. It is a Performance testing focused on ensuring the application under test gracefully handles increases in work load. It is focused on performance of software as a whole. Scalability testing is usually done by performance engineering team.
Objective of scalability testing is to test the ability of the software to scale up with increased users, increased transactions, increase in database size etc. It is not necessary that software’s performance increases with increase in hardware configuration, scalability tests helps to find out how much more workload the software can support with expanding user base, transactions, data storage etc.
Security testing involves the testing of Software in order to identify any flaws and gaps from security and vulnerability point of view. Security Testing is generally carried out by specialized team of software testers. Objective of security testing is to secure the software from external or internal threats from humans and malicious programs. Following are the main aspects which Security testing should ensure:
A security measure that protects against the disclosure of information to parties other than the intended recipient that is by no means the only way of ensuring the security.
A measure intended to allow the receiver to determine that the information provided by the system is correct. In addition to Confidentiality, this involves additional information that requires algorithmic checks rather than encoding the information.
This involves confirmation of identity of a person by tracing to the origins of artifacts ensuring that an information requested is from a trusted computer/program.
Access control is an example of authorization. This is the process of determining that a requester is allowed to receive a service or allowed to perform an operation.
Assuring the information and services are ready for use and kept available to authorized users as and when they need it.
Cross-site Scripting (XSS)
Cross-site scripting uses known vulnerabilities in web-based applications, their servers, or plug-in systems on which they rely. Exploiting one of these and by finding ways of injecting malicious scripts into web pages, an attacker can gain elevated access-privileges to sensitive page content, session cookies, and a variety of other information maintained by the browser on behalf of the user. Cross-site scripting attacks are therefore a special case of code injection.
There are 3 types of XSS – Stored, Reflected and DOM-based.
The Stored XSS vulnerability is the most powerful kind of XSS attack. A Stored XSS vulnerability exists when data provided to a web application by a user is first stored persistently on the server (in a database, filesystem, or other location), and later displayed to users in a web page without being encoded using HTML entity encoding.
A real life example of this would be the Samy MySpace Worm found on MySpace in October of 2005.
These vulnerabilities are the most significant of the XSS types because an attacker can inject the script just once. This could potentially hit a large number of other users with little need for social engineering, or the web application could even be infected by a cross-site scripting virus.
The Reflected XSS vulnerability is by far the most common and well-known type. These holes show up when data provided by a web client is used immediately by server-side scripts to generate a page of results for that user. If unvalidated user-supplied data is included in the resulting page without HTML encoding, this will allow client-side code to be injected into the dynamic page.
A classic example of this is in site search engines: if one searches for a string which includes some HTML special characters, often the search string will be redisplayed on the result page to indicate what was searched for, or will at least include the search terms in the text box for easier editing.
If all occurrences of the search terms are not HTML entity encoded, an XSS hole will result.
For example, if an attacker hosts a malicious website which contains a link to a vulnerable page on a client’s local system, a script could be injected and would run with privileges of that user’s browser on their system. This bye-passes the entire client-side sandbox, not just the cross-domain restrictions that are normally bye-passed with XSS exploits.
This is a way to ensure that a message transferred has been sent and received by the parties claiming to have sent and received the message by creating a handshake between the parties through dispatch and delivery acknowledgements.
From a digital security perspective, this guarantees that the message sender cannot later deny having sent the message and that the recipient cannot deny having received the message.
This testing confirms that the program can restrict access to authorized personnel and that the authorized personnel can access the functions available to their security level. Security testing basically checks, how good the software’s authorization mechanism is, how strong the authentication is, how the confidentiality and integrity of the data is maintained, what is the availability of the software in an event of malicious attack by hackers and malicious programs. Security testing requires good knowledge of application, technology, networking, security testing tools.
This is also used to discover potential vulnerabilities through version detection that may highlight deprecated versions of software or firmware.
Performing a periodic Security / Risk Audit and Review can be of great help to identify the gaps and resolve the security or risk-based issues.
See Penetration Testing and Vulnerability Testing also.
Server testing is primarily stress-oriented testing that include client/server I/O, network stress, CPU consumption, and memory consumption. The specific tests you must run depends on the features that you implement on the server. Several kinds of stress tests get run against a server, including basic system functionality, system stress and shutdown/restart tests.
System Functionality Test – The system functionality tests are individual tests of the capabilities of the system. Some tests are run for every system, and some tests only run if the capability exists in the system.
System Stress Test – The System Stress Test consists of several server scenario workloads that operate from the user level address space that is applied to the system to exercise the system hardware, system-specific devices and drivers, network and storage adapters and drivers, and any filter drivers that might be part of the system configuration, such as multipath storage drivers, storage or file system filter drivers, or intermediate layer network drivers. The workloads applied are SQL I/O Simulation, Local Storage I/O, Disk Stress with Verification, Client-Server Storage I/O and Network Traffic. These workloads automatically scale to the number of network and storage adapters in the system that have connected clients or storage devices, respectively.
Shutdown / Restart Test – The server test also includes a shutdown and restart test. This test signals the system to shut down and restart. The test records the event log information related to shutting down and restarting the system, such as vetoes that prevent shutdown, the startup event, and any driver errors that are received after restarting the system. This test makes sure that all device drivers in the system comply with system shutdown, do not veto, and cleanly restart in the system without conflicting with other drivers.
Server Virtualization Validation (SVV) Test – Two kinds of virtualization tests are run against a server, including virtual machine functionality tests and SVV System functionality tests. The system can be a standalone server or a virtual machine. The Virtual Machine Functionality Tests are individual tests of the capabilities of the product’s virtual machine implementation. The SVV System functionality tests validate the functionality of the following of the virtual machine – Virtual PCI I/O, Virtual BIOS, Virtual Timers, Virtual Plug-n-Play functions.
Server systems might have additional functionality beyond that which is required for Server Certification. The additional features for which a system can test and qualify are as follows:
Fault Tolerant Test – To confirm the ability of a fault-tolerant system hardware, devices, and drivers to have a hardware failure and continue to operate without impacting clients that are connected to the server over on the network.
Power Management Test – To validate that the systems supports the CPU related feature flag, processor states, and other functionality needed for the Server to manage the power of the system.
In addition to the above, there are various other tests like Hardware Certification Kit Harness Tests, Boot/Secure-Boot/ReBoot Tests, Debug Capability Test, Recovery Test, Robustness Test, Disk Stress Test, Timer Tests, PCI Hardware Compliance Test, Plug-n-Play Tests (with and without I/O devices), USB related tests, DVD Drive Tests, Memory Tests, Stability Tests, Reliability Tests, Network Connectivity Tests, Wireless Connectivity Tests, Domain Controller Test, Utilization Tests, etc.
NOTE: The above said tests are commonly applicable for almost all types of servers. Depending upon the type of the server and its intended functionality additional specific tests are designed to meet the requirements.
A quick-and-dirty test that the major functions of a piece of software work. Originated in the hardware testing practice of turning on a new piece of hardware for the first time and considering it a success if it does not catch on fire.
Smoke testing is a type of testing that is carried out by software testers to check if the new build” build-version-and-release-number/build provided by development team is stable enough i.e., major functionality is working as expected in order to carry out further or detailed testing. Smoke testing is intended to find “show stopper” defects that can prevent testers from testing the application in detail. Smoke testing carried out for a build is also known as build verification test.
Soak Testing is a type of performance testing, wherein software running is subjected to high load over a prolonged duration of time. Soak testing may go on for few days or even for few weeks.
Soak testing is a type of testing that is conducted to find errors that result in deterioration of software performance with continued usage. Soak testing is extensively done for electronic devices, which are expected to run continuously for days or months or years without restarting or rebooting. With growing web applications soak testing has gained significant importance as web application availability is critical for sustaining and success of business.
For example, running several times with huge transactions in an entire day (or night) greater than expected in a busy day, to identify and performance problems that appear after a large number of transactions has been executed.
This testing is also called Aging or Longevity Testing.
A set of activities conducted with the intent of finding errors in software.
This is a type of performance testing performed by performance engineering team. Objective of spike testing is to check how software responds to workloads that are sent in very short span of time and which are not constant over period of time.
This is a non-functional test intended to test one of the software quality attributes i.e. “Stability”. Stability testing focuses on testing how stable software is when it is subject to loads at acceptable levels, peak loads, loads generated in spikes, with more volumes of data to be processed. Scalability testing will involve performing different types of performance tests like load testing, stress testing, spike testing, soak testing, spike testing etc.,
Analysis of a program carried out without executing the program. Static Testing is a form of testing wherein approaches like reviews and walkthroughs are employed to evaluate the correctness of the deliverable.
In static testing software code is not executed instead it is reviewed for syntax, commenting, naming convention, coding standards, size of the functions and methods etc.
Static testing usually has check lists against which deliverables are evaluated. Static testing can be applied for requirements, design, test case, user manual, installation documents by using approaches like reviews or walkthroughs.
Testing that verifies the program under test stores data files in the correct directories and that it reserves sufficient space to prevent unexpected termination resulting from lack of space. This is external storage as opposed to internal storage.
Stress Testing is a type of performance testing, in which software is subjected to peak loads and even to a break point to observe how the software would behave at breakpoint. Testing conducted to evaluate a system or component at or beyond the limits of its specified requirements to determine the load under which it fails and how. A graceful degradation under load leading to non-catastrophic failure is the desired result. Often Stress Testing is performed using the same process as Load Testing but employing a very high level of simulated load.
Stress testing also tests the behavior of the software with insufficient resources like CPU, Memory, Network bandwidth, Disk space etc. Stress testing enables to check some of the quality attributes like robustness and reliability.
This testing type includes the testing of Software behavior under abnormal conditions. Taking away the resources, applying load beyond the actual load limit is Stress testing.
The main intent is to test the Software by applying the load to the system and taking over the resources used by the Software to identify the breaking point. This testing can be performed by testing different scenarios such as:
- Shutdown or restart of Network ports randomly.
- Turning the database on or off.
- Running different processes that consume resources such as CPU, Memory, server etc.
Testing based on an analysis of internal workings and structure of a piece of software.
Once all the components are integrated, the application as a whole is tested rigorously to see that it meets Quality Standards. This testing attempts to discover defects of the entire system rather than of its individual components.
System Testing includes multiple software testing types that will enable to validate the software system as a whole
(software, hardware and network) against the requirements for which it was built. Different types of tests (GUI testing, Functional testing, Regression testing, Smoke testing, load testing, stress testing, security testing, stress testing, ad-hoc testing etc.,) are carried out to complete system testing” system-testing/system testing.
System testing is so important because of the following reasons:
System Testing is the first step in the Software Development Life Cycle, where the application is tested as a whole.
The application is tested thoroughly to verify that it meets the functional and technical specifications.
The application is tested in an environment which is very close to the production environment where the application will be deployed.
System Testing enables us to test, verify and validate both the business requirements as well as the Applications Architecture.
System Integration Testing (SIT):
This is a type of testing conducted by software testing team. As the name suggests, focus of System integration testing is to test for errors related to integration among different applications, services, third party vendor applications etc., As part of SIT, end-to-end scenarios are tested that would require software to interact (send or receive data) with other upstream or downstream applications, services, third party application calls etc.,