SOA testing permits to test SOAP, XML and REST based messaging which prevents a service endpoint in order to assess the robustness and resilience of the service. Normally testing happening in the below four primary areas.
Functional testing:-Functional testing helps to verify the behavior of the services and build the regression test to automate testing and baseline expected behavior of services to quickly assess and validate functionality through the lifecycle of service revisions. The functional testing is often first in the SOA development lifecycle, and adopted early-on in the development and implementation phase. The functional testing becomes widely adopted by development teams both because it is free and because the uses cases are often limited to simple unit testing of service messaging.
Performance testing:- It provide the concurrent loading agent framework which can determine throughput and capacity statics of the back end services through the range of input and client load variances to validate Service Level Agreement rates and helps to identify bottlenecks and potential architectural weaknesses and performance dependencies.
Interoperability testing:- It maximizes the interoperability by evaluating both design characteristics of a services and runtime adherence to standards and best practices.
Security testing:- It assesses the risk and reliability of a service with regard to vulnerability, data-leakage, data privacy, and data integrity. For each web service is unique which based on a particular schema and this schema defines the input and response message structure. This structure defines how to communicate with the services. The various security and identity specifications set forth by the W3C and OASIS includes a framework to test for the level of data integrity, data privacy, and access control on the service transactions and endpoint itself.
Governance testing: – It refers to the Standards and Policies that govern the design, build and implementation of a SOA solution and the Policies that must be enforced during runtime.
Example of SOA governance policy types:-
1) Quality of Service policies on Performance, Security and Transactions
2) Regulatory policies – Sarbanes-Oxley
3) Business policies – rules
4) Audit policies – what events need to be logged, how long must events be kept, etc
5) Infrastructure policies – access, backups, disaster recovery and failover.
Service component level testing :- Service-component-level testing or Unit testing is to take the smallest piece of testable software in the application, isolate it from the remainder of the code, and determine whether it behaves exactly as the expected one. Each Component is tested separately before integrating it into a service or services.
1) Formal peer reviews of the code and it complies with organization standards and to identify any potential performance and security defects or weaknesses.
2) Quality entry and exit are not specify for this level of testing but are achieved before moving to the next level of testing.
Service level testing:- In service reuse each service is delivered from service level/phase of testing with a comprehensive statement of quality and even a Guarantee.
1) In the formal peer reviews of the code helps to complies with organization standards and to identify any potential interoperability, performance and security defects or weaknesses.
2) Functional, performance and security regression suites to be executed against the service. This will need to the help of automated test tools and the development of sophisticated harnesses and stubs.
3) Quality entry and exit criteria are not only defined for this level of testing, but are achieved before delivering the service to the next level of testing.
SOA is loosely coupled with complex interdependencies and a SOA testing approach must follow the same pattern.
Integration level testing:- The integration test phase will ensure the service interfaces.
Process or Orchestration level testing:- It ensures services are operating collectively as specified.
System level testing:- This phase is also known as user acceptance test phase. It will test the SOA technical solution has delivered the defined business requirements and defined business acceptance criteria.
SOA testing challenges
1) Many services will not have a user interface that will need a new breed of tools to assist with the testing.
2) Infinite ‘service’ consumers and users are possible.
4) SOA can be boundless. Services may be used by applications yet to be developed or by consumers outside the enterprise.
5) ‘Loosely-coupled’ connections that permits unforeseen applications to take advantage of ever-expanding capabilities.
6) SOA is driven by business processes that not only cross technologies but span organizations. Test teams will helps to provide a broader set of domain and technical knowledge.
7) Data Driven Testing (No GUI)
8) Difficulty in isolating the bugs due to presence of multiple layers
9) Difficult to test Secure Web Services; especially those requiring encrypted and signed test data
10) Lack of effective reporting and debugging capabilities in available tools, that is essential for Web Services
11) Increased number of integration and interoperability because of multiple service involvement and communication among them
12) Involvement of 3rd party services in end-to-end solution
13) Different services may have different authorization / authentication approach
14) Performance testing problems due to:
1) Small service large overhead
2) Service required specific hardware specification
3) Distributed service may have network latency
SOA test phases and test types