Cloud and QA Environments

Today, organizations are facing a lot of challenges associated with QA environments like unavailability of environments, lack of skills to manage environments, coordinating with multiple vendors who manage these environments, etc. These challenges inhibit the efficiency of QA teams, which eventually impacts the organization’s business. These inherent challenges, along with the cloud evolution, have been catalysts in driving organizations to explore possible cloud adoption for the creation of QA environments. Senior managers will learn about the challenges faced by organizations with regards to traditional QA environments, and the possible benefits that cloud adoption could bring to this space.

Challenges with traditional QA environments

Shared QA environments

Limited hardware assets leads to sharing of resources across different QA teams. The sharing of environments includes applications, middleware, databases, specialized software and testing tools between different groups, leading to delays in testing due to high levels of inter-dependencies and different work priorities amongst the different teams. Shared environments force non-functional testing like performance testing to be carried out in a scaled down QA environment, which lacks the exact simulation of business requirements.

Mismatch and unavailability of infrastructure

Testing often happens in QA environments where the underlying infrastructure does not comply with the recommended hardware configurations. Even if it did, QA environments are often unavailable due to routine infrastructure maintenance activities. The unavailability of QA environments within an organization forces teams to explore external service providers who provision infrastructure, usually with a lock-in period. This obviously increases the cost of the corresponding project, driving down ROI significantly.

Lack of a standard methodology for building QA environments

Non-availability of a cohesive method that includes building, using and managing test environments, significantly restrains the ability of QA teams to respond to business units that need QA environments. Further, non-standard practices lead to multiple cycles for problem identification, analysis and refinement of QA environments.

Lack of skills to manage QA environment

Lack of skilled teams to manage and maintain QA environments will lead to significant effort being spent in setting up QA environments. Skilled resources are expensive, which increases costs and makes it hard to procure.

No centralized team to manage QA environments

Organizations end up having isolated teams which support different QA environments. In certain scenarios, application development teams manage environment and data, while in others it is managed by infrastructure teams. Different teams follow their own processes for environment management. This leads to a lack of single ownership for all QA environments.

High level of multi-vendor coordination

To avail various testing services, test infrastructure tools, hardware, etc., organizations end up dealing with multiple vendors. Dealing with multiple vendors involves setting up stringent operational level agreements (OLA) in order to ensure delivery of the application, on-time and within acceptable costs.

Multi-vendor engagements often require close monitoring of the progress on projects with multiple vendor coordination involving leased environments, leased testing tools, etc. Besides the coordination challenges that get thrown up by this model, the burden on “management time” is immense.

QA methodology remains stagnant as technology evolves

Organizations fail to revise their QA methodologies to match technology evolutions like SOA and cloud computing. This results in validation gaps which lead to defect prone applications going live. With the evolution of technology, more applications are migrating to or are being built on newer technologies like cloud and SOA, which necessitates the need for the organization to have a robust QA methodology in place to test these new-age applications.

Is the cloud the solution?

A separate QA environment dedicated to application validation can address most of the challenges stated above. In the current traditional models, organizations end up owning many hardware and software assets or leasing additional infrastructure from external infrastructure service providers. Both these options increase the CAPEX of the organization which does not go down well in the current economic conditions with decreasing spends in IT. Traditional QA environments involve procurement and leasing which further delays application go-live due to factors like procurement lead time, contract negotiation with external infrastructure provisioning.

Businesses can find means to effectively address QA environment concerns with the adoption of the cloud. In the cloud, organizations will benefit from features like demand provisioning, elasticity, resource sharing, availability and security. Organizations will also be able to move from traditional CAPEX models to OPEX models, leveraging the on-demand and pay-per-use model of computing resources for their testing and QA infrastructure needs. This would result in significant cost savings for the businesses. The pay-per-use model can also help organizations reduce the maintenance overhead and help them focus more on their business rather than spending effort/management time over environment procurement/leasing, environment management and infrastructure vendor management.

With the adoption of the cloud, organizations would also be able to effortlessly bring in centralization of QA, thus putting an end to issues arising due to the lack of standard methodologies for building QA environments and non-availability of skilled resources.

The QA environmental need is the perfect opportunity for organizations to begin their cloud adoption journey before making any decision on moving applications to the cloud. Leveraging cloud for the QA environment needs will help organizations address their test environment challenges as well as help them achieve benefits like shorter release cycles, business flexibility and better business service levels. Businesses need to ensure that they have all the necessary roadmaps and knowledge to help them transform their traditional test infrastructure to a cloud-based test infrastructure.

The current dynamic needs of business and volatile economy have led to organizations demanding CIOs to meet increasing business demands successfully, with shrinking budget allocations. Every CIO has started focusing and analyzing each and every IT division operation, significant investments made and subsequent ROI generated. One of the most significant elements that gets noticed and scrutinized is the QA infrastructure cost. This is because nearly 30 to 50% of servers in the organization are utilized by the QA teams, according to Cloud computing: Innovative solutions for test environments by IBM Global Services. Hence if these assets are underutilized, the investments in them are also underutilized, significantly impacting ROI.

The evolution of cloud has made organizations sit up and start thinking about how they can leverage the advantages of cloud as an infrastructure or as a platform or even as software, to overcome the challenges of today’s dynamic business and IT needs. In Cloud computing: Addressing software QA issues, we discussed the challenges associated with traditional QA environments and how cloud was a solution to overcome these challenges. In this post, we share an in-depth analysis of the various factors and explain the benefits that make QA environment the perfect place for CIOs to begin cloud adoption.

Use case evaluation for cloud adoption

Infosys recently embarked on research which evaluated the popular cloud use cases against parameters like business risk, business value, relative simplicity and cloud technology maturity for cloud adoption. The analysis covers the following cloud use cases in the forms of cloud as a software, platform and infrastructure:

SaaS (Software as a Service): Online collaboration solutions, enterprise applications and    business/industry applications

PaaS (Platform as a Service): Web 2.0 applications, databases and middleware

IaaS (Infrastructure as a Service): Storage, server and networks, production custom applications and QA/DEV environments

The table below rates the typical cloud cases as High (H), Medium (M) or Low (L) against each parameter – business risk, business value, relative simplicity and cloud technology maturity.

By evaluating the ratings for each parameter, we will be able to deduce the most optimal use case for cloud adoption, from an overall perspective.

Table 1: The use case evaluation for cloud adoption

Business risk

The business risk associated with migrating live applications into the cloud right away is quite high for organizations. If a failure occurs, it would have a direct and immediate impact on the business of the organization. However, cloud adoption with QA environments would be far more appropriate for organizations as the business risk associated with such an adoption was found to be comparatively lower when compared with other cloud use cases.

Business value

It is quite evident that business value of cloud adoption is quite high with the SaaS model when it covers enterprise applications such as CRM, ERP, etc. Organizations usually stand to gain immediately with SaaS as they get a ready-to-go market solution with a very short turnaround time. However, as per the table (Table 1), organizations also gain significantly with the SaaS model when leveraged in the form of cloud for their QA environment needs. This is because of the increased asset utilization, reduced proliferation, greater serviceability and greater agility with provisioning that cloud is able to provide for QA/Dev environments.

Relative simplicity or ease of implementation

SaaS and PaaS use cases require integration; secure authentication and secure policy enforcement which increases the complications during implementations. However, cloud adoption in QA environments stands out due to the relative ease in implementation.

Cloud technology maturity

Cloud technology has high maturity levels across:

  • SaaS in the form of online collaboration solutions
  • PaaS in the form of Web 2.0 applications and databases
  • IaaS in the form of storage, server and networks and QA environments

The evolution of SalesForce CRM (form of SaaS), Windows Azure (form of PaaS), Amazon EC2 (form of IaaS) implies that cloud technology is mature because of the dynamic convergence of information technology, business model and consumer experience. These use cases are a good place to begin cloud adoption from a cloud technology maturity perspective.

Overall recommendation

The one use case that stands out distinctly and strongly, across all parameters, is the adoption of cloud in QA/Dev environments. Advantages such as increased asset utilization, reduced proliferation, greater agility in servicing requests and faster release cycle times, position QA environments as the most optimal use case for cloud adoption from an overall recommendation standpoint.

Benefits delivered by cloud-based QA environments

Let us now look into the key benefits delivered by cloud-based QA environments:

Dynamic and scalable provisioning

With cloud-based QA environments, organizations can quickly provision/de-provision virtual machines on demand, drastically reducing the provisioning time from several months to a few minutes. This ability to scale gives organizations an edge with high quality services and diverse QA environment requirements. It also helps business focus on core areas, by reducing the time spent on procurement operations.

Reduced time to market

Test cycles have always been seen as critical paths for release to production. Cloud adoption in a QA environment facilitates faster on-demand provisioning of resources, increase in productivity and shorter lifecycles for application development and testing, which significantly contributes to faster time to market. Interestingly in traditional QA environments, 30% of defects in production phase for an application were primarily due to wrongly configured test environments. Cloud eliminates this, thus reducing time to market immediately.

Greater environment control

With the cloud adoption for QA environments, multiple channels requesting QA environments for various projects are consolidated into a single channel, significantly reducing server and application sprawl. This leads to a better control over the environment.

Reduced TCO and improved resource utilization

The capability to share environments due to virtualization improves resource utilization, thus reducing associated costs of hardware and software licenses. Cloud-based QA environments bring in significant cost savings of almost 50% on IT support costs, helping organizations move from a CAPEX to an OPEX mode.

Conclusion

There is no doubt that the QA environment is an apt place for organizations to begin their cloud adoption journey. It is recommended that organizations explore and evaluate their internal QA environment infrastructure for conversion into a secure private enterprise cloud. In the event of unavailability of internal infrastructure, organizations need to partner and engage with an external cloud service provider, with the ability to provide the infrastructure for QA as a service in a pay-as-you-use model. Cost savings achieved through such infrastructure optimization can be reinvested by the organization into core business projects to bring in the much needed innovation to drive overall enterprise sustainability and market relevance.

Determining the cloud model that best meets your business requirements.

Once organizations have made the decision to take the cloud route for their QA requirements, the next challenge in store is determining the right cloud deployment model which is suitable for their business needs and size. In this third part of our three-part series, we will give decision makers the information they need to evaluate cloud deployment models and choose the ideal model for their organization.

Evaluating your existing QA infrastructure

To properly make a decision about cloud models, an in-depth understanding and evaluation of the existing QA infrastructure against the following parameters is required.

QA infrastructure requirements:

An organization’s demand for QA infrastructure depends on all application requirements, environment needs for different types of testing, the duration of the testing cycles and the frequency of testing in a given calendar year.

Current QA infrastructure availability:

It is recommended that the organization gauges its existing QA infrastructure assets and makes an inventory of all the related hardware and software assets. Then, evaluate the needs and see if the current demand for QA infrastructure can be met with what’s available.

Availability of budget:

It’s important to assess whether an organization is keen on moving from a CAPEX to an OPEX mode for their QA environments, the willingness to allocate budget for investments in the cloud and the budget amount. This factor would play a key role in determining the right cloud deployment model for the organization.

Application release calendar:

The demand for the QA infrastructure also depends on the release calendars for all applications in the organization which would also factor in the shared and dedicated QA environments for some applications.

Evaluation and applicability scenarios of cloud deployment models

After evaluating the current QA infrastructure, look into scenarios that would ideally fit each cloud deployment model. Let’s review the pros, cons and applicability scenarios and organizations ideally deemed fit for each cloud deployment model. There are primarily four kinds of widespread cloud deployment models to explore from an infrastructure perspective, which include private, public, virtual private and the hybrid cloud.

Enterprise private cloud

The enterprise private cloud is essentially a cloud resource pool that is within an organization’s network and firewall. They are created from already-owned and existing hardware and software assets.

Pros

  • Optimal utilization of an organization’s existing assets.
  • On-demand provisioning that can be customized to the QA infrastructure needs.
  • Higher security and compliance with regulations and standards since the cloud is setup within the organization’s firewall.
  • The organization can use the time and resources saved from managing the environment, on more important and core business activities.

Cons

  • Additional CAPEX would be required to setup a private cloud along with an investment in hardware assets and tools needed for automating cloud provisioning and managing services.

Applicability scenarios

The enterprise private cloud would be deemed fit where organizations:

  • Already have adequate hardware which suffices their current QA infrastructure needs and is underutilized.
  • Would be able to manage future QA infrastructure demands and accommodate all application release cycles with the current availability.
  • Are willing to invest in virtualization, cloud management software, SAN storage if needed and server class machines to manage the cloud resource pool.

Ideal for:

  • Large organizations that have an underutilized QA infrastructure.
  • Small and medium-sized organizations that lack QA infrastructure assets and need them for a longer duration.

Public cloud

A public cloud is a cloud deployment model where the cloud resource pool is outside the organization’s firewall and built using a cloud service provider’s hardware and software assets.

Pros

  • On-demand provisioning with no CAPEX involved.
  • No vendor lock-in concerns. 
  • No resources are required to manage the public cloud since the cloud service vendor takes care of the same.

Cons

  • Concerns on data privacy, security and compliance with regulations and standards.

Applicability scenarios

The public cloud would be deemed fit where organizations:

  • Do not own any QA infrastructure related hardware assets.
  • Have no intent to make an investment in its QA infrastructure.
  • Is short on resources to manage its QA environments.

Ideal for:

  • Small and medium-sized organizations that do not own any QA infrastructure and have short- term testing requirements.

Virtual private cloud

Virtual private clouds are third party public clouds or segments of public cloud that have additional features for security and compliance.

Pros

  • On-demand provisioning with no CAPEX involved.
  • No resources are required to manage the public cloud since the cloud service vendor takes care of it.
  • No vendor lock-in concerns. 
  • Compliance with data security, privacy, standards and regulations is possible by using public cloud instances. These public cloud instances are not shared with other organizations who are also cloud service subscribers with the same vendor.

Cons

  • Additional compliances like the SAS 70 validation would be needed from the cloud service provider.

Applicability scenarios

The virtual private cloud would be deemed fit where organizations:

  • Do not own any QA infrastructure related hardware assets.
  • Do not want to make a CAPEX investment for their QA infrastructure.
  • Are short on resources for managing their QA environments.
  • Have the prime responsibility of complying with standards, regulations, data privacy and security.

Ideal for:

  • Organizations of all sizes that do not own any QA-related infrastructure assets but have short term testing requirements, require security and compliance to standards.

Hybrid cloud

Hybrid cloud is a combination of two or more cloud deployment models (which includes private cloud, public cloud and virtual private cloud).

Pros

  • Improved utilization of an organization’s existing assets.
  • On-demand provisioning can be customized to the QA infrastructure needs of the organization.
  • All long-term QA environment needs can be managed with the private cloud and short-term/ sporadic QA environment needs that cannot be accommodated with the existing asset can be managed in a public cloud without any additional CAPEX.
  • Data security, privacy, standards and regulations can be complied with, by using private cloud instances and non-critical application testing can be moved into public clouds.

Cons

  • Integration between public and private clouds can be a challenge when applications from these types of cloud deployment models need to interact with each other for simulating end-to-end testing scenarios.

Applicability scenario

The hybrid cloud would be deemed fit where organizations:

  • Own hardware assets related to QA infrastructure to a considerable extent.
  • Are willing to invest in virtualization, cloud management software, if needed in SAN storage and server class machine for managing the cloud resource pool.
  • Have standards, regulations, data privacy and security requirements to comply with.
  • Cannot completely manage the future QA infrastructure demands with their available hardware assets.

Ideal for:

  • Large organizations that have the capability to handle majority of their long-term QA infrastructure needs within their own private clouds and have certain short-term/sporadic QA infrastructure needs which can be handled in a public cloud.

Businesses of all sizes can begin their cloud adoption journey with QA environments with a suitable cloud deployment model. It is clearly evident from the evaluation of the different forms of cloud, that while adopting them, long-term resources need to be moved to private clouds, while the short-term and sporadic resources to public clouds in a pay-as-you-use mode, which would help in achieving an effective ROI.