test automation Archives

acceptance testing, agile, AI, BDD, bot, ci/cd, container, continues integration, development, devops, digital, speed, Technique, Tools, הראל, הראל לירם, לירם, לירם הראל

7 tips for improving load speed

Plan for performance

Are you building a new website? Be sure to discuss the importance of performance early on and set targets. That way, you have a faster load speed from the beginning and don’t have to implement fixes later.

Step 1: test, step 2: test…

Are you seeing a pattern here? 😉 Testing is crucial! Before you launch, load and test your website multiple times to make sure you can handle the traffic of real site visitors. This is especially important for sites with complex hosting, such as load-balanced configuration.

Implement some “quick wins”

To be clear, there’s no “get fast quick” scheme for site load speeds. But there is a tried-and-true template that will put you ahead of the curve. That includes making use of modern image formats, enabling compression on the server via Gzip, and leveraging browser cache. Find some more low-hanging fruit here.

Careful of your images!

Good websites have great graphic content – but they also take into account how images impact load speed. You can improve image performance by considering file formats, image compression, and lazy loading.

Think of the mobile visitors

More and more people surf the web on their phone these days, which makes mobile-optimized sites a huge priority! Since mobile users tend to use slower, less stable Internet connections, Accelerated Mobile Pages (AMPs) are a great way to get them content faster.

Prioritize above-the-fold

First impressions matter – and your above-the-fold content can make or break them! Consider inline styling for above-the-fold, then loading your code in chunks. This type of asynchronous loading can create a faster perceived load time for the user.

Assess your external scripts

Third-party scripts are a great tool – but can make your website feel a little crowded. Assess the performance of external scripts on your site load speed, and replace or remove those that are negatively impacting user experience.

Technique, Tools

This way you will get a complete and clear picture of the QA findings

Smart integration of the worlds of manual, automated and mass testing is a weakness in many companies, and has a direct impact on the decision whether to release a product or version to market or return it for further work. A new approach offers a holistic solution and optimization in the world of testing. A software company must have a good idea, great developers and creative interface designers, but without strict and thorough software testers all work can go down the drain. The QA departments are responsible for ensuring that the software or application exits the company gates without any glitches, no matter who the user is, which operating system is installed on the device and in which language it is running; They are an important part of a chain that determines the customer experience, and accordingly the success of the product and its compliance with consumer expectations.

Software testers perform the tests in a variety of methods and ways. Some are designed to make sure there are no glitches, some are usability tests, some are done manually by the company’s QA team, some are performed by masses (Crowd Testing) and some using automated tools. In order to enter the market with a perfect product, companies must perform a combination of tests: Test automation provides more reliable and faster results, enabling versions of applications to be released to the market more quickly. When manual tests are added to this, which are amplified with mass testing (Crows-testing), a picture is obtained that complements the coverage gaps of the automatic tests, helps to verify faults and provides a complete and reliable overview of the product quality.

Indeed, most companies and organizations rely on automated and manual testing in their testing strategy, but most of them run the tests simultaneously and separately and are not synchronized with each other, so that unnecessary and less quality tests are often performed. At the end of the process the QA managers need to concentrate all the feedback from all the tests into one place, and only then start working on improving and correcting the problems found.

leads to unnecessary and cumbersome work, waste of time and resources and the need for double budgeting. A little mess and disorder in the results obtained is enough, and the conclusions may harm the company’s business goals and the chances of success of the software or application. To date, QA departments have required a great deal of effort to manage the vast amount of information flowing from the variety of quality tests performed and to examine the information on many dashboards, in order to get a snapshot of the tests, their quality and results.

All the information in one place

results, the integrated testing approach has been developed, which allows for a holistic perspective on all test results and thus make decisions quickly and efficiently. This approach combines in one place all the information and results of all types of tests – understandable and incomprehensible, manual or automatic; Design tests, usability, accessibility and more. This way the QA teams have a look at and complete control over all the testing processes and all the results, and they can easily conduct an in-depth investigation of each problem.

An integrated solution can completely reduce the complexity and loads often created as a result of the multiplicity of testing platforms, and even makes it possible to grow and expand the work environment for all business units.

An integrated solution includes several components and features. The dashboard, the main screen in the system, shows the complete list of tests – manual or automatic – and whether they succeeded or failed. When this information appears in one place, it is easier to identify patterns of faults and the connections between different failed tests, so that managers can make a quicker and more informed decision about releasing the product or continuing to work on it.

The integrated approach seamlessly supports the CI / CD workflow process, and when a new software or version is ready for testing you can quickly create a new test cycle and see the results on the screen in real time.

First the results of the automatic tests appear, as they are faster, and then the manual tests and the mass tests. From there you can perform manual tests or repeat mass tests, in order to prevent False Negatives from the automatic tests, and the repeat test results are also updated on the dashboard. All test history is available at all times and can be used to understand trends and strengthen testing strategy, and test results and bugs can also be exported to other systems like JIRA.

Each company must conduct tests before the product arrives on the market, and each one makes its own considerations in choosing the types of tests. But testing management, which leads to making decisions that affect the product and the company, seems to be a weakness in many of them, a point that can be strengthened using tools that exist in the market and can save valuable time in all departments and management.

Tools

Tools Supporting for Testing

Test Tool Considerations

Test tools can be used to support one or more testing activities. Such tools include:

Tools that are directly used in testing, such as test execution tools and test data preparation tools
Tools that help to manage requirements, test cases, test procedures, automated test scripts, test results, test data, and defects, and for reporting and monitoring test execution
Tools that are used for analysis and evaluation
Any tool that assists in testing (a spreadsheet is also a test tool in this meaning)

Test Tool Classification

Test tools can have one or more of the following purposes depending on the context:

Improve the efficiency of test activities by automating repetitive tasks or tasks that require significant resources when done manually (e.g., test execution, regression testing)
Improve the efficiency of test activities by supporting manual test activities throughout the test process
Improve the quality of test activities by allowing for more consistent testing and a higher level of defect reproducibility
Automate activities that cannot be executed manually (e.g., large scale performance testing)
Increase reliability of testing (e.g., by automating large data comparisons or simulating behaviour)

Tools can be classified based on several criteria such as purpose, pricing, licensing model (e.g., commercial or open source), and technology used. Tools are classified in this article according to the test activities that they support.

Some tools clearly support only or mainly one activity; others may support more than one activity, but are classified under the activity with which they are most closely associated. Tools from a single provider, especially those that have been designed to work together, may be provided as an integrated suite.

Some types of test tools can be intrusive, which means that they may affect the actual outcome of the test. For example, the actual response times for an application may be different due to the extra instructions that are executed by a performance testing tool, or the amount of code coverage achieved may be distorted due to the use of a coverage tool. The consequence of using intrusive tools is called the probe effect.

Some tools offer support that is typically more appropriate for developers (e.g., tools that are used during component and integration testing). Such tools are marked with “(D)” in the sections below.

Tool support for management of testing and test-ware

Management tools may apply to any test activities over the entire software development lifecycle. Examples of tools that support management of testing and test-ware include:

Test management tools and application lifecycle management tools (ALM)
Requirements management tools (e.g., traceability to test objects)
Defect management tools
Configuration management tools
Continuous integration tools (D)

Tool support for static testing

Static testing tools are associated with the activities and benefits described in the static testing page. Examples of such tool include:

Static analysis tools (D)

Tool support for test design and implementation

Test design tools aid in the creation of maintainable work products in test design and implementation, including test cases, test procedures and test data. Examples of such tools include:

Model-Based testing tools
Test data preparation tools

In some cases, tools that support test design and implementation may also support test execution and logging, or provide their outputs directly to other tools that support test execution and logging.

Tool support for test execution and logging

Many tools exist to support and enhance test execution and logging activities. Examples of these tools include:

Test execution tools (e.g., to run regression tests)
Coverage tools (e.g., requirements coverage, code coverage (D))
Test harnesses (D)

Tool support for performance measurement and dynamic analysis

Performance measurement and dynamic analysis tools are essential in supporting performance and load testing activities, as these activities cannot effectively be done manually. Examples of these tools include:

Performance testing tools
Dynamic analysis tools (D)

Tool support for specialised testing needs

In addition to tools that support the general test process, there are many other tools that support more specific testing for non-functional characteristics.

Benefits and Risks of Test Automation

Simply acquiring a tool does not guarantee success. Each new tool introduced into an organisation will require effort to achieve real and lasting benefits. There are potential benefits and opportunities with the use of tools in testing, but there are also risks. This is particularly true of test execution tools (which is often referred to as test automation).

Potential benefits of using tools to support test execution include:

Reduction in repetitive manual work (e.g., running regression tests, environment set up/tear down tasks, re-entering the same test data, and checking against coding standards), thus saving time
Greater consistency and repeatability (e.g., test data is created in a coherent manner, tests are executed by a tool in the same order with the same frequency, and tests are consistently derived from requirements)
More objective assessment (e.g., static measures, coverage)
Easier access to information about testing (e.g., statistics and graphs about test progress, defect rates and performance)

Potential risks of using tools to support testing include:

Expectations for the tool may be unrealistic (including functionality and ease of use)
The time, cost and effort for the initial introduction of a tool may be under-estimated (including training and external expertise)
The time and effort needed to achieve significant and continuing benefits from the tool may be under-estimated (including the need for changes in the test process and continuous improvement in the way the tool is used)
The effort required to maintain the test work products generated by the tool may be under-estimated
The tool may be relied on too much (seen as a replacement for test design or execution, or the use of automated testing where manual testing would be better)
Version control of test work products may be neglected
Relationships and interoperability issues between critical tools may be neglected, such as requirements management tools, configuration management tools, defect management tools and tools from multiple vendors
The tool vendor may go out of business, retire the tool, or sell the tool to a different vendor
The vendor may provide a poor response for support, upgrades, and defect fixes
An open source project may be suspended
A new platform or technology may not be supported by the tool
There may be no clear ownership of the tool (e.g., for mentoring, updates, etc.)

Special Considerations for Test Execution and Test Management Tools

In order to have a smooth and successful implementation, there are a number of things that ought to be considered when selecting and integrating test execution and test management tools into an organisation.

Test execution tools

Test execution tools execute test objects using automated test scripts. This type of tools often requires significant effort in order to achieve significant benefits.

Capturing test approach: Capturing tests by recording the actions of a manual tester seems attractive, but this approach does not scale to large numbers of test scripts. A captured script is a linear representation with specific data and actions as part of each script. This type of script may be unstable when unexpected events occur, and require ongoing maintenance as the system’s user interface evolves over time.
Data-driven test approach: This test approach separates out the test inputs and expected results, usually into a spreadsheet, and uses a more generic test script that can read the input data and execute the same test script with different data.
Keyword-driven test approach: This test approach, a generic script processes keywords describing the actions to be taken (also called action words), which then calls keyword scripts to process the associated test data.

The above approaches require someone to have expertise in the scripting language (testers, developers or specialists in test automation). When using data-driven or keyword-driven test approaches testers who are not familiar with the scripting language can also contribute by creating test data and/or keywords for these predefined scripts. Regardless of the scripting technique used, the expected results for each test need to be compared to actual results from the test, either dynamically (while the test is running) or stored for later (post-execution) comparison.

Model-Based testing (MBT) tools enable a functional specification to be captured in the form of a model, such as an activity diagram. This task is generally performed by a system designer. The MBT tool interprets the model in order to create test case specifications which can then be saved in a test management tool and/or executed by a test execution tool.

Test management tools

Test management tools often need to interface with other tools or spreadsheets for various reasons, including:

To produce useful information in a format that fits the needs of the organisation
To maintain consistent traceability to requirements in a requirements management tool
To link with test object version information in the configuration management tool

This is particularly important to consider when using an integrated tool (e.g., Application Lifecycle Management), which includes a test management module, as well as other modules (e.g., project schedule and budget information) that are used by different groups within an organisation.

Effective Use of Tools

Main Principles for Tool Selection

The main considerations in selecting a tool for an organisation include:

Assessment of the maturity of the own organisation, its strengths and weaknesses
Identification of opportunities for an improved test process supported by tools
Understanding of the technologies used by the test object(s), in order to select a tool that is compatible with that technology
Understanding the build and continuous integration tools already in use within the organisation, in order to ensure tool compatibility and integration
Evaluation of the tool against a clear requirements and objective criteria
Consideration of whether or not the tool is available for a free trial period (and for how long)
Evaluation of the vendor (including training, support and commercial aspects) or support for non-commercial (e.g., open source) tools
Identification of internal requirements for coaching and mentoring in the use of the tool
Evaluation of training needs, considering the testing (and test automation) skills of those who will be working directly with the tool(s)
Consideration of pros and cons of various licensing models (e.g., commercial or open source)
Estimation of a cost-benefit ratio based on a concrete business case (if required)

As a final step, a proof-of-concept evaluation should be done to establish whether the tool performs effectively with the software under test and within the current infrastructure or, if necessary, to identify changes needed to that infrastructure to use the tool effectively.

Pilot Projects for Introducing a Tool into an Organisation

After completing the tool selection and a successful proof-of-concept, introducing the selected tool into an organisation generally starts with a pilot project, which has the following objectives:

Gaining in-depth knowledge about the tool, understanding both its strengths and weaknesses
Evaluating how the tool fits with existing processes and practices, and determining what would need to change
Deciding on standard ways of using, managing, storing, and maintaining the tool and the test work products (e.g., deciding on naming conventions for files and tests, selecting coding standards, creating libraries and defining the modularity of test suites)
Assessing whether the benefits will be achieved at reasonable cost
Understanding the metrics that you wish the tool to collect and report, and configuring the tool to ensure these metrics can be captured and reported

Success Factors for Tools

Success factors for evaluation, implementation, deployment, and on-going support of tools within an organisation include:

Rolling in the tool to the rest of the organisation incrementally
Adapting and improving processes to fit with the use of the tool
Providing training, coaching, and mentoring for tool users
Defining guidelines for the use of the tool (e.g., internal standards for automation)
Implementing a way to gather usage information from the actual use of the tool
Monitoring tool use and benefits
Providing support to the users of a given tool
Gathering lessons learned from all users

It is also important to ensure that the tool is technically and organisationally integrated into the software development lifecycle, which may involve separate organisations responsible for operations and/or third party suppliers.

agile

Agile Software Development

Basics of Agile Software Development

A tester on an Agile project will work differently than one working on a traditional project. Testers must understand the values and principles that underpin Agile projects, and how testers are an integral part of a whole-team approach together with developers and business representatives. The members in an Agile project communicate with each other early and frequently, which helps with removing defects early and developing a quality product.

Agile Software Development and the Agile Manifesto

In 2001, a group of individuals, representing the most widely used lightweight software development methodologies, agreed on a common set of values and principles which became known as the Manifesto for Agile Software Development or the Agile Manifesto [Agile-manifesto]. The Agile Manifesto contains four statements of values:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

The Agile Manifesto argues that although the concepts on the right have value, those on the left have greater value.

Individuals and Interactions

Agile development is very people-centred. Teams of people build software, and it is through continuous communication and interaction, rather than a reliance on tools or processes, that teams can work most effectively.

Working Software

From a customer perspective, working software is much more useful and valuable than overly detailed documentation and it provides an opportunity to give the development team rapid feedback. In addition, because working software, albeit with reduced functionality, is available much earlier in the development lifecycle, Agile development can confer significant time-to-market advantage. Agile development is, therefore, especially useful in rapidly changing business environments where the problems and/or solutions are unclear or where the business wishes to innovate in new problem domains.

Customer Collaboration

Customers often find great difficulty in specifying the system that they require. Collaborating directly with the customer improves the likelihood of understanding exactly what the customer requires. While having contracts with customers may be important, working in regular and close collaboration with them is likely to bring more success to the project.

Responding to Change

Change is inevitable in software projects. The environment in which the business operates, legislation, competitor activity, technology advances, and other factors can have major influences on the project and its objectives. These factors must be accommodated by the development process. As such, having flexibility in work practices to embrace change is more important than simply adhering rigidly to a plan.

Principles

The core Agile Manifesto values are captured in twelve principles

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
Deliver working software frequently, at intervals of between a few weeks to a few months, with a preference to the shorter timescale.
Business people and developers must work together daily throughout the project.
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
Continuous attention to technical excellence and good design enhances agility.
Simplicity—the art of maximising the amount of work not done—is essential.
The best architectures, requirements, and designs emerge from self-organising teams.
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.

The different Agile methodologies provide prescriptive practices to put these values and principles into action.

Whole-Team Approach

The whole-team approach means involving everyone with the knowledge and skills necessary to ensure project success. The team includes representatives from the customer and other business stakeholders who determine product features. The team should be relatively small; successful teams have been observed with as few as three people and as many as nine. Ideally, the whole team shares the same workspace, as co-location strongly facilitates communication and interaction. The whole-team approach is supported through the daily stand-up meetings involving all members of the team, where work progress is communicated and any impediments to progress are highlighted. The whole-team approach promotes more effective and efficient team dynamics.

The use of a whole-team approach to product development is one of the main benefits of Agile development. Its benefits include:

Enhancing communication and collaboration within the team
Enabling the various skill sets within the team to be leveraged to the benefit of the project
Making quality everyone’s responsibility

The whole team is responsible for quality in Agile projects. The essence of the whole-team approach lies in the testers, developers, and the business representatives working together in every step of the development process. Testers will work closely with both developers and business representatives to ensure that the desired quality levels are achieved. This includes supporting and collaborating with business representatives to help them create suitable acceptance tests, working with developers to agree on the testing strategy, and deciding on test automation approaches. Testers can thus transfer and extend testing knowledge to other team members and influence the development of the product.

The whole team is involved in any consultations or meetings in which product features are presented, analysed, or estimated. The concept of involving testers, developers, and business representatives in all feature discussions is known as the power of three.

Early and Frequent Feedback

Agile projects have short iterations enabling the project team to receive early and continuous feedback on product quality throughout the development lifecycle. One way to provide rapid feedback is by continuous integration.

When sequential development approaches are used, the customer often does not see the product until the project is nearly completed. At that point, it is often too late for the development team to effectively address any issues the customer may have. By getting frequent customer feedback as the project progresses, Agile teams can incorporate most new changes into the product development process. Early and frequent feedback helps the team focus on the features with the highest business value, or associated risk, and these are delivered to the customer first. It also helps manage the team better since the capability of the team is transparent to everyone. For example, how much work can we do in a sprint or iteration? What could help us go faster? What is preventing us from doing so?

The benefits of early and frequent feedback include:

Avoiding requirements misunderstandings, which may not have been detected until later in the development cycle when they are more expensive to fix.
Clarifying customer feature requests, making them available for customer use early. This way, the product better reflects what the customer wants.
Discovering (via continuous integration), isolating, and resolving quality problems early.
Providing information to the Agile team regarding its productivity and ability to deliver.
Promoting consistent project momentum.

Aspects of Agile Approaches

There are a number of Agile approaches in use by organisations. Common practices across most Agile organisations include collaborative user story creation, retrospectives, continuous integration, and planning for each iteration as well as for overall release. This subsection describes some of the Agile approaches.

Agile Software Development Approaches

There are several Agile approaches, each of which implements the values and principles of the Agile Manifesto in different ways. In this article , three representatives of Agile approaches are considered: Extreme Programming (XP), Scrum, and Kanban.

Extreme Programming

Extreme Programming (XP), is an Agile approach to software development described by certain values, principles, and development practices.

XP embraces five values to guide development: communication, simplicity, feedback, courage, and respect.

XP describes a set of principles as additional guidelines: humanity, economics, mutual benefit, self-similarity, improvement, diversity, reflection, flow, opportunity, redundancy, failure, quality, baby steps, and accepted responsibility.

XP describes thirteen primary practices: sit together, whole team, informative workspace, energised work, pair programming, stories, weekly cycle, quarterly cycle, slack, ten-minute build, continuous integration, test first programming, and incremental design.

Many of the Agile software development approaches in use today are influenced by XP and its values and principles. For example, Agile teams following Scrum often incorporate XP practices.

Scrum

Scrum is an Agile management framework which contains the following constituent instruments and practices:

Sprint: Scrum divides a project into iterations (called sprints) of fixed length (usually two to four weeks).
Product Increment: Each sprint results in a potentially releasable/shippable product (called an increment).
Product Backlog: The product owner manages a prioritised list of planned product items (called the product backlog). The product backlog evolves from sprint to sprint (called backlog refinement).
Sprint Backlog: At the start of each sprint, the Scrum team selects a set of highest priority items (called the sprint backlog) from the product backlog. Since the Scrum team, not the product owner, selects the items to be realised within the sprint, the selection is referred to as being on the pull principle rather than the push principle.
Definition of Done: To make sure that there is a potentially releasable product at each sprint’s end, the Scrum team discusses and defines appropriate criteria for sprint completion. The discussion deepens the team’s understanding of the backlog items and the product requirements.
Time-boxing: Only those tasks, requirements, or features that the team expects to finish within the sprint are part of the sprint backlog. If the development team cannot finish a task within a sprint, the associated product features are removed from the sprint and the task is moved back into the product backlog. Time-boxing applies not only to tasks, but in other situations (e.g., enforcing meeting start and end times).
Transparency: The development team reports and updates sprint status on a daily basis at a meeting called the daily scrum. This makes the content and progress of the current sprint, including test results, visible to the team, management, and all interested parties. For example, the development team can show sprint status on a whiteboard.

Scrum defines three roles:

Scrum Master: ensures that Scrum practices and rules are implemented and followed, and resolves any violations, resource issues, or other impediments that could prevent the team from following the practices and rules. This person is not the team lead, but a coach.
Product Owner: represents the customer, and generates, maintains, and priorities the product backlog. This person is not the team lead.
Development Team: develops and test the product. The team is self-organised: There is no team lead, so the team makes the decisions. The team is also cross-functional.

Scrum (as opposed to XP) does not dictate specific software development techniques (e.g., test first programming). In addition, Scrum does not provide guidance on how testing has to be done in a Scrum project.

Kanban

Kanban is a management approach that is sometimes used in Agile projects. The general objective is to visualise and optimise the flow of work within a value-added chain. Kanban utilises three instruments:

Kanban Board: The value chain to be managed is visualised by a Kanban board. Each column shows a station, which is a set of related activities, e.g., development or testing. The items to be produced or tasks to be processed are symbolised by tickets moving from left to right across the board through the stations.
Work-in-Progress Limit: The amount of parallel active tasks is strictly limited. This is controlled by the maximum number of tickets allowed for a station and/or globally for the board. Whenever a station has free capacity, the worker pulls a ticket from the predecessor station.
Lead Time: Kanban is used to optimise the continuous flow of tasks by minimising the (average) lead time for the complete value stream.

Kanban features some similarities to Scrum. In both frameworks, visualising the active tasks (e.g., on a public whiteboard) provides transparency of content and progress of tasks. Tasks not yet scheduled are waiting in a backlog and moved onto the Kanban board as soon as there is new space (production capacity) available.

Iterations or sprints are optional in Kanban. The Kanban process allows releasing its deliverables item by item, rather than as part of a release. Time-boxing as a synchronising mechanism, therefore, is optional, unlike in Scrum, which synchronies all tasks within a sprint.

Collaborative User Story Creation

Poor specifications are often a major reason for project failure. Specification problems can result from the users’ lack of insight into their true needs, absence of a global vision for the system, redundant or contradictory features, and other miscommunications. In Agile development, user stories are written to capture requirements from the perspectives of developers, testers, and business representatives. In sequential development, this shared vision of a feature is accomplished through formal reviews after requirements are written; in Agile development, this shared vision is accomplished through frequent informal reviews while the requirements are being written

The user stories must address both functional and non-functional characteristics. Each story includes acceptance criteria for these characteristics. These criteria should be defined in collaboration between business representatives, developers, and testers. They provide developers and testers with an extended vision of the feature that business representatives will validate. An Agile team considers a task finished when a set of acceptance criteria have been satisfied.

Typically, the tester’s unique perspective will improve the user story by identifying missing details or non-functional requirements. A tester can contribute by asking business representatives open-ended questions about the user story, proposing ways to test the user story, and confirming the acceptance criteria.

The collaborative authorship of the user story can use techniques such as brainstorming and mind mapping. The tester may use the INVEST technique [INVEST]:

Independent
Negotiable
Valuable
Estimable
Small
Testable

According to the 3C concept, a user story is the conjunction of three elements:

Card: The card is the physical media describing a user story. It identifies the requirement, its criticality, expected development and test duration, and the acceptance criteria for that story.
The description has to be accurate, as it will be used in the product backlog.
Conversation: The conversation explains how the software will be used. The conversation can be documented or verbal. Testers, having a different point of view than developers and business representatives, bring valuable input to the exchange of thoughts, opinions, and experiences. Conversation begins during the release-planning phase and continues when the story is scheduled.
Confirmation: The acceptance criteria, discussed in the conversation, are used to confirm that the story is done. These acceptance criteria may span multiple user stories. Both positive and negative tests should be used to cover the criteria. During confirmation, various participants play the role of a tester. These can include developers as well as specialists focused on performance, security, interoperability, and other quality characteristics. To confirm a story as done, the defined acceptance criteria should be tested and shown to be satisfied.

Agile teams vary in terms of how they document user stories. Regardless of the approach taken to document user stories, documentation should be concise, sufficient, and necessary.

Retrospectives

In Agile development, a retrospective is a meeting held at the end of each iteration to discuss what was successful, what could be improved, and how to incorporate the improvements and retain the successes in future iterations. Retrospectives cover topics such as the process, people, organisations, relationships, and tools. Regularly conducted retrospective meetings, when appropriate follow up activities occur, are critical to self-organisation and continual improvement of development and testing.

Retrospectives can result in test-related improvement decisions focused on test effectiveness, test productivity, test case quality, and team satisfaction. They may also address the testability of the applications, user stories, features, or system interfaces. Root cause analysis of defects can drive testing and development improvements. In general, teams should implement only a few improvements per iteration. This allows for continuous improvement at a sustained pace.

The timing and organisation of the retrospective depends on the particular Agile method followed. Business representatives and the team attend each retrospective as participants while the facilitator organises and runs the meeting. In some cases, the teams may invite other participants to the meeting.

Testers should play an important role in the retrospectives. Testers are part of the team and bring their unique perspective. Testing occurs in each sprint and vitally contributes to success. All team members, testers and non-testers, can provide input on both testing and non-testing activities.

Retrospectives must occur within a professional environment characterised by mutual trust. The attributes of a successful retrospective are the same as those for any other review as is discussed in previous articles.

Continuous Integration

Delivery of a product increment requires reliable, working, integrated software at the end of every sprint. Continuous integration addresses this challenge by merging all changes made to the software and integrating all changed components regularly, at least once a day. Configuration management, compilation, software build, deployment, and testing are wrapped into a single, automated, repeatable process. Since developers integrate their work constantly, build constantly, and test constantly, defects in code are detected more quickly.

Following the developers’ coding, debugging, and check-in of code into a shared source code repository, a continuous integration process consists of the following automated activities:

Static code analysis: executing static code analysis and reporting results
Compile: compiling and linking the code, generating the executable files
Unit test: executing the unit tests, checking code coverage and reporting test results
Deploy: installing the build into a test environment
Integration test: executing the integration tests and reporting results
Report (dashboard): posting the status of all these activities to a publicly visible location or e-mailing status to the team

An automated build and test process takes place on a daily basis and detects integration errors early and quickly. Continuous integration allows Agile testers to run automated tests regularly, in some cases as part of the continuous integration process itself, and send quick feedback to the team on the quality of the code. These test results are visible to all team members, especially when automated reports are integrated into the process. Automated regression testing can be continuous throughout the iteration. Good automated regression tests cover as much functionality as possible, including user stories delivered in the previous iterations. Good coverage in the automated regression tests helps support building (and testing) large integrated systems. When the regression testing is automated, the Agile testers are freed to concentrate their manual testing on new features, implemented changes, and confirmation testing of defect fixes.

In addition to automated tests, organisations using continuous integration typically use build tools to implement continuous quality control. In addition to running unit and integration tests, such tools can run additional static and dynamic tests, measure and profile performance, extract and format documentation from the source code, and facilitate manual quality assurance processes. This continuous application of quality control aims to improve the quality of the product as well as reduce the time taken to deliver it by replacing the traditional practice of applying quality control after completing all development.

Build tools can be linked to automatic deployment tools, which can fetch the appropriate build from the continuous integration or build server and deploy it into one or more development, test, staging, or even production environments. This reduces the errors and delays associated with relying on specialised staff or programmers to install releases in these environments.

Continuous integration can provide the following benefits:

Allows earlier detection and easier root cause analysis of integration problems and conflicting changes
Gives the development team regular feedback on whether the code is working
Keeps the version of the software being tested within a day of the version being developed
Reduces regression risk associated with developer code refactoring due to rapid re-testing of the code base after each small set of changes
Provides confidence that each day’s development work is based on a solid foundation
Makes progress toward the completion of the product increment visible, encouraging developers and testers
Eliminates the schedule risks associated with big-bang integration
Provides constant availability of executable software throughout the sprint for testing, demonstration, or education purposes
Reduces repetitive manual testing activities
Provides quick feedback on decisions made to improve quality and tests

However, continuous integration is not without its risks and challenges:

Continuous integration tools have to be introduced and maintained
The continuous integration process must be defined and established
Test automation requires additional resources and can be complex to establish
Thorough test coverage is essential to achieve automated testing advantages
Teams sometimes over-rely on unit tests and perform too little system and acceptance testing

Continuous integration requires the use of tools, including tools for testing, tools for automating the build process, and tools for version control.

Release and Iteration Planning

As mentioned in this article, planning is an on-going activity, and this is the case in Agile lifecycles as well. For Agile lifecycles, two kinds of planning occur, release planning and iteration planning.

Release planning looks ahead to the release of a product, often a few months ahead of the start of a project. Release planning defines and re-defines the product backlog, and may involve refining larger user stories into a collection of smaller stories. Release planning provides the basis for a test approach and test plan spanning all iterations. Release plans are high-level.

In release planning, business representatives establish and prioritise the user stories for the release, in collaboration with the team. Based on these user stories, project and quality risks are identified and a high-level effort estimation is performed.

Testers are involved in release planning and especially add value in the following activities:

Defining testable user stories, including acceptance criteria
Participating in project and quality risk analyses
Estimating testing effort associated with the user stories
Defining the necessary test levels
Planning the testing for the release

After release planning is done, iteration planning for the first iteration starts. Iteration planning looks ahead to the end of a single iteration and is concerned with the iteration backlog.

In iteration planning, the team selects user stories from the prioritised release backlog, elaborates the user stories, performs a risk analysis for the user stories, and estimates the work needed for each user story. If a user story is too vague and attempts to clarify it have failed, the team can refuse to accept it and use the next user story based on priority. The business representatives must answer the team’s questions about each story so the team can understand what they should implement and how to test each story.

The number of stories selected is based on established team velocity and the estimated size of the selected user stories. After the contents of the iteration are finalised, the user stories are broken into tasks, which will be carried out by the appropriate team members.

Testers are involved in iteration planning and especially add value in the following activities:

Participating in the detailed risk analysis of user stories
Determining the testability of the user stories
Creating acceptance tests for the user stories
Breaking down user stories into tasks (particularly testing tasks)
Estimating testing effort for all testing tasks
Identifying functional and non-functional aspects of the system to be tested
Supporting and participating in test automation at multiple levels of testing

Release plans may change as the project proceeds, including changes to individual user stories in the product backlog. These changes may be triggered by internal or external factors. Internal factors include delivery capabilities, velocity, and technical issues. External factors include the discovery of new markets and opportunities, new competitors, or business threats that may change release objectives and/or target dates. In addition, iteration plans may change during an iteration. For example, a particular user story that was considered relatively simple during estimation might prove more complex than expected.

These changes can be challenging for testers. Testers must understand the big picture of the release for test planning purposes, and they must have an adequate test basis and test oracle in each iteration for test development purposes as discussed in earlier articles. The required information must be available to the tester early, and yet change must be embraced according to Agile principles. This dilemma requires careful decisions about test strategies and test documentation.

Release and iteration planning should address test planning as well as planning for development activities. Particular test-related issues to address include:

The scope of testing, the extent of testing for those areas in scope, the test goals, and the reasons for these decisions.
The team members who will carry out the test activities.
The test environment and test data needed, when they are needed, and whether any additions or changes to the test environment and/or data will occur prior to or during the project.
The timing, sequencing, dependencies, and prerequisites for the functional and non-functional test activities (e.g., how frequently to run regression tests, which features depend on other features or test data, etc.), including how the test activities relate to and depend on development activities.
The project and quality risks to be addressed.

In addition, the larger team estimation effort should include consideration of the time and effort needed to complete the required testing activities.

agile

Agile software development

The fundamentals of agile software development

Agile software development and the agile manifesto

In 2001, a group of individuals, representing the most widely used lightweight software development methodologies, agreed on a common set of values and principles which became known as the Manifesto for Agile Software Development or the Agile Manifesto [Agilemanifesto]. The Agile Manifesto contains four statements of values:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

The Agile Manifesto argues that although the concepts on the right have value, those on the left have greater value.

Individuals and interactions

Agile development is very people-centered. Teams of people build software, and it is through continuous communication and interaction, rather than a reliance on tools or processes, that teams can work most effectively.

Working software

Customer collaboration

Responding to change

Agile principles

The core Agile Manifesto values are captured in twelve principles:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
Deliver working software frequently, at intervals of between a few weeks to a few months, with a preference to the shorter timescale.
Business people and developers must work together daily throughout the project.
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
Continuous attention to technical excellence and good design enhances agility.
Simplicity—the art of maximizing the amount of work not done—is essential.
The best architectures, requirements, and designs emerge from self-organizing teams.
At regular intervals, the team reflects on how to become more effective, then tunes andadjusts its behavior accordingly.

The different Agile methodologies provide prescriptive practices to put these values and principles into action.