Testing phases within the Gaming Software Development Lifecycle
Test types during the gaming quality assurance phase
Compliance testing is a very involved process. There are lengthy submission forms to be filled and ITL compliance fees are often much higher than gaming quality assurance testing fees. Therefore, prior to submitting a product to an ITL, a machine manufacturer should ensure the quality of the product being submitted by performing gaming quality assurance testing.
Gaming quality assurance testing activities align with the fundamental test process. The following test activities are performed to ensure the product is tested for quality:
- Test planning
- Test monitoring and control
- Test analysis
- Test design
- Test implementation
- Test execution
- Test completion
Gaming quality assurance testing is an iterative process with development. Defects are logged and then the product is returned to development to fix the logged defects. Once the logged defects are fixed, the product comes back to gaming quality assurance for further testing. This cycle continues until the product reaches the quality levels desired, as defined by the gaming quality assurance testing exit criteria.
The test types performed during gaming quality assurance testing include, but are not limited to:
- Localisation testing – testing to determine that all screens have proper language translations and any other localisation items such as date/time or numbering formats are done correctly.
- Functional testing – testing to determine that all functions work as designed and intended.
- Performance testing – testing to determine that response time is acceptable.
- Memory leak testing – testing to determine no performance issues arise due to lack of proper memory management by the software.
- Install-ability testing – testing to determine that the product can be easily installed by the casino operator without any issues.
- Portability testing – testing to determine that the product can be installed on all platforms it needs to run on.
- System integration testing – testing to determine that the product can integrate with systems/components, different versions of an application interface or able to communicate to a system.
- Operational testing – testing to determine that the product can function and be operated in a production environment, including reliability, security, recovery and failover testing.
- Pre-compliance testing – testing to determine that all the regulations and standards are met prior to submitting the product to the ITL. This helps ensure a minimal number of submissions of the product to compliance testing.
- Customer acceptance testing – Prior to submitting to compliance testing, some products are submitted to the client (i.e., lottery or casino) for customer acceptance testing to ensure all features function as the client expected.
Compliance testing occurs when a machine manufacturer wants to enter a jurisdiction with a new or modified product, be it a game, platform, hardware or system. Based on the jurisdiction, the machine manufacturer needs to submit the product with a request for certification to either an ITL or government-based compliance test lab. In some jurisdictions, it goes through both.
The formal submission documents are fully detailed, what is being submitted and what certifications are being requested for the product.
Once the product has passed compliance testing, the test lab will provide a certificate of compliance evidencing the certification of the rules and regulations that were to be met.
Once the regulatory commission has seen proof of the required certifications, it will allow the product to be installed in the gambling establishments in their jurisdictions.
When performing compliance testing, standard test plans are created and specific compliance checklists are used. These may include, but are not limited to:
- Jurisdictional specifications – usually defined by a governing body such as the federal, state and/or provincial government.
- ITL defined standards – defined by an ITL in the gambling industry.
- Other gaming related standards – some jurisdictions require other standards be adhered to. For example, some jurisdictions may require that gaming machines and systems in a jurisdiction are Game to System (G2S) protocol compliant. The G2S compliance checklist is defined by the Gaming Standards Association, the association that has defined the G2S
Many areas of the compliance testing will be the same as those performed in gaming quality assurance testing, but they are tested against the jurisdictional specifications and not the game specifications.
Some of the areas that are covered during compliance testing include:
- Rules of play – testing to determine that the rules meet the jurisdictional specifications.
- RNG, Payout Percentages, odds and non-cash awards – testing to determine that the payout percentage is within the range regulated in that jurisdiction.
- Bonus games – testing to determine that the game meets bonus regulations.
- Electronic metering – testing to determine that all meters required to be
monitored within that jurisdiction are being reported.
- Game history – testing to determine that the game history tracks, at a minimum, the number of games required by the jurisdiction.
- Power-up and power-down – testing to determine that the power up and down functionality works as per the jurisdictional specifications.
- Setup and Configuration – testing to determine that only configurations that are permitted within the jurisdiction can be enabled.
The Gaming Ecosystem
The gaming industry ecosystem overview
The gaming industry ecosystem is composed of the following organisations:
- Game Developers – develop casino games not specific to a gaming machine model. These games are usually distributed by a manufacturer or casino.
- Machine Manufacturers – make and sell the hardware, platforms, operating systems and games, developed in house or sub-contracted.
- Independent Test Labs – test and certify that the game software, hardware, firmware, platform and operating system follow all the jurisdictional rules for each location where the game will be played.
- Regulatory Commissions – approve every game played in their jurisdiction after the ITL certifies that the game meets the commission’s jurisdictional specifications.
The regulatory commission licenses the machine manufacturer to deploy the game in casinos or on online gaming sites in that specific jurisdiction. A game may be shipped to a casino before licensing; however, it cannot be deployed. The game must be licensed by the regulatory commission before it is deployed into the jurisdiction. Should any major defects be found in the casino, the regulatory commission can force the machine manufacturer to pull their game out of all casinos or demand that the online sites remove access to the game in that jurisdiction.
Video lottery terminals and their ecosystem
As indicated by the name, VLTs always have a video display for the game. VLTs either have standalone or server-based outcome architectures. In the standalone model, each VLT contains an RNG from which game outcomes are generated. In the server-based outcome architecture, VLTs obtain their outcomes from the server. This architecture has two possible models: the RNG model or the pre-determined finite pool model. In the server-based RNG model, the server generates the outcome it will provide to the VLT using an RNG located in the host. In the pre-determined finite pool model, the server obtains the outcome from a database of pre-determined outcomes. This model is similar to instant tickets and is often referred to as electronic instant tickets.
The types of games typically found on a VLT are: mechanical reel games, poker games and keno games. Most VLTs are multi-game machines, meaning multiple games are available for a player to choose from through a screen menu.
VLTs are frequently operated in a distributed environment over a Wide Area Network. For example, a few VLTs deployed in bars and/or pubs are connected to a central server through a Wide Area Network connection.
The VLT ecosystem is composed of:
- The EGM
- The site controller and/or bank controller
- The systems/servers used for monitoring and/or managing functionality
The EGMs are the machines on which the players choose to play the games. Each machine communicates to a site controller and/or bank controller and one or more central servers through a communication interface board using an electronic communication language referred to as a protocol. When VLTs are installed in a distributed environment, each retail location has a site controller to which the VLTs at that location are connected. The site controller serves multiple functions:
- Communicates and monitors VLTs to ensure they are online.
- Records game play transactions, cash-in/cash-out transactions and security
- May act as a protocol converter by translating the protocol implemented on the VLT to the protocol understood by the central server.
- Provide retailers with the ability to:
- Register players for player tracking cards
- Validate and pay out cash tickets
When VLTs are installed in a venue environment (i.e., a non-distributed environment), they are connected to a bank controller which functions like a site controller minus the retailer functions. A bank controller can support connection of several hundred VLTs, whereas one site controller typically supports fewer than 100 connected VLTs.
The VLTs and bank controllers and/or site controllers are connected to various central servers based on the functionality offered by a jurisdiction. At a minimum, VLTs installed in a venue environment include the following:
- A casino accounting system, which is responsible for monitoring the amounts wagered and paid on each VLT.
- A VLT CMS, which provides the ability to monitor game play, track, record and report security exceptions at the VLT and/or site controller, and monitor network availability in order to ensure continuous VLT operations in the event of communication loss.
Other central servers may include additional features, not limited to:
- A cashless wagering server, which allows for cashless transactions either through ticket-in/ticket-out (TITO) functionality or through electronic funds transfer (EFT).
- A distributed game content management server, which controls the selection, scheduling, distribution and auditing of VLT software to VLTs at remote retail sites.
- A player services server, which supports player loyalty, player rewards and
responsible gaming functionality.
- A progressive server, which manages progressive game play.
- A business intelligence server, which provides data warehousing and business analytics.
The other servers available are based on the functionality offered in the jurisdiction.
Slot machines and their ecosystem
Slot machines may have a video display or mechanical reels which have actual physical reels that spin.
Slot machines outcome architectures come from the Indian Gaming Regulatory Act, a 1988 US Federal law that establishes the jurisdictional framework that governs Indian Gaming in the US, This law provides definitions for Class I, Class II and Class III architectures. Class I relates to traditional Indian gaming and will not be discussed further in the context of casino gaming. Class II and Class III define the two outcome architectures used by slot machines. Class II (also know as electronic bingo) is defined in the Act as “the game of chance commonly known as bingo whether or not electronic, computer or other technological aids are used”. Class III (also known as traditional slot machines) has a broad definition in the Act. It states “all forms of games that are neither Class I nor Class II. Games commonly played at casinos, such as slot machines and table games, e.g., blackjack, craps, roulette, etc., fall in the Class III category.
The types of games typically found on a slot machine are: mechanical reel games, bingo games, poker games and keno games. Many slot machines are single-game machines, meaning only one game is available for play on the gaming machine.
Slot machines are typically operated in a venue environment such as a casino.
Slot machines (also known as Vegas style slot machines) are:
- Casino gaming machines with mechanical reels or a video display.
- Machines that have an RNG that is local to that machine.
Machines usually include a currency input device, such as a coin acceptor or a note acceptor, and a currency output device, such as a coin hopper.
The slot machine ecosystem is composed of:
- The slot machines
- A slot machine interface board (SMIB)
- A data collection unit or bank controller
- Central servers
Each slot machine contains an SMIB that is linked to the data collection unit or bank controller. Historically, an SMIB was a small board that was put into every mechanical or electro-mechanical machines. These early SMIBs connected to a wiring harness that would detect when a mechanical meter was incremented, or a mechanical door switch was opened. As time passed, these SMIBs evolved and now communicate electronically with the gaming machine and are often responsible for implementing the protocol that is used to communicate with the data collection unit or bank controllers and the remote central servers. The SMIBs, at a minimum, capture:
- The amounts wagered by the player.
- The amounts paid out to the player.
- And if the player is using a player card, any player data tracked by the casino.
Data collection units or bank controllers, as suggested by the name, are used to collect and store the data obtained from the SMIBs.
The data collected by the data collection unit or bank controller is communicated to the servers to update the data needed for the functionality provided by the servers.
Bingo machines (also know as electronic bingo machines), are:
- Machines that look and feel like slot games but are actually a game of electronic bingo.
- Machines on which the outcomes are obtained from a centralised bingo server.
- Machines that offer cashless input methods such as TITO or EFT. These
gambling machines do not have currency input/output devices.
- Machines on which the games:
- Are played exclusively against other players rather than against the house or against a player acting as a bank.
- Are based on multiple players playing for a common prize.
- Continue until there is a winner.
Each bingo machine contains an SMIB that is linked to the bingo server and other servers. The SMIBs, at a minimum, capture:
- The amounts wagered by the player.
- The outcome obtained from the server and the corresponding results.
- The amount paid out to the player.
The Network Switches are used to provide multi-player capabilities. Once a minimum number of players required for a specific game is met, the actual bingo game can start. The bingo server is the system that allows players to join a game until the group is at the minimum required and that provides the outcomes to the slot machines.
There are other central servers, such as the casino accounting server that tracks the amounts wagered and amount won, and the reporting server that allows the casino operators to report on the collected data.
Lottery and its ecosystem
The lottery ecosystem is composed of systems and devices deployed at the lottery and at each retail location.
The main device is the point of sale (POS) lottery terminal. The POS lottery terminal facilitates the sale of traditional lottery tickets by allowing the retail employee to either scan a selection slip containing the player selected numbers, or to select a Quick Pick option where the POS lottery terminal randomly selects numbers for the player. The POS lottery terminal then prints the tickets on the attached printer. The POS lottery terminal facilitates the sale of instant tickets by scanning the instant ticket sold. The attached customer display unit (CDU) allows the customer to view all steps of the sales transaction. The POS lottery terminal must coordinate all lottery ticket sale transactions with the lottery CMS. The POS Lottery terminal, printer and CDU/PDU unit are either separate devices or, in some cases, these devices can be integrated into one unit. When a player is ready to validate a ticket, the player can choose to have the retail employee scan the ticket using the POS Lottery terminal or perform the validating themselves on a POS Self-Serve Terminal.
The final device at the retail location is the multimedia display. The multimedia display is used for in-store advertising of lottery products, upcoming lottery promotions and winning numbers.
Once the numbers are drawn, the numbers are entered into the lottery CMS. Using the data stored in the database of the CMS, reports can be generated indicating how many winning traditional lottery tickets were sold and which retail location sold the ticket(s). For instant tickets, the barcode data of each ticket is stored in the CMS database. This allows the lottery employees to generate reports indicating how many tickets remain unsold and manage replenishment of physical tickets to retailers. The lottery CMS is responsible for storing all transactional data of tickets sold at each retail location. It also manages the advertising content to be displayed on the multimedia display units at the retail locations and downloads the appropriate content to the POS lottery terminal from which the multimedia display unit displays the content.
Lotteries are beginning to introduce alternative means by which to purchase lottery tickets. For example, purchases can be made at self-serve vending machines for instant tickets, self-serve ATMs like kiosks for traditional lottery tickets or online at their website for lottery tickets. These alternative devices and components must also coordinate all sale transactions with the lottery CMS.
Some of the areas that are covered during functional testing of the lottery ecosystem include:
- Online game rules and functionality.
- Scratch ticket management.
- Player account management.
- Geolocation functionality.
- Player services manager.
- Multi-player gaming
- Player loyalty and rewards.
- Account based play “cashless”.
- Responsible gaming functionality
- Game play functionality and playability.
- RNG algorithm and math rules.
- Artwork versus requirements
- Host accounting and reporting – determine that the game pays out what it should and that the money at play goes to the client if they win, or to the casino if the client loses.
- Tournament and real time event setup and management.
- Multiple game engines functionality and capabilities.
- Integrations with external gaming sites and mobile devices.