Dear players!

As we have previously announced, today we will be covering your questions about bug testing with some ‘behind-the-scenes’ insight. Please note that we have tried to make our answers as detailed as possible, and as such they can be somewhat long at times. We hope you will find it interesting!

TESTING

Q: Do you have any testers? Why do the regular players have to put effort into finding and reporting bugs?

A: Yes, as a matter of fact there are two QA departments (quality assurance teams tasked with testing), one’s a part of the development team, the other one works correspondingly from the localization unit. All new game versions, or builds, released by the development team undergo thorough testing. Nevertheless, it’s necessary to note that mass-testing and internal testing are two entirely different concepts. In a complex game some errors may be relatively difficult to reproduce and as such, it is quite possible that some of them won’t be revealed during the internal tests. Other issues may arise only once the new version goes live, as they’re often nearly impossible to recreate without ‘real’ actions in not any less real environment — on the game server.

Apart from that, no matter how thorough you are, it’s practically impossible to dish out every bug out there, given that it’s an endless amount of potential actions and combinations that may take weeks, or better yet, months to run for even the most well-equipped and staffed QA team. In order to ensure we manage to eliminate as many bugs as possible before the release, we’ve created the PTS.

Q: When the bugs are reported via the Support Center, how are they processed?

A: First things first, it’s worth nothing that the support team doesn’t work with tackling and fixing in-game bugs. That being said, they surely log all the received information and pass it on to the development team and QA.

Each and every report of this kind is processed by support team specialists. From there on, the CS specialist checks with the current database of known bugs, and if the problem had been previously marked as known, and no further information is required, the report is closed with an appropriate response. If the error or bug encountered happens to be new and additional information is required, this information is requested from the player. Afterwards the completed reports are added to the very same database that QA and the developers further work with.

As you can see, all bug reports processed by the support team reach the developers but in a more refined fashion.

GENERAL INFORMATION ABOUT BUGS

Q: Why do bugs even exist?

A: Anyone familiar with coding or tech understands the simple connection: the more complicated a mechanism or a system is, the higher the likelihood of bugs appearing when that system is changed.

The same analogy can be drawn in the game: Warface’s evolved over the several years of its existence into a super-hard system, with each component being closely connected to the others. As such, even the least impactful of changes can lead to malfunctions in other systems or areas. Bugs are inevitable in any game; finding and fixing them is a constant and integral part of game development. Unfortunately, one needs to accept that as long as the game is changed and improved, bugs will continue to appear.

Q: What are the factors that affect how fast these bugs are fixed?

A: The first and most important factor is how critical the error is. All bugs and errors can be divided into three key categories: critical, the ones that interfere with the core game processes (the ones that make it impossible to play), severe — these affect a large share of players but do not impede the aforementioned core game processes, and low priority errors — these barely affect the game at all.

For example, an error that makes you unable to repair a weapon will be critical: this game update will definitely not reach the main server; even if the error isn’t revealed in either testing stages, it’ll be fixed urgently by any means possible ahead of other tasks.

The recent problem with bomb carrying can be called critical: it had practically made the “Plant the Bomb” game mode unplayable and was fixed as quickly as possible with a follow-up update.

The severe or general errors can be loading issues, to compare these are something along the lines of, having 1 out of 20 players affected or even that gunshot sound bug we’ve seen. These either do not affect the majority of the players or they don’t escalate to more serious issues, thus leaving the game playable. This kind of errors is fixed in planned monthly updates. This category contains a lot of bugs and errors that are difficult to fix as they are often connected to other areas of the game. Sometimes several consecutive fixes are needed to eliminate the source of the problem, and it may take several months.

Low priority bugs and errors are connected to faulty animations that don’t affect the in-game processes as much, specific texture or visual bugs: eye candy and so forth. Pretty much, it’s the stuff that is purely cosmetic or esthetic and doesn’t cross any core game functions at all. Errors of this sort appear frequently, and seeing as they’re the easiest and quickest to fix, they appear the most in the bugfixes lists every update.

These categories aside, it’s necessary to remember that each issue is unique and some are naturally harder than the others. The first step to solving a problem is locating it, along with all the possible causes and which areas of the code are affected. In this case, if the QA testers are unable to recreate the problem, this process becomes a tad more difficult altogether as we have to gain more information from the players. The usual descriptions such as “I don’t deal damage”, “I get teleported” or “I cannot join a match” is just not enough for the developers, as more direct and/or technical information is required: recordings, screenshots, client logs. These need to be — well, first, received and — carefully studied, and that’s why reporting issues to the support team is important, as well as to provide the requested information further.

Once the source of the problem is determined, we start fixing the bug. This process can also take quite some time given that each developer is dealing with several problems at a time, and each bug or problem can affect several areas at once.

Q: How come the bugs sometimes make it from PTS to the main server when found?

A: PTS is the last testing stage for the game update before its release. This stage is essential for finding the critical issues (if any) that haven’t been found during internal testing. Each PTS launch follows a strict schedule and any delay leads to postponing other updates. The longer each game update stays on PTS, the lower the chance that other updates will follow on time. Otherwise, we’d have to cut down on the new features introduced in these patches which is definitely not something you guys would want. The perfect example of this is the January update: the month was simply short due to the holidays and the update just contained fewer features than usual, and that brought some negative feedback. That’s why each bug found on PTS is studied carefully and separately, and each is assigned its own status.

If the error isn’t critical or doesn’t affect the majority of players, the update is greenlighted to go live, and each bug found is logged for the future updates. Although, there are exceptions to this rule: some problems may be less impactful but quite unpleasant or potentially irritating. If there are means to issue a fix right away, the problem will likely be fixed before the update’s out.

Q: Why do some errors don’t appear on PTS at all?

A: This is quite a difficult question and there’s no single answer there. First of all, the PTS still differs in a variety of ways, and the simplest is the server workload, the cause of many problems in the game. Secondly, if the error isn’t constant — which means, it’s caused only given specific conditions and not by the majority of players — it’s often hard to spot given that not everyone gets on the PTS. Thirdly, the bug may simply originate in game modes that aren’t frequently played. The best example is weapon changes. The update usually contains 2 new guns, while the rest receive various changes. Obviously, the majority of players will rush to check out the new guns and not bother using the older ones as much. That’s one way for an error to pass undetected.

As such, this leads to a rather simple conclusion: if you’re headed for the PTS, please remember that it stands for “Public Test Server”; your goal shouldn’t be to just have fun but to check out the features noted in the changelogs, to find bugs. Be sure to let us know if you find something working incorrectly, we’re fairly thorough when it comes to your PTS feedback.

AND SOME MORE ABOUT SPECIFIC BUGS

Q: When will the smoke bug (disabling the “smoke” effect”) be fixed?

A: It’s a rather complex issue that unfortunately depends on communicating and collaborting with NVIDIA, as this action is performed with the help of their software. Our developers are working with NVIDIA and have issued several fixes. Still, some of them are temporary or too harsh on the players (resetting settings and so on). We keep working on the issue and each update releases new solutions. The next major fix we’re hoping for is planned for April.

Q: Sound “loop” bug: how come it’s taking so long to fix?

A: At present time, this problem’s one of the first to be fixed. While it doesn’t affect general gameplay, it is increasingly irritating. Unfortunately, the source of the problem is just a little deeper than expected: it affects the gunshot sounds, character call-outs and shout-outs and several other points. One of the past updates has only partially fixed the problem, and we’re planning to “cure” it altogether in one of the nearest future updates.

Q: What are you doing to fix the “teleporting” issues?

A: “Teleports” are not a stand-alone problem as such, but a result of overlapping of several other systems running simultaneously, and neither can be just removed from the game for good. This problem has deep ties with the ping issues, performance/capability on the client side as well as some internal game systems. Unfortunately, there’s no clear-cut solution that could just, “remove this, add that, and the teleports are gone”, however it doesn’t mean we’re not working on it.

Right now we’re gradually adding new elements and systems to register movement in the game that should solve some of the problems. Nevertheless, it’s a massive change to the game, even if it doesn’t seem as comparable as, say, adding new maps, and we have to be careful adding it in bit by bit. The new system is tested carefully, scheduled to affect only a small amount of players, after that we analyze the stats and logs to see whether we add a new element or keep working on the new one. We’ve been at it for several months.

Well, that’s it for today! This surely isn’t the full story as to how internal testing goes and why these or other problems may appear, but if you find this sort of articles useful — suggest a topic, ask questions, and we’ll be sure to work on some new ones. Can’t wait to hear what you guys think!

---