Naoki Yoshida Details Ungarmax Exploit And Actions Taken

FFXIV Producer and Director Naoki Yoshida has posted today on the official forums detailing the timeline of the Ungarmax exploit, their plans going forward, and what actions will be taken against those who used it. Yoshida-san talked about the bug late last month, saying that they would be investigating and taking appropriate action.

You can read the full post below.

The Ungarmax Exploit and Our Response

FFXIV Producer and Director Yoshi-P here.
I’d like to comment further on the situation surrounding the bug which allowed players to execute the squadron limit break Ungarmax outside of command missions.While we reviewed a great deal of information obtained from various sources, including social network sites, we found much of it was conflicting. Accordingly, we wanted to ensure that we made a clear and accurate assessment of the circumstances and underlying cause, and from that determined what preventative measures could henceforth be taken.Since I wanted to touch on how our conclusions impact our development and operations teams, as well as QA and our bug reporting flow, this post ended up being extremely long. I ask that you set aside sufficient time to read it in full. I’ve also included a timeline and summary to help you all understand the situation to this point.– Timeline of Events

  • Ungarmax is implemented in internal build along with command missions for 4.1.
  • Due to action implementation oversight, bug allowing usage of Ungarmax outside of command missions is introduced.
  • Bug remains undiscovered by development and operations teams, and is ultimately pushed live.
  • Bug is discovered by players and is intentionally exploited to execute Ungarmax outside of command missions.
  • Player executing Ungarmax outside of command missions is observed by another player, who reports it three times as cheating.
  • Reports of cheating are received by the Special Task Force, which begins to monitor the reported player.
  • The STF is unable to confirm the usage of third-party tools to facilitate cheating and ceases monitoring the reported player.
  • At a later date, the reported player is confirmed to have used a third-party tool to cheat and is permanently banned.
  • The Ungarmax bug remains unknown to the development, operations, and QA teams for an extended period of time.
  • On the afternoon of 2/27 JST, information regarding the Ungarmax bug is spread by several players aware of it.
  • On the evening of 2/27 JST, the development, operations, and QA teams commence an investigation and discussion as to how to proceed.
  • At 2/28 4 PM JST, the servers are taken down for emergency maintenance.
  • Two hours later, at 6 PM JST, the servers are brought online and the Ungarmax bug has been addressed.
  • The work flow for the development teams is revised in an effort to eliminate similar bugs in the future.
  • The work flow for the STF and operations teams is revised in an effort to ensure that similar reports are fully investigated.
  • Disciplinary actions are planned for players who intentionally and/or repeatedly exploited this bug in order to clear content.

In summary, this bug has been addressed, and Ungarmax should no longer be usable in this fashion. Furthermore, in an effort to reduce the occurrence of similar bugs, as well as to ensure that the reports we receive are afforded all due consideration, we have reviewed and revised the process by which the STF escalates issues to the development teams, the GMs, and the QA teams.

At this time, we are in the final stages of determining what disciplinary actions to take against players who intentionally and/or repeatedly exploited this bug in order to clear content.

This concludes the summary of recent events and our actions taken in response.
The FFXIV team has been and always will be fully committed to operating in a fair and open manner. We hope we can continue to count on your patronage in the years to come.

As previously mentioned, what follows is a far longer and more detailed explanation of recent events. Should you wish to read it in full, I suggest you set aside sufficient time to do so.

– A Detailed Explanation of the Bug and Our Preventative Measures

To review, the Ungarmax bug is as follows:

Provided certain conditions are met, the squadron limit break Ungarmax can be triggered at will outside of command missions.

The conditions for reproducing this bug are as follows:

  • Player has unlocked adventurer squadrons
  • Player has unlocked command missions
  • Player has filled at least one level of the limit gauge

When a player attempts to execute a class, job, or general action with the keyboard, mouse, or gamepad, several things must happen before the action is actually executed and processed.
1. Action input is registered by player client.
2. Action input is converted to packets and sent from player client to server.
3. Server-side check to determine if action can be executed is performed.
4. If confirmed possible, server-side calculations are performed and results are sent to player client via packets.
5. Results packets are received by player client, which processes and displays animations, effects, relevant figures, and so forth.
6. If confirmed impossible, input is processed as error and results are sent to player client via packets.

As you can see, a successful server check must always be performed before an action is permitted to be executed. An action input alone should never be sufficient, and a sever check that confirms whether or not an action is valid is essential to ensuring that cheating is not taking place.

However, because the squadron limit break Ungarmax was not implemented as part of the battle system by the usual team, the normal procedures that ensure such actions are checked by the server were not observed. Moreover, the routine requests that the server-side checks for all relevant actions be aggressively tested by the QA team were insufficient. As a result, this exceedingly simple bug went unnoticed and made it into the release build. This goes to show how a single oversight in a patch containing a plethora of new actions and content can have drastic repercussions.
Our Assessment:
In developing and expanding the content of FFXIV, a great many individuals had to work in parallel. As a result, the minimum standards and procedures for creating new actions were neglected, and not all team members were sufficiently aware of the importance of ensuring server side checks are performed. Additionally, the squadron limit break Ungarmax was intended to be an order given to your unit members and therefore should not have been usable via the /ac macro. These mistakes created a situation in which players could easily exploit this bug to their advantage.
Our Response:
The battle team, which is responsible for the creation of most new actions, will once again share with other teams and planners their standard methods and procedures. The QA teams will review their guidelines with an eye towards performing more stringent checks in the future.

Moving forward, all new actions will be:
1. Tested to ensure server-side checks are being processed correctly, regardless of intended functionality.
2. Flagged unusable with /ac macros by default, and added to a white list of valid actions only upon further review.

These and other precautions will be taken regardless of whether or not formal requests have been received, in addition to the standard quality assurance checks to ensure that actions are performing to spec. It is our belief that by taking such measures to reduce developer error that we will prevent the occurrence of similar bugs in the future.

– Our Efforts to Ensure Reports of Cheating are Properly Processed

As some of you may be aware, there were claims made on external sites that this bug was reported to us four months ago. In order to ascertain the truth of the matter, we investigated all relevant communications and log data.

Ultimately, we concluded that three reports of cheating were made via the support desk between October 15 and 16, 2017. All reports were made by the same player regarding the usage of Ungarmax by another player in the Royal City of Rabanastre, where it should not have been possible.

Although an investigation was opened into these claims, it was eventually closed without informing the development or QA teams. Because of this, despite the player’s reports, the bug remained unknown to us for some time. Again, we deeply apologize for the resulting problems.
Our Assessment:
The main purpose of the Special Task Force is to investigate illicit activities in FFXIV. Manipulation of packet data, teleport and speed hacks, RMT, and other similar actions that may be considered cheating fall within their purview.
When they received these reports and opened their investigation, they did not focus on the possibility that the player was exploiting a bug so much as the possibility the player was utilizing a new third-party tool. Therefore, although they monitored the reported player for a time, when they did not observe them using any such tools, they closed the investigation. While the STF has been instrumental in ensuring that our players can enjoy a fair and stable gameplay environment, in this instance their specific approach backfired.
Our Response:
Within the FFXIV operations teams, there are two in particular that are responsible for resolving in game issues. The GMs that you all know comprise one, and the STF is the other. After reviewing the various responsibilities assigned to both, we have revised the overall workflow to ensure that neither group becomes too compartmentalized, and to ensure that all reports regardless of content are eventually forwarded to the development and QA teams. That being said, because this change alone will drastically increase the quantity of reports received, they will be classified and reviewed as priority dictates by the relevant individuals to reduce the risk of a report slipping through the cracks.

– Disciplinary Actions to Be Taken against Players Who Exploited the Ungarmax Bug

A great many players, as well as our own development and operations teams, learned of this bug after a post was made on Reddit. We have since analyzed the battle log data from all players on record from the release of 4.1 to the date of the aforementioned post, as well as the date when maintenance was performed to address the issue. The results are as follows:
(Of reports received, 66 in total came from all regions)

– Prior to midnight JST, 2/27, 25 characters across 21 accounts were confirmed to have used Ungarmax in 8-man duties.
Our Assessment and Response:
The players of these 25 characters were confirmed as having exploited the bug prior to knowledge of it becoming widespread. Therefore, they not only chose to use Ungarmax outside of command missions, but actively concealed the existence of the bug allowing them to do so from others. Accordingly, their accounts are being suspended.

– After midnight JST, 2/27, 1110 characters were confirmed to have exploited this bug to execute Ungarmax under abnormal circumstances.
Our Assessment:
Of these 1110 characters, 457 exploited the bug only once, and in either public or housing areas, or in undersized parties in older instanced content. Given the circumstances, we believe these instances were simply players seeking to confirm whether or not the bug did indeed exist. Of the remaining characters, 523 characters used it more than once, albeit only a few times. However, 130 characters were confirmed to have repeatedly exploited this bug. Players will be treated differently according to these factors.
As an additional note, while during our investigation we confirmed that several players exploited this bug while undertaking the Unending Coil of Bahamut prior to 2/27, and one player after, none successfully completed the duty.
Our Response:
Players we judge to have exploited this bug not to progress through content, but solely out of curiosity, will not be subject to disciplinary action. Not only was this a major bug that was created by developer error, it persisted for months due to our failure to properly process user reports, and the resulting usage by curious players upon word of it spreading is understandable. Our analysis of battle and other log data from the period of when information on the bug was disseminated until the bug was fixed confirms that these players did not profit from its usage, and we see no reason to hold them to account. That being said, we must strongly encourage you all to not experiment with these bugs in the future, and to that end will be sending mails to all concerned.
Players who exploited this bug more than once, if only a few times, will be regarded as having done so intentionally, and therefore will be subject to a 24-hour account suspension. However, players who exploited this bug a significant number of times will be more harshly penalized.

Finally, I would like to deeply apologize to you all yet again for the inconvenience this has caused. We are committed to doing everything in our power to provide you with a stable and enjoyable gameplay experience free of bugs. However, I must reiterate that to exploit such a clear and obvious bug like this is to violate the terms of service, and ask that you not do so.

We hope you will continue to enjoy your time in the world of FINAL FANTASY XIV for years to come.