The primary purpose of the ASEAN meetings, of course, is to help guide ASEAN member states. Here in Kuala Lumpur, economic ministers met to discuss progress towards meeting the ASEAN Economic Community (AEC) and other elements of the ASEAN integration agenda.

Officials also met with a variety of counterparts from other countries that are dialogue partners with ASEAN. Some of the counterparts that are also Trans-Pacific Partnership (TPP) members, including Australia, Canada, Japan, New Zealand, and the United States, held informal bilaterals between themselves and the ASEAN TPP members (Brunei, Malaysia, Singapore and Vietnam) to try to break through remaining issues in the TPP negotiations.

Minsters from the 16 member countries of the Regional Comprehensive Economic Partnership (RCEP) also met to discuss progress after 9 rounds of talks.

Finally, various industry groups and associations, like the EU-ASEAN Business Council and the East Asia Business Council, snagged a few minutes of the time and attention of trade ministers during assorted sideline events.

Many of the ASEAN discussions underway are likely to include at least one comment such as, “ASEAN is on track to achieve that AEC at the end of the year, with 80% (or 90%) of the objectives already finished.” Or, “As the ASEAN Blueprint (or scorecard) shows, we have already accomplished 80 (or 90) percent of our objectives.”

What makes these statements puzzling is that ASEAN dropped the scorecard some time ago. The last time ASEAN published the results, the scorecard only covered the period through 2011. There is really no way to know how close or how far ASEAN might currently be from meeting the targets attached to the AEC.

Yet the idea of a scorecard seems rather firmly anchored. Where did the original impetus for the scorecard come from?

ASEAN faces at least two distinct challenges in implementing commitments. First, ASEAN’s methods of negotiation are unusual. The grouping uses something called the “ASEAN – X” (ASEAN minus X) approach. Under this approach, somewhere between all 10 members and no members actually implement any given commitment.

This soft, persuasive approach means that members have sufficient flexibility and policy space to move ahead with commitments when they believe the time is right for their specific developmental status and domestic conditions. The expectation is that all 10 members will eventually arrive at the same set of outcomes, since members that disagreed with the original objective could have rejected the approach or the commitment outright from the beginning.

The ASEAN-X system, however, means that enforcement of commitments may always be a problem.

Second, the member states do not like confrontation. The region has been peaceful for all these decades in part, most argue, from the “ASEAN Way” of handling disagreements. Such an approach requires discretion and careful dialogue over long periods of time to help build up trust and communication.

The lack of confrontation means that the usual path of dispute settlement, taken by states in economic partnerships and free trade agreements, is not an option for ASEAN. Member states are unlikely to agree to allow one another to actually be taken to arbitration by another member over failure to implement ASEAN commitments. (Note, however, that ASEAN members have, on rare occasions, moved trade disputes over to the World Trade Organization system if the violation of ASEAN commitments can also be viewed as a WTO violation.)

So, imagine that you are tasked with getting 10 member states to get to the same outcomes if the pathway is flexible and you cannot count on any sort of dispute settlement system to be actually used by members to ensure enforcement of commitments. What system would you recommend?

One solution is to use some sort of “naming and shaming” approach to call out laggards. However, if such a system were seen as too harsh and critical, it would never get past the member states.

Hence, ASEAN defaulted to the creation of a blueprint. The original blueprint was a fixed number of commitments that ASEAN members had agreed to implement on the path to the AEC. These commitments were broken down into four broad areas.

Of most interest to the business community were promises made in the first pillar, “Single Market and Production Base.” This included commitments towards free flows of goods, services, investment, skilled labor and freer movement of capital. The blueprint for achieving these objectives was broken down into phases, starting in 2007-2009 and concluding in 2015 with the launch of the AEC.

The commitments would be tracked by the use of an ASEAN Scorecard that could measure progress towards achieving each of the items in the blueprint. Critically, to get approval of the members, the report card had several interesting features. The report is an aggregated account of progress—no single member is ever praised or punished specifically for implementing or failing to implement any given commitment.

The scorecard is a binary system. Members are given “credit” for progress made towards the objective or are not. There is no attempt to measure actual implementation. The member states themselves must provide the information to the Secretariat for tabulation.

At the Secretariat, officials add up (behind the scenes) the check marks for progress and report out either “fully implemented” or “not fully implemented.” As an example, the first scorecard for free flow of goods showed 9 items fully implemented and 0 items not implemented between 2007-2009. The record reported in Phase 2, 2010-2011, was more mixed with 23 items implemented and 24 items not fully implemented. It is not clear what items are even being measured.

This highlights more challenges with the blueprint/scorecard system. Over time, members added additional items. This made the scorecard more of a moving target. A member state might have thought it would receive credit for 80% of commitments in phase 2, only to discover that with new, unmet commitments added to the scorecard, it (and ASEAN) might receive a score closer to 70% or worse.

The easiest commitments are always likely to go first. The tougher parts of integration dealing with the more sensitive items are most likely to appear in the blueprint at the end of the process. Hence, under whatever system members might have devised to address implementation challenges for the AEC, progress towards the end was likely to slow down.

In the years since the last publication of scorecard results, officials and other stakeholders have had discussions about revising the system. But given the extreme unwillingness of participants to have anything that might appear to be bad news or backsliding on commitments, members appear to have decided to abandon the entire scorecard exercise. The lack of an actual scorecard now makes repeated references in 2015 to the scorecard and blueprints so strange to hear this week in Malaysia.