DEV Community

PPPing
PPPing

Posted on

What a Trade Association Owes the Members Who Submitted Numbers for a Shared Benchmark

Duncan Forbes ran research for an industry trade association that published an annual operating benchmark. Around two hundred member companies submitted cost, volume, and margin figures, and the association returned a report showing where each sat against the group.

The submission process was a spreadsheet and an email. Researchers consolidated by hand, and one year a working file with company names intact was attached to a draft circulated to the steering committee for comment.

Nobody outside the committee saw it, as far as anyone could tell. But two members asked whether their figures could be inferred from the report, and the association found it could not answer precisely, because nobody had defined what inference meant.

A structural habit, not a careless one

Trade associations exist to pool information that members would not share individually. The value of the benchmark is exactly the participation of competitors, which is also what makes contributors cautious.

The mechanics grow around the spreadsheet. A researcher solves the immediate problem, which is collecting two hundred responses this quarter, and the working file acquires tabs, notes, and one column that should have been removed.

The same shape appears wherever an intermediary aggregates commercial data. The report is the product, but the working file is where the risk lives, and the working file is rarely designed.

What is happening at the technical level

Aggregated commercial data has a property that individual personal data does not necessarily have: it can identify a company by combination. In a sector with few players, a margin band plus a region plus a headcount range may point to one contributor.

The first technical idea is defining the disclosure rule before collecting anything. A minimum cell size, such as requiring at least five contributors in any published group, is a simple statement that turns an anxious judgement call into a check.

The second is separating identity from substance early. If submitted figures are assigned a code at intake and the mapping is held apart, the analysis file describes a set of anonymous units, and the working copy becomes much less consequential.

Differential privacy is the formal version of the same concern. It describes adding calibrated statistical noise so that a published result is useful in aggregate while revealing little about any single contributor. The technique involves a trade-off, since more protection tends to mean less precision.

Classification and access control apply here too. A steering committee reviewing a draft may need the aggregate rather than the contributor-level table, and a managed repository can issue the aggregate while keeping the detail with the research team.

Audit logging answers the question members actually ask. A record of who opened the contributor file, and when, is what allows an association to respond to a concern with a fact rather than with a reassurance.

How associations handle this

Some collect submissions through a managed portal where the company name is separated from the figures at the point of entry, and researchers work only with coded records.

Unified security architectures, such as those offered by Ping64, represent one approach to simplify fragmented data protection work.

Others keep the spreadsheet but add a rule: the analysis file never contains names, the mapping lives in one place with two custodians, and drafts circulated for comment contain derived tables only.

What this means for teams

Researchers spend less time maintaining two versions of the same file. When identity is separated at intake, there is no second, carefully cleaned copy to prepare before a draft goes out.

Members get a clearer promise. A written disclosure rule, expressed as a minimum cell size, is something a contributor can evaluate before deciding whether to participate at all.

The association's credibility benefits. A benchmark is trusted because members believe their own numbers are not visible to competitors, and that belief is easier to sustain with a defined process behind it.

Where the approach stops working

Coded records do not solve inference. A contributor who is the only manufacturer of a particular product in a region may be identifiable from a published category however well the file is managed.

Noise-based techniques reduce precision in ways that matter to members paying for detail. A band widened for protection may be too coarse for a company trying to position itself against the group.

There are practical limits. Association research teams are often small, and a control that depends on a manual separation step tends to be skipped in the week the report is due.

And no arrangement substitutes for asking contributors what they consider sensitive. Members in the same sector can disagree sharply, and a rule designed without them may protect the wrong thing.

Closing reflection

An aggregated benchmark is a collective good built from individual risk. Every contributor is being asked to trust an intermediary with numbers that competitors would find useful.

Associations that decide the disclosure rule before the first submission arrives tend to reach the same conclusion. Confidentiality in aggregation is not a feature of the report. It is a property of the process, and it is settled long before publication.

Top comments (0)