Metrics and Measurement: What to Track, Why, and How

What metrics should be tracked, and why? What do they actually tell you? Three unique perspectives will be explored from compliance, risk, and legal officers at Biogen Idec, PSEG, and OfficeMax. The trio will demonstrate, discuss, and debate the data they measure, the metrics they track, and the reasons for both.

Featuring:

These are my notes, live from the session:

How do compliance professionals demonstrate that what they are doing is effective and efficient? It’s difficult because compliance success is usually about what did not happen.

OfficeMax has some measurements that are key to retailers, including inventory shrink.

One risk that have to managed at a public utility is options trading and financial risks. They do lots of  market trading. That means they also have trader compliance issues. They have the classic compliance requirements of wall street, although it’s from the end user perspective.

They are highly regulated so there is compliance risk, of failing to follow the complex rules. There is strategic risk in designing the business operations. Lastly there is tactical risk in deploying the strategy and meeting the requirements of compliance.

Biogen tracks metrics around policy development. They track how long it takes and how much it cost. If it’s taking lots of outside resources, maybe they will consider bringing a resource internally.

One key discussion of the panel was the business impact of the measurements. Ideally, the measurements should impact business decisions and business strategy.

The panel emphasized the need for collaboration across the enterprise. Other units are already measuring business operation. Take advantage of the existing information. You can get better information (and save costs).

The more you understand the business and the more you demonstrate your knowledge of the business, the more successful you will be. Metrics for the sake of metrics is useless.

It’s great to generate stories about the bullets you dodged by identifying issues and risks before they have an impact on the business. Try to transform compliance from a value-add from merely being a cost-center.

Betting the Corporation: Compliance or Defiance

Lawrence D. Finder, Ryan D. McConnell & Scott L. Mitchell drafted a paper surveying the sixteen corporate deferred prosecutions and non-prosecution agreements entered into by the Department of Justice in 2008.

Betting the Corporation: Compliance or Defiance? Compliance Programs in the Context of Deferred and Non-Prosecution Agreements – Corporate Pre-Trial Agreement Update – 2008

In 2008, every agreement contained some sort of corporate compliance reform provision – continuing a trend we have seen over the last few years. This trend is the focus of this update. Aside from building on prior observations, this piece attempts to draw empirical observations about the types of compliance programs that come out of corporate pre-trial agreements. The authors recognize there is no one-size fits all template for corporate compliance programs. But by examining compliance programs in the context of DPAs and NPAs, the authors strive to provide a picture of what types of compliance measures are negotiated by the DOJ and corporate targets to resolve internal control and other business deficiencies that resulted in criminal wrongdoing. We hope that this will provide some guidance for attorneys and other professionals who deal with compliance issues.

The authors note that one of the big changes in 2008 was the DOJ’s implementation of a new charging policy. (You can find it at 9-28.000 of the U.S. Attorney’s Manual.) Although the policy is no longer associated with a particular person (like the 2006 McNulty memo, the  2003 Thompson memo and the 1999 Holder memo), the nine factors for charging a corporation are still the same:

  1. the nature and seriousness of the offense;
  2. pervasiveness of wrongdoing;
  3. the company’s history of similar conduct;
  4. the company’s timely and voluntary disclosure;
  5. the existence and effectiveness of a pre-existing compliance program;
  6. the company’s remedial actions;
  7. the collateral consequences (including harm to shareholders) of a conviction;
  8. the adequacy of prosecution of individuals; and
  9. the adequacy of civil or regulatory remedies

There is a new statement in USAM 9-28.200:” In certain instances, it may be appropriate, upon consideration of the factors set forth herein, to resolve a corporate criminal case by means other than indictment. Non-prosecution and deferred prosecution agreements, for example, occupy an important middle ground between declining prosecution and obtaining the conviction of a corporation.”

A second change in 2008 was the issuance of the Morford Memo that addresses the use of corporate monitors, providing guidance on issues that may arise in the selection of a monitor and the monitor’s duties.

2008 STATISTICS:

Total Number of Agreements: 16
Number of Privilege Waivers: 2   (13%)
Number of Agreements with Compliance Monitors: 6   (38%)
Number of Agreements With Compliance Reforms: 16 (100%)

The link above is to a draft copy of the paper. The final version is scheduled to be published in the South Texas  Law Review in May 2009.

OCEG Webcast on Code of Conduct

Scott Mitchell, Chairman and of the Open Compliance & Ethics Group, and Brett Curran, Director of GRC and Privacy at Axentis, conducted a webinar on the Code of Conduct.  The powerpoint slides are free, but the webinar itself requires a premium membership.

These are some metrics they propose for measuring the performance of a Code of Conduct:

  • Reach – Percentage that receives the Code of Conduct
  • Certification Coverage – percentage that certifies they understand and will uphold the code of conduct
  • Training Coverage – percentage that are trained about the contents of the Code of Conduct
  • Awareness – percentage that report they what the code is and what is says
  • Mastery – percentage that proves through testing that they know the Code and what it says
  • Reporting Readiness – percentage that know to report violations
  • Readability – Flesch reading score
  • Operationalization – percentage that believes that the organization actually adheres to the Code
  • Organizational Alignment – percentage that believe that the Code accurately reflects the true values of the organization
  • Personal Alignment – percentage that believe that the Code is aligned with their personal values
  • Reporting – percentage that believe that Code violations are actually reported
  • Questions – number of questions received
  • Incidents – number of reported or discovered incidents of violation