Determining effectiveness has been on my mind in large part since the release of the Department of Justice’s (DOJ) Evaluation of Corporate Compliance Programs (Evaluation). Obviously the new by-word from the Evaluation is operationalization but a key in determining operationalization is determining your compliance program effectiveness. I put that question to Vincent DiCianni, CEO and founder of Affiliated Monitors and Eric Feldman, SVP of Affiliated Monitors recently.
Feldman began by explaining that you need to consider both outcomes and outputs. Outcomes will show you the results of specific actions, such as investigations and conclusions to them. DiCianni added that the numbers are attractive because they can form a “straight line” about your compliance program is function. Yet DiCianni cautioned the numbers only give you one view of a compliance program. You also need to consider the qualitative side of the equation.
This is where outputs are equally important as the form the qualitative portion of determining compliance program effectiveness. More importantly you cannot conflate the two. Feldman explained that hotline data is good example, so if your number of hotline reports drops dramatically, the company may well believe their compliance program is effective. However, Feldman cautioned this could be a tenuous conclusion “because just as easily one could conclude that your culture has taken a turn for the worse, that employees are afraid of retaliation, they don't have faith and trust in the anonymity of your hotline system and therefore they're just not reporting, but things are still going on. In fact, there may be more activity going on”.
Some important consideration are such softer measures as how employees feel about whether the company is committed to a speak-up culture. Feldman noted that by interviewing employees, you can determine if they feel “comfortable going to their managers and if their managers are involved, going to upper level management, Ethics and Compliance Office, or a corporate reporting hotline if and when they see misconduct, or do they mind their own business and look the other way because they're afraid something will happen to them?” The best way to make that determine is through in person interviews.
Another key way to determine if you have any effective compliance program is to see if there is a correlation about what a company says on paper on its vision, mission and values around compliance. Here a key metric is performance incentives, bonuses, promotions and assignments. Feldman explained you must ascertain if the financial packages are based solely on hitting your numbers “or are there elements that balance out the financial measures with ethical measures, integrity measures. For example, is a manager is effectively disseminating the ethics message and building an ethical culture in his or her work group and are they rated on that in a performance appraisal, that should be part of their bonus system.”
One valuable resource to assist the compliance practitioner in this task is entitled “Measuring Compliance Program Effectiveness: A Resource Guide”, and was issued by the Health Care Compliance Association (HCCA) and the Department of Health and Human Services, Office of Inspector General (OIG) in March 2017. Although it was publicly released after the Justice Department Evaluation, it was drafted prior to that documents release and hence did not have the benefit of the DOJ’s thinking on measuring compliance program effectiveness.
The document is an excellent resource on not only “what to measure” but equally important “how to measure” the seven elements of a compliance program as detailed in the US Sentencing Guidelines. While the focus is towards the health care industry, the concepts are broad enough for any industry or compliance practitioner to use to determine the effectiveness of their compliance program. Did I mention the cost - it is available at no charge on the OIG website.
Once again, although focused on health care compliance, the Resource Guide is practical for the non-health care compliance professional. Further, it ties into many of the concepts articulated in the Evaluation. For example, in the Evaluation, Prong 2. Senior and Middle Management, the following questions appear under the heading Oversight – What compliance expertise has been available on the board of directors? Have the board of directors and/or external auditors held executive or private sessions with the compliance and control functions? What types of information have the board of directors and senior management examined in their exercise of oversight in the area in which the misconduct occurred?
In the Evaluation under Prong 3. Autonomy and Resources, the following questions appear under the heading Funding and Resources – How have decisions been made about the allocation of personnel and resources for the compliance and relevant control functions in light of the company’s risk profile? Have there been times when requests for resources by the compliance and relevant control functions have been denied? If so, how have those decisions been made?
These are a just couple of examples of how a compliance professional can begin to think through the questions laid out by the DOJ in its Evaluation. Moreover, by using the Resource Guide, you will be able to more fully determine the operationalization of your compliance program. The stated purpose is to give compliance professionals “as many ideas as possible, be broad enough to help any type of organization, and let the organization choose which ones best suit its needs.” Yet it is decidedly not a checklist but rather allows any Chief Compliance Officer (CCO) to assess the effectiveness (and operationalization) of their program.
It also allows the tailoring and measurement of how you manage your company’s risks. As the Resource Guide states, “The frequency of use of any measurement should be based on the organization’s risk areas, size, resources, industry segment, etc. Each organization’s compliance program and effectiveness measurement process will be different.”
DiCianni concluded by emphasizing the need for both a quantitative and qualitative approach to measuring compliance program effectiveness. Numbers are important but they only tell part of the equation. He stated, “Both are very important, but I think without having consideration of both sides of the equation, I do not will obtain a full understanding of how effective compliance program is in its operation.”
Three Key Takeaways
For more information on how an independent monitor can help improve your company’s ethics and compliance program, visit this month’s sponsor Affiliated Monitors at www.affiliatedmonitors.com.