A Meta Analysis of Materiality Studies

The Supreme Court and the Public Company Accounting Oversite Board (PCAOB) has said that an amount is material if there is a substantial likelihood it will influence a reasonable investor’s judgment. The American Institute of Certified Public Accountants (AICPA) has said that an amount is material if there is a substantial likelihood it will influence a reasonable user’s judgment. The Financial Accounting Standards Board (FASB) has refused to define materiality. The Securities and Exchange Commission (SEC) has said that qualitative factors can make even small amounts material. Reasonable implies a consensus of opinion. This article is a meta-analysis of 31,155 materiality decisions made by 335 cohorts in 48 studies with the objective of defining what is reasonable. A cohort is a group of like individuals faced with a common materiality decision. Materiality in this study is measured as a percentage of net income. The mean threshold of materiality is 7.84% and the median is 6.81%. Both thresholds are substantially higher than the often-discussed threshold of 5.0%. A quarter of the participants in these studies set the threshold of materiality at 11.90% and the threshold for a statistically significant difference from the consensus is 17.51%. Ultimately, materiality will be decided through civil and criminal litigation. Finders of fact, usually jurors , will be asked to determine what a reasonable investor would conclude. Few jurors have the training and experience of investors , so without context, they can only guess what a reasonable investor would conclude. This study provides that context.

"Misstatements, including omissions, are considered to be material if there is a substantial likelihood that, individually or in the aggregate, they would influence the judgment made by a reasonable user based on financial statements." The AICPA in their Standards of Attestation (AICPA 2019b) says a practitioner may assume that intended users (a) have a reasonable knowledge of the subject matter, (b) understand appropriate levels of materiality, and (c) understand any inherent uncertainties involved in measuring or evaluating the subject matter.
"The Supreme Court of the United States has held that a fact is material if there is a substantial likelihood that the… fact would have been viewed by the reasonable investor as having significantly altered the 'total mix' of information made available." In formulating their test for materiality, the Supreme Court noted that, "Some information is of such dubious significance that insistence on its disclosure may accomplish more harm than good." They further state that if the standard of materiality is unnecessarily low, management might simply bury shareholders in an avalanche of trivial information to avoid litigation (TSC Industries v. Northway 1976).
In TSC Industries, the Supreme Court rejected the materiality formulation of the appeals court below (Northway v. TSC Industries 1975) which said that an item is material if a reasonable shareholder might consider an item important. It also rejected the holdings of appeals courts in Gerstle v. Gamble-Skogmo, Inc. (1973) and Smallwood v. Pearl Brewing Co. (1974) which held that an item is material if a reasonable man would attach importance to the fact misrepresented or omitted. Instead, the Supreme Court adopted the "substantial likelihood" test. The Financial Accounting Standards Board (FASB) takes the position that materiality is a legal issue that can be changed by legislation or the courts. Therefore, it declined to provide authoritative guidance (FASB 2015).

SEC Standards SAB-99
SEC Staff Accounting Bulletin 99 (SAB-99 1999) complicates the assessment of materiality by citing five "qualitative factors" which might cause small misstatements to reach the threshold of materiality including: (a) changing a loss into a profit, (b) masking a change in earnings or other trends, (c) hiding a failure to meet analysts' expectations, (d) affecting compliance with loan covenants or (e) impacting executive compensation. The SEC's fear is that companies will use the concept of materiality to manage earnings. There is a substantial body of literature on earnings management (Legoria 2012).
Courts have long recognized that materiality is not simply a numerical issue. Since courts have said there are no bright lines in assessing materiality, the issue might turn on management intent. A decision to buy or sell an asset, close a plant or run it overtime, to incur an expense or defer that expense might just be prudent management. In doing so, a company might legitimately consider an item immaterial and stumble across one of the qualitative factors in SAB-99. The securities law issue is whether this constitutes an attempt to make a prohibited misleading statement (SEA 1934).
Direct evidence of intent is rarely available. Given the difficulties of determining intent, it seems unrealistic to require, as does SAB-99, that regulators, auditors and others should be able to divine the intent of management in making materiality judgments (Zabel 2002).
An even greater problem with SAB-99 is that it invites jurors, as finders of fact, to speculate as to whether and under what circumstances the factors listed in SAB-99 would lead to a "substantial likelihood" that they would influence an investor's judgment.
One might argue it is simply common sense that changing a loss to a profit is material. But common sense is not evidence. Jurors cannot be asked to speculate on what constitutes a "substantial likelihood" that an item significantly alters the total mix of information without sufficient credible evidence.

Evidential Standard
Arguably, the SEC holds itself out as an expert in securities matters. The Federal Rules of Evidence says that expert testimony must be based on sufficient facts or data (FRE 2011 702(b)). Courts are the gatekeepers of admissibility. The admissibility of all expert testimony is governed by Federal Rules of Evidence (FRE 2011 104(a)) which says a proponent has the burden of establishing that pertinent admissibility requirements are met (Daubert v. Merrel Dow Pharmaceuticals, Inc. 1993;Bourjaily v. United States 1987). These evidentiary standards apply to both the SEC and private litigants.
Is it enough for the SEC to argue that the tests listed in SAB-99 would influence someone's judgment? If SEC assertions about SAB-99 criterion for making an item material are not supported by facts or data in the form of credible studies, they are inadmissible as evidence.
As of this writing, no studies directly on point for the five SAB-99 criteria have been found. In considering studies to support SAB-99 it is important to differentiate between studies that purport to document earnings management and studies that document the circumstances in which such alleged earnings management is a substantial factor in influencing an investor's or user's judgment.

Standards for the Finders of Fact
The Supreme Court chose to use the term investor in determining materiality. It could have chosen to use the term someone, anyone or a reasonable person, but it did not. It said an item is material if there is a "substantial likelihood" that the… fact would have been viewed by the reasonable investor as having significantly altered the 'total mix' of information made available (TSC Industries v. Northway 1976;Basic, Inc. v. Levinson 1988).
Investors often have skill and learning that others do not, and may have a risk tolerance that others do not. This makes it difficult, though not impossible, for the ordinary juror to determine how a reasonable investor would assess information.
The AICPA says a fact is material if there is a "substantial likelihood" that it would influence the judgment made by a reasonable user (AICPA 2019a, AICPA 2019b). The AICPA Standards of Attestation (AICPA 2019b) says a practitioner may assume that intended users have a reasonable knowledge of the subject matter. Such an assumption cannot be made for a typical juror. A typical juror might not even be able to read a financial statement. That makes it difficult for jurors to assess whether an item constitutes a material change to a financial statement.
Given the distinction between an investor or user on the one hand and a typical juror on the other hand, it is important that jurors be given a broad context within which they can judge the facts.

Methodology
Materiality is measured in a number of ways. For example, Petroni (1996) measures materiality as a function of assets, Madden (1972) measures materiality in terms of sales, Kuene (2012) models materiality decisions as a function of audit fees, the number of analysts following a company and other qualitative factors, and Cho (2003) measures materiality as a function of the change in stock price. There are also a large number of studies that discuss materiality and risk but never quantify their findings (Shawver 2008;Popova 2018). This meta-analysis analyzes materiality as a function of net income.
Meta-analysis integrates data from a number of studies. Table 1 Source Material lists the 48 studies used in this meta-analysis. Data is not always reported the same way. Data must be standardized for analysis (Whitlock, 2005). Reasonable assumptions and estimates are made to convert source material findings to a standardized form. For example, where ranges are reported, the midpoint of those ranges are used. One study measured materiality in terms of Earnings per Share (EPS) (Rose 1970) and changes in EPS were used as a proxy for changes in net income. Interpretation of graphs was required for some studies (Messier 1983;Hofstedt 1977). Pretax income was converted to after tax income for other sources (DeZoort 2003;Nelson 2004;Wright 1997;Hofstedt 1977) using average corporate tax rates (Compustat 2006). One source reported materiality via regression analysis (Waters 1997). Other sources provided data in the form of frequency within ranges (Chewning 1998;Chewning 1989;Fesler 1989;Morris 1984). One source reported data from six interest groups (Pattillo 1975) and the number of individuals in the study, so data were apportioned to the groups as recommended by DeCoster (2004). The balance of the sources either explicitly provided materiality as a percentage of net income or provided information to compute it. The threshold of materiality was computed as a weighted average across studies as shown in equation (1).
Σ n i Where M is mean materiality, n i is the number of materiality decisions in cohort i, and M i is the materiality of cohort i.
The standard deviation of materiality is given as equation (2). σ = (Σ (n i x (M i -M) 2 ) / ((Σ n i ) -1))) .5 Where n i is the number of materiality decisions in cohort i, M i is the materiality of cohort i, and M is the overall mean materiality computed in equation (1).

Findings
The mean threshold of materiality is 7.84% and the median is 6.81% across 31,155 material decisions made by 335 cohorts as shown in Table 2 Descriptive Statistics. There is little difference between materiality measured in an experiment such as a case study or questionnaire and materiality measured by an analysis of historical decisions. This is surprising because there are no consequences in an experiment and potentially significant consequences in real-world decision making. The mean threshold of materiality ranges from a low of 7.00% for managers and board members to a high of 10.76% for bankers and creditors as shown in Table 3 Materiality by Interest Group. Logic might dictate that bankers and creditors would be more sensitive than managers and board members and their materiality thresholds would be lower.
On the other hand, it could be that bankers and creditors have a more realistic and skeptical view of financial statements. The median threshold of materiality for bankers and creditors at 11.0% is also much higher than the median threshold for managers and board members at 6.73%. Differences in the threshold of materiality may seem small among interest groups when measured in terms of percentages. However, some of those differences are statistically significant as shown in Table 4 Perceptual Differences by Interest Group. The pooled standard deviation among interest groups was computed using equation (3).
Where σ ij is the pooled standard deviation, σ i is the standard deviation of interest group i, N i is the number of materiality decisions in interest group i, σ j is the standard deviation of interest group j, and N j is the number of materiality decisions in interest group j.
Equation (4) was used to compute Z values, where X i is the mean materiality of group i, X j is the mean materiality of group j and σ ij is the pooled standard deviation. . Z values are in the named columns Auditors, Bankers, etc. Statistical significance at the 0.1 level is denoted with an *, at the 0.05 level by ** and at the 0.01 level by ***.

Discussion
The AICPA has defined a fact as material if there is "a substantial likelihood" it would influence a reasonable user's judgment (AICPA 2019a, AICPA 2019b). The PCAOB and the Supreme Court define a fact as material if there is a "substantial likelihood" it would influence a reasonable investor's judgment (PCAOB 2010, TCS Industries v. Northway 1976. Arguably, there is a substantial difference between an investor and a user. Since investors have more at stake, it is reasonable to think they have more skill and learning than users. It is also likely they have a greater risk tolerance than users. The FASB has taken the position that materiality is a legal issue and has declined to define it. The AICPA, PCAOB and Supreme Court all reject a bright line test of materiality such as 3% or 5% in favor of a facts and circumstances test. The SEC (SAB-99 1999) has listed five factors which might make even quantitatively small items material. Among them are items that (a) change a loss into a profit, (b) mask a change in earnings or other trends, (c) hide a failure to meet analysts' expectations, (d) affect compliance with loan covenants or (e) impact executive compensation. While these factors may be important in the limit, the SEC provides no evidence as to the circumstances in which each of these factors would have a "substantial likelihood" of influencing the judgment of either a reasonable user or investor.
At some point, the swing from losses to profits is significant enough to be material, but what is that point? If a company with ten billion in sales shows a loss of $0.01 per share versus a profit of $0.01 a share, does that meet the "substantial likelihood" test? Or are both numbers so close to zero that it makes no difference?
How much of a change in trend is necessary to trip the conditions of SAB-99? Is it the difference between a 2% growth rate and a 3% growth rate? Is it the difference between 10% and 11%? Is it more or is it less? SAB-99 cites no studies that indicate the circumstances under which any of its five qualitative factors provide a "substantial likelihood" that they would influence an investor's judgment.
Whether something is material is a question of fact. Questions of fact are decided by jurors, not the SEC, courts or plaintiffs. Without a body of evidence to provide context, jurors are left to speculate about the nature of materiality and what SAB-99 factors mean. Worse, there is the risk that jurors might consider SAB-99 factors bright line tests in contravention of the Supreme Court's holding in TSC Industries v. Northway (1976).
Without studies to determine whether and when SAB-99 factors create a "substantial likelihood" of influencing a reasonable investor, SEC opinions fail to meet the test of admissibility set forth in Federal Rules of Evidence (FRE 2011 702(b), FRE 2011 104(a)). It is possible that the SEC will perform such studies in the future and it is possible that such studies will illuminate whether and when their five factors create a "substantial likelihood" that they will influence a reasonable investor's decision. Until then, the Federal Rules would seem to bar jurors from speculating as to their evidential weight. Future researchers could determine whether and under what circumstances the five factors in SAB-99 would influence a reasonable investor's judgment.
What then can the SEC, courts, juries and private litigants rely on? There is a large body of peer reviewed literature that provide context for what reasonable investors and users would have done in similar circumstances.
This study is a meta-analysis of 48 studies that measure 31,155 materiality decisions under a variety of circumstances as a function of net income. Participants in these studies had at least some of the skill and learning of users, if not that of investors. This data is the type of evidence contemplated by the Federal Rules of Evidence (FRE 2011 702(b), FRE 2011 104(a)).
While not setting a bright line threshold of materiality, this study provides jurors with context for reaching their own materiality decision. It is for a jury to decide whether a given item creates a "substantial likelihood." Perhaps a "substantial likelihood" only exists where there is a statistically significant difference between the consensus opinion as to materiality and the materiality decision which is the subject of given litigation. Again, this is a matter for the jury to decide.

Conclusion
An item is material if there is a "substantial likelihood" it would influence an investor's or a user's judgment. The courts and other authorities have rejected any bright line test of materiality. That leaves the materiality in any particular circumstance to the finders of fact, that is to jurors. Few jurors have the skill and learning of either investors or users of financial statements.
The SEC in SAB-99 has attempted to introduce de facto rules of thumb as to when an item is material. Those rules of thumb are not evidence. To date, the SEC has yet to cite studies on point as to when and under what circumstances those factors create a "substantial likelihood" of influencing an investor's or a user's judgment. Studies can become evidence upon which an expert opinion may be based. Until then, an argument can be made that the SEC's factors will only invite the jury to speculate as to whether SAB-99 criterion constitute a bright line test in contravention to TSC Industries v. Northway (1976).
Jurors, as finders of fact, must be provided with substantial credible evidence regarding what is reasonable in the context of materiality. Otherwise, they are left to speculate as to what a reasonable investor would do. This study provides context so that jurors can reach their own opinion based on a substantial body of evidence.