Statistical Forensics

Mathematics is the art of giving the same name to different things. -Jules Henri Poincaré (1854-1912).

objects

Statistical forensics is the use of mathematics in the analysis of evidence.

Statistical forensics is most useful in the investigation of alleged scientific and financial misconduct.

It can assist in exonerating the falsely accused, and it can be used to identify wrongdoing. But in most cases it merely discovers that wrongdoing has occurred.

The wrongdoers are individuals or companies that choose to intentionally misrepresent data given to government agencies in support of grant applications, license applications, patent applications or publications. They may be employees that have gone rogue or they may be a product of the culture of a firm in a competitive industry.

Instances of academic and scientific misconduct are, unhappily, no longer rare.

Scientific misconduct on data collection and collation is a threat to the health, safety and general well being of the public. The most damaging impact is in research related to health care systems, the pharmaceutical industry and weapons development.

Statistical forensics is finding its way into law courts in the form of forensic witnesses who serve either the plaintiff or the defendant by untangling the mass of allegations that can be made concerning the collection of research data.

For those who may be unfamiliar with the issues involved in the detection of deceptive statistical practices, please see this straightforward and enlightening quotation from the US Department of Health and Human Services, Office of Research Integrity, Department of Technical Assistance. Note that this was published about events around the turn of the century.

Statistical Forensics: Check Rightmost Digits for Uniform Distribution. Numbers are often recorded beyond the repeatability of the experimental procedure. When counts or measurements are recorded to higher precision than can be repeated in replications of an experiment, the rightmost digits of the recorded numbers have little biological meaning. Consider a count of radioactivity for a biological preparation, for example, 5179. In a recount of the sample, or in a replication of the assay, it is unlikely that the rightmost digits will be the same. Thus, with three repetitions, 5179, 5118 and 5134 could be expected.

The rightmost digits of these three numbers differ. Thus xx 79 differs from xx 18, and, in turn, both differ from xx 34.

In large samples of numbers, such rightmost digits often occur with the same frequency, like lottery digits where each of the digits 0, 1, 2, . . . , 9 has the same expectation. Statistically speaking, rightmost digits are approximately uniformly distributed in many circumstances.

In one ORI case, the respondent’s notebook contained fabricated counts as well as un-fabricated counts. For the fabricated counts the radioactive spots on the experimental sheets had not been excised and hence could not have been counted in the scintillation counter. The un-fabricated counts were supported by counter tapes.

Investigators from ORI’s Division of Investigative Oversight (DIO) compared rightmost digits of fabricated and un-fabricated counts. The fabricated digits differed significantly from uniform. The un-fabricated digits did not so differ. (The respondent accepted voluntary exclusion from receiving Federal funds for 3 years.)

In another case, one column of a published table of numbers was not supported by notebook data. DIO investigators found that the rightmost digits of the unsupported column differed significantly from uniform. The rightmost digits of the supported columns did not so differ. (The paper was retracted, and in a related Department of Justice settlement, the Government recovered over $1 million from two universities.)

To succeed in fabricating data, the fabricator must make the leftmost digits exhibit the desired biological magnitudes. Rightmost digits, given little thought, may be subject to personal preferences of the moment, and hence not uniform. Even when instructed to “make up” numbers with uniform digits, many subjects appear unable to do so. (See “Data Fabrication: Can people generate Random Digits?” J.E. Mosimann, C.V. Wiseman and R.E. Edelman, Accountability in Research, 4, 31-55, 1995).

In cases of scientific misconduct, un-scientific details, like rightmost digits, are worthy of attention.”

This is from a web presented statement, updated February 2002, by the US Department of Health and Human Services, Office of Research Integrity, Department of Technical Assistance. For further information, go to their website.

A more recent case is particularly poignant. The researcher is a well-qualified scientist,  Han, Dong-Pyou. The following is taken from the Findings of Research Misconduct and Administrative Actions 2013 DEPARTMENT OF HEALTH AND HUMAN SERVICES,    Office of the Secretary.

“Findings of Research Misconduct

AGENCY: Office of the Secretary, HHS.

ACTION: Notice.

SUMMARY: Notice is hereby given that the Office of Research Integrity (ORI) has taken final action in the following case:

Dong-Pyou Han, Ph.D., Iowa State University of Science and Technology: Based on the report of an inquiry conducted by the Iowa State University of Science and Technology (ISU), a detailed admission by the Respondent, and additional analysis conducted by ORI, ORI and ISU found that Dr. Dong-Pyou Han, former Research Assistant Professor, Department of Biomedical Services, ISU, engaged in research misconduct in research supported by National Institute of Allergy and Infectious Diseases (NIAID), National Institutes of Health (NIH), grants P01 AI074286, R33 AI076083, and U19 AI091031.

ORI and ISU found that the Respondent falsified results in research to develop a vaccine against human immunodeficiency virus-1 (HIV-1) by intentionally spiking samples of rabbit sera with antibodies to provide the desired results. The falsification made it appear that rabbits immunized with the gp41-54 moiety of the HIV gp41 glycoprotein induced antibodies capable of neutralizing a broad range of HIV-1 strains, when the original sera were weakly or non-reactive in neutralization assays. Falsified neutralization assay results were widely reported in laboratory meetings, seven (7) national and international symposia between 2010 and 2012, and in grant applications and progress reports P01 AI074286-03, -04, -05, and -06; R33 AI076083-04; U19 AI091031-01 and -03; and R01 AI090921-01. Specifically:

a. Respondent falsified research materials when he provided collaborators with sera for neutralization assays from (i) rabbits immunized with peptides from HIV gp41-54Q (and related antigens HR1-54Q, gp41-54Q-OG, gp41-54Q-GHC, gp41-54Q-Cys and Cys-gp41-54Q) to assay HIV neutralizing activity, when Respondent had spiked the samples with human IgG known to contain broadly neutralizing antibodies to HIV-1; and (ii) rabbits immunized with HIV gp41-54Q to assay HIV neutralizing activity, when Respondent had spiked the samples with sera from rabbits immunized with HIV-1 gp120 that neutralized HIV.

b. Respondent falsified data files for neutralization assays, and provided false data to his laboratory colleagues, to make it appear that rabbits immunized with gp41-54Q and recombinant Lactobacillus expressing gp41-64 (LAB gp41-64) produced broadly reactive neutralizing antibodies, by changing the numbers to show that samples with little or no neutralizing activity had high activity.

Dr. Han has entered into a Voluntary Exclusion Agreement and has voluntarily agreed for a period of three (3) years, beginning on November 25, 2013:

(1) to exclude himself from any contracting or subcontracting with any agency of the United States Government and from eligibility or involvement in nonprocurement programs of the United States Government referred to as “covered transactions” pursuant to HHS’ Implementation (2 C.F.R. Part 376 et seq) of OMB Guidelines to Agencies on Governmentwide Debarment and Suspension, 2 C.F.R. Part 180 (collectively the “Debarment Regulations”); and

(2) to exclude himself voluntarily from serving in any advisory capacity to the U.S. Public Health Service (PHS) including, but not limited to, service on any PHS advisory committee, board, and/or peer review committee, or as a consultant.”

A popularization of the subject is published in “False positives: fraud and misconduct are threatening scientific research,” The Guardian, Thursday 13 September 2012 18.12 BST.”

It is online with readers’ commentary at: http://www.guardian.co.uk/science/2012/sep/13/scientific-research-fraud-bad-practice

For a general description of our company, see: “Home“.

Statistical skeptics often find humor in the use of numbers by politicians. “Politicians are our greatest friend. Who cannot make money by speculating against a politician’s promise?”  So said The Gnome of Zürich in response to British Minister Harold Wilson’s speech devaluing the British Pound by 14% in 1967. The Prime Minister stated:  “From now on, the pound abroad is worth 14 per cent or so less in terms of other currencies. That doesn’t mean, of course, that the Pound here in Britain, in your pocket or purse or in your bank, has been devalued.”

Scholars who are attending our clinics should check their visa requirements: see actvisas.com.au

Copyright 2002-2021

Universal Measurement Systems Pty Ltd is an Australian company serving clients through mathematically based consulting services.