Why the ATF's Firearm Testing Procedures
are Scientifically Invalid
by Len Savage
Owner, Historical Arms LLC
Following on from the article and information written below, there is news!
Click on the thumbnail to right to see relevant document.
After very aggressive action by JPFO and others, the BATFE decided to back down.
This article has been re-formated Feb 2012. When pictures expand - note - there is an enlargement option icon lower right on most.
With the amount of documentation now publicly available it has become obvious that there is a serious problem within the Firearms Technology Branch (FTB) of the Bureau of Alcohol, Tobacco, Firearms, and Explosives (ATF).
The technicians who work at FTB testify before the courts as "experts" on the technology of firearms. They may or may not have any real-world or industry experience. It is a fact that no technician at FTB has ever held a federal firearms license or ever designed a firearm. Unfortunately the problem runs even deeper than that. The recent public exposure of an incompetent FTB technician, Michael J. Cooney (U.S. v. Glover), which resulted in the dismissal with prejudice of a federal prosecution of an innocent citizen, raises troubling questions about the legal validity of past prosecutions in which Mr. Cooney testified, and possibly those of other FTB technicians.
Congress has given the ATF the task of "classifying" firearms - for instance, determining whether a firearm is a common, semi-automatic that fires one shot with each trigger pull or whether it is a machine gun, designed to fire multiple rounds on one trigger pull (full auto). Numerous gun owners and gun makers have been bankrupted or imprisoned because the ATF stated that their firearms were "illegal machine guns" rather than semi-autos. If the ATF's classifications were accurate, then this would just be a matter for lawyers and lawmakers. But there is ample evidence – and not only in the case of Mr. Cooney -- to indicate that the ATF's classifications are arbitrary and inaccurate. The ATF seemingly does not employ consistent testing criteria and standards.
The core issue is the methodology (or lack of) in making firearm classifications. The ATF does not employ the time-honored and well-honed methods of scientific inquiry.
The Scientific Method
Common Mistakes in Applying the Scientific Method:
Most high school and college science text books warn of common mistakes when applying the scientific method. I will quote four that apply to FTB.
♦ "When testing a hypothesis (educated guess), the scientist or technician may have a preference for one outcome or another, and it is important that this preference not bias the results or their interpretation."
♦ "The most fundamental error is to mistake the hypothesis for an explanation of a phenomenon, without performing tests. Sometimes "common sense" and "logic" tempt us into believing that no test is needed." (Remember the SGMB semi-auto and the safety selector?)
♦ "Sometimes, however, a scientist or technician may have a strong belief that the hypothesis is true (or false), or feels internal or external pressure to get a specific result. In that case there may be a psychological tendency to find something wrong." (For an example of this view an actual ATF test-firing on the DVD "BATFE Fails the Test," available direct from The JPFO Store. Without any disassembly or detailed examination, an FTB "expert" insisted a semi-automatic rifle was a "machine gun" solely because it would occasionally fire more than one round on a single trigger pull; a few minutes examination of the gun's components showed that the gun's aging parts were merely malfunctioning.)
♦ "Another common mistake is to ignore or rule out data that does not support the hypothesis". (Example: In testing of the BM-3000 upper, FTB stated in documents that its jury-rigged device fell apart in half of the tests. When evaluating the results, however, the FTB simply chose to ignore that half of the tests. In another case, when the ATF's duct-taped and plastic cable-tied Inter Ordnance PPSH parts blew apart on the second round fired, the FTB chose to completely ignore the fact that the ATF-created device exploded. It fired twice, therefore Inter Ordnance was responsible for a manufacturing and selling an "illegal machine gun."
Science is distinguished from other forms of inquiry by its requirement of systematic experimentation. The scientific method is designed to reduce the influence of individual, social, or institutional bias on experimental findings.
In short the scientific method is a process in which scientists or technicians try to build an accurate (that is reliable, consistent, repeatable, and NON-arbitrary) representation of the world or things in it. It has four steps that must be followed:
1. Observe and describe a phenomenon (example: A researcher thoroughly examines an FN-FAL, which is suspected of being a machine gun).
2. Formulate a hypothesis (educated guess) to explain the phenomenon (example: The presence of a full-auto safety selector installed in the lower receiver of a FN-FAL-type firearm could cause the firearm to fire fully automatic, and thus meet the statutory definition of a machine gun).
3. Use the hypothesis to predict the existence of other phenomenon (example: The lower receiver had to be modified or redesigned to make the firearm function in full-auto mode).
4. Perform experimental tests of the predictions that can be replicated by other independent experimenters using written testing protocols and uniform evaluation standards (example: Test fire the suspected FN-FAL on the range using standardized procedures and various types of ammunition, keeping careful notes and video-recording the testing process).
The ATF Method
While FTB does use some of the above steps, tech branch "experts" tend pick and choose indiscriminately among the four steps and also to perform the steps in a faulty manner. For instance, they may well observe and describe, but if they have no real world or industry experience, do they know what they are looking at or looking for?
Example: I submitted an SGMB semi-auto rifle to FTB for classification. They observed that if they removed the safety selector, the sear did not disconnect with every pull of the trigger. (A safety selector, usually called a "safety," is used to render the firearm in a condition to fire if the trigger is pulled, or conversely to prevent it from firing. The sear is the component that releases the hammer.) Upon cursory examination, this configuration appeared to enable the firearm to fire multiple times on one trigger pull, and therefore the ATF declared it a "machine gun" -- without ever test firing it.
They failed, however, to observe that the firearm had a second disconnector that operated independently of the trigger pull. (A disconnector is a component that prevents full-auto function; it is a part that distinguishes a semi-auto from a machine gun.) Operating off of the bolt carrier movement, this second disconnector prevented full-auto fire. Due solely to the ATF's own lack of firearms knowledge and incomplete testing methods, they wrongly classified this SGMB as a machine gun - even though in real-world practice it was incapable of and not designed for full-auto fire.
Letter to Len Savage (pdf format)
Response by Len Savage (pdf format)
More BATFE Letters (pdf format)
The lack of consistent testing standards and criteria by FTB is well documented. When a firearm is submitted to FTB for classification firearms makers can receive no clear answer to their questions as to what tests may or may not be performed. The ATF simply has no standard test procedures.
For instance, the FTB often attempts to demonstrate that a semi-automatic version of a firearm could be "readily restorable" into a machine gun.[1] If the gun is "readily restorable" it may be illegal to possess or manufacture without special licensing.
To prove how "readily restorable" a firearm is, one of the agency's "experts" may do something as quick and simple as perform a test fit of full-auto components into the semi-automatic firearms – about 15 minutes work. An "expert" in another case may give a similar firearm to a machinist and allow him eight hours in a modern machine shop and try to make the firearm into a machine gun. Yet each expert may testify in court that he performed reliable tests to determine that a semi-automatic weapon was "readily restorable" -- that is, quickly and easily convertible into a possibly illegal weapon.
No owner or manufacture has any idea which standard will be applied to his own firearm. Yet freedom and/or economic survival may hang on the ATF's decision.
Plastic cable ties, duct tape, and shoelaces
While FTB criteria for testing and classifying firearms can flip-flop at will, that is only one aspect of the ATF's violation of accepted scientific method. At times, ATF methodology becomes downright bizarre.
For instance, in attempting to "prove" that a semi-automatic firearm is, or can easily be made, full-auto, ATF "experts" have been known to attach a variety of devices to the gun being tested. They commonly, for instance, fasten new parts to the firearm or remove parts from the firearm, then hold the resulting device together with duct tape, plastic cable ties, or small metal bars before test firing it. They use these aids because otherwise the components of the jury-rigged test weapon will not hold together on their own. Such a device would be useless in the real world, yet the ATF freely uses these Rube Goldberg contraptions to "prove" that a weapon is illegal, and that the original maker or owner of the firearm is committing a federal crime!
These strange ATF-created lab contraptions can also be so dangerous that the testers hide behind barriers to protect themselves against exploding firearms. Yet ATF agents may still tell a jury that such a weapon is a usable "machine gun."
On September 30, 2004, the ATF sent a letter to a Mr. Brian Blakely stating that if a person tied an ordinary household item (in this case, a shoestring) to the trigger of a firearm in order to induce that firearm to shoot multiple rounds on one trigger pull, then that household item was -- in and of itself -- a machine gun. (According to law and BATFE interpretation, any components designed to induce a weapon to fire in full-auto mode are themselves machine guns, even if they are inert and incapable of firing anything when standing alone). But if that is true, then are not the duct tape, plastic cable ties, and small metal bars also machine guns? Are these ATF-applied devices not intended to convert otherwise ordinary semi-automatic firearms into machine guns? And is not the ATF - rather than the gun maker or owner -- the culprit in that case?
We have seen that the ATF uses arbitrary testing criteria and faulty methods. But worse, strange techniques like the above violate yet another aspect of the scientific method: The methods are not chosen to discover the facts. They are designed to produce whatever results fit the ATF's hypothesis - regardless of the facts.
Consider the example of aftermarket upper receivers for the M-11/M-10. This family of firearm is normally magazine-fed via a pistol grip in its lower receiver. If the firearm is full-auto, the lower receiver is considered a machine gun, while the standard upper receiver is not a machine gun.
The industry has produced aftermarket upper receivers that can also be used to feed rounds into the chamber of the M-11 or M-10. Some of these are magazine-fed. Some are belt-fed.
The FTB has decided – without testing – that the magazine-fed upper receivers are not machine guns. For instance, Andrew Smith's and Capt. Monty Mendonhall's "Soumi" upper, or my own Historic Arms "Calico mag" upper are not classified by the ATF as machine guns.
On the other hand, the FTB has decreed that the belt-fed upper receivers, like Socomm’s M-11/RPD and my own Historic Arms BM-3000 upper, are machine guns, in and of themselves.
Why? you might ask. While these components may look different, they are all replacement upper receivers for the M-11/M-10. They are all open- bolt in operation, and all of them incorporate a device that feeds ammunition directly into the upper receiver.
The most astonishing thing is that RPD-style uppers were "proven" to be machine guns after being subjected to a test in which they were clamped down in a shooting vice with a chain, and copious plastic cable-ties and a small metal bar were added to capture the recoil spring system, and pull back the cocking handle and release. The firearm did fire, although it was unsafe, uncontrollable, and impractical (unless you can tote a vise and shooting table around in your trench coat).
Now, it would actually be far easier to make a magazine-fed upper to fire as a "machine gun" using this strange and dangerous method of testing. DO NOT EVER TRY THIS, it is unsafe, and could kill you! But it is a fact: Using the ATF's own testing methods, the "non-machine gun" component would perform just like the "machine gun" component – in fact, would do so more easily. The FTB is aware of this; I've discussed it with their personnel. Yet they persist in proclaiming that the belt-fed uppers are "machine guns" and the magazine-fed uppers non-machine guns.
Why? No one can be sure of their motivations, but based on the fact that they tested the belt-feds and not the magazine-feds, it appears they simply "knew" (as any TV viewer who ever watched a late-night war movie "knows") that anything belt-fed must be a machine gun. Therefore they devised a test to prove what they already assumed.
|
|
|
Conclusions
Inconsistent testing standards, faulty testing methods, and tests seemingly designed to prove a foregone conclusion, are a major inconvenience and financial problem for a manufacturer. For a defendant in a felony firearms case the ATF's lack of scientific method could mean a very long prison sentence for an innocent citizen, or a "get out of jail free card" for a real criminal.
Without clearly defined testing criteria and standards, and without test results available to all interested parties, in every case (another chronic problem), the FTB will continue to contradict itself and produce faulty judgments and faulty evidence in legal cases.
The FTB, the American firearms industry, prosecutors, and defense teams all need a published and readily available procedural manual of testing criteria and standards. And those procedures must rigorously follow scientific method.
The current lack of standards and the inability to replicate test results invalidates FTB conclusions at best. At worst, the ATF's methods and resulting claims could be construed as conspiracy to deny civil rights to American citizens.
The FTB resists the creation of industry oversight committee that would allow genuine experts - gunsmiths, firearms manufacturers, and federal firearms license holders - to have input into testing procedures. Currently, those whose run afoul of faulty ATF methods and resulting accusations of illegal activity and/or financial hardship have no means of redress other than a civil suit. A method of checking and balancing ATF behavior needs to be established to prevent institutional bias from overwhelming the justice system and the firearms industry with bad science and false accusations.
[1] "Readily restorable" is a term used in the National Firearms Act of 1934 (NFA) and the Gun Control Act of 1968 (GCA). The term was never technically defined when the laws were written or enacted. In practice, the term "readily restorable" was written into the law to forbid any firearm design that could be converted from semi-automatic to full-auto with "a quick modification." "Quick modification" can be interpreted by FTB to mean "15 minutes with simple hand tools" or "eight hours in a modern machine shop" depending on the whim of the examiner.
References
1. Wilson, E. Bright. "An Introduction to Scientific Research" (Mcgraw-Hill, 1952)
2. Kuhn, Thomas. "The Structure of Scientific Revolutions" (Univ. Of Chicago Press, 1962).
3. Barrow, John. "Theories of Everything" (Oxford Univ. Press, 1991).
4. nsrl.rochester.edu