Calibration Tap Drill Chart

Tap Drill Sizes (inch) for a 75% Thread

In general, you can find the tap-drill for any size 60-degree thread, Subtract one pitch length from the major diameter.

Formula: Major Dia. minus One Pitch Length equals tap-drill size

English Example for 3/8-16 thread: .375 – .0625 = .3125 tap-drill (5/16)

Metric Example for M6 X 1 thread: 6mm – 1mm = 5mm tap-drill

Tap Size Thread Form Tap Drill
0-80 UNF 3/64
1-64 UNC 53
1-72 UNF 53
2-56 UNC 50
2-64 UNF 50
3-48 UNC 47
3-56 UNF 45
4-40 UNC 43
4-48 UNF 42
5-40 UNC 38
5-44 UNF 37
6-32 UNC 36
6-40 UNF 33
8-32 UNC 29
8-36 UNF 29
10-24 UNC 25
Tap Size Thread Form Tap Drill
1/4-28 UNF 3
5/16-18 UNC F
5/16-24 UNF I
3/8-16 UNC 5/16
3/8-24 UNF Q
7/16-14 UNC U
7/16-20 UNF 25/64
1/2-13 UNC 27/64
1/2-20 UNF 29/64
9/16-12 UNC 31/64
9/16-18 UNF 33/64
5/8-11 UNC 17/32
5/8-18 UNF 37/64
11/16-11 UNS 19/32
11/16-16 UNS 5/8
3/4-10 UNC 21/32
3/4-16 UNF 11/16
7/8-9 UNC 49/64
7/8-14 UNF 13/16
1-8 UNC 7/8
1-12 UNF 59/64
1-14 UNS 15/16

Calibration Terms

ABERRATION: An optical phenomenon resulting from the failure of a lens or mirror to produce a good image.

ABSOLUTE PRESSURE: Actual pressure on a confined gas, irrespective of the atmosphere on the outside.

ABSOLUTE TEMPERATURE: The temperature measured from absolute zero as in the Kelvin and Rankine scales.

ABSOLUTE ZERO: The lowest temperature theoretically attainable (at which the kinetic energy of atoms and molecules is minimal).

ABSORPTION: (1) Loss of energy traveling through a medium. (2) Internal taking up of one material by another. (3) Transformation of radiant energy into other forms of energy when passing through a material substance.

ACCELERATION: Rate of change of velocity.

ACCOMMODATION: Changes in focus of the crystalline lens to adjust the eye of various object distances.

ACCURACY: (1) The closeness of agreement between a test result and the accepted reference value (ISO 5725-1). (2) Closeness of agreement between the result of measurement and a true value of the measured. Accuracy is a qualitative concept (VIM:1993).

A/D: Analog-to-digital conversion.

ADJUSTMENT(OF A MEASURING INSTRUMENT): The operation of bringing a measuring instrument into a state of performance suitable for its use.

ADSORPTION: Adhesion of one substance to the surface of another.

ALPHA: The current amplification factor when connected in a common base configuration.;

ALTERNATING CURRENT (AC): Current which reverses polarity at a uniform frequency.

ALTIMETER: An instrument that measures the height above ground.

AMBIENT TEMPERATURE: Temperature of the air in the immediate vicinity.

AMMETER: A meter that measures the flow of electrical current in amperes.

AMPERES: The basic unit of electric current adopted under the Systeme International d’Unites of measurements aimed at providing calibration.

CAPILLARITY: The characteristic of a liquid to be raised or depressed in a tube of small bore. This action is caused by a combination of cohesive, adhesive, and surface tension forces.

CAVITATION: Process in which small bubbles are formed and implode violently. This results in aggressive cleaning action in ultrasonic cleaners.

CELSIUS TEMPERATURE SCALE: A temperature scale based on mercury in glass thermometer with the freezing point of water defined at 0 degree C and the boiling point of water defined at 100 degree C, both under conditions of normal atmospheric pressure.

CENTER OF INSTRUMENT: The intersect point of the vertical, horizontal, and optical axis of a transit or similar instrument when perfectly calibrated.

CERTIFY: Provide evidence for or authorize officially.

CERTIFIED REFERENCE MATERIAL (CRM): Reference material, by a certificate, one or more of whose property values are certified by a procedure which establishes its traceability to an accurate realization of the unit in which the property values are expressed, and for which each certified value is accompanied by an uncertainty at a stated level of confidence (ISO Guide 30:1992).

CENTRIPETAL FORCE: The inward force on a body moving in a curved path around another body.

CGS SYSTEM: The common metric system of units (centimeter-gram-second).

CHARACTERISTIC: A property that helps to differentiate between items of a given population. Note: The differentiation may be either quantitative (by variables) or qualitative (by attributes).

CLINOMETER: An instrument used by surveyors in order to measure an angle of inclination or elevation.

COEFFICIENT OF LINEAR EXPANSION: The change in unit length in a solid when its temperature is changed 1 degree.

COEFFICIENT OF VOLUME EXPANSION: The change in unit volume of a solid when its temperature is changed 1 degree.

COHESION: The intermolecular force that holds together the molecules in a solid or liquid.

COLLIMATION: The process of aligning the optical axis of optical systems to the reference mechanical axes or surfaces of an instrument, or the adjustment of two or more optical axes with respect to each other.

COLLIMATOR: An instrument designed to produce collimated (parallel) rays of light usually equipped with displacement and tilt graticules.

COMPARATOR: An instrument for comparing some measurement with a fixed standard.

COMPLEX VIBRATION: The combination of two or more sinusoidal vibrations existing simultaneously.

COMPOUND: Two or more substances combined in definite proportions by weight and united chemically.

CONDENSATE: Steam which rises and cools to a liquid.

CONDUCTIVITY: The transmission of heat or electricity or sound.

CONFORMITY: Fulfillment of specified requirements.

CONTACTS: Elements used to mechanically make or break an electric circuit.

CONTINUOUS DUTY: A device able to operate continuously with no off or rest period.

CONTRACT REVIEW: Systematic activities carried out by the supplier before signing the contract to ensure that requirements for quality are adequately defined, free from ambiguity, documented, and can be realized by the supplier.

CONTRACTOR: Supplier in a contractual situation

CONVECTION: Transmission of energy or mass in a medium by movement of the medium itself.

CONVERSION CHART: Must be used to convert a parts per million reading to micromho or vice versa because the ppm scales are non linear and the micromho scales are linear. Because of the curve, there is no set ratio so one must refer to the chart.

CORRECTION: The value added algebraically to the uncorrected result of a measurement to compensate for systematic error.

CORRECTIVE ACTION: Action taken to eliminate the causes of an existing nonconformity defect or other undesirable situation in order to prevent recurrence

CREEP: The long term change in dimensional characteristics of a body under load, in an elastic force measurement device. This term refers to the change in reading which occurs when a constant load is applied for a period of time.

CRITICAL ANGLE: The angle between and at which there is neither refraction or internal reflection.

CRITICAL SIZE: For fissionable material, the minimum amount of a material which will support a chain reaction.

CRYOGENIC: The science of refrigeration pertaining to the methods for producing and measuring very low temperatures.

DAMPING: (1) The prevention of free swinging or vibration by some means, usually friction or resistance. (2) The dissipation of energy with motion or time.

DECAY TIME: The time required for the trailing edge of a pulse to decrease from 90 percent to 10 percent of its maximum amplitude.

DEFECT: Nonfulfillment of an intended usage requirement of reasonable expectation, including one concerned with safety.

DEGREE OF DOCUMENTATION: Extent to which evidence is produced to provide confidence that specified requirements are fulfilled.

DEMINERALIZATION: Removal of mineral constituents from water.

DEIONIZATION: Removal of ionized minerals and salts from a solution by a two phase ion exchange procedure.

DENSITY: The mass per unit volume. CGS unit: gm/cm

DI WATER: Deionized water.

DIAL INDICATOR: This is a mechanical lever system used for amplifying small displacements and measuring it be means of a pointer which transverses a graduated dial.

DIALYSATE METER: Verifies the total concentration of ionized salts in dialysate solutions used in hemodialysis or kidney equipment.

DIFFERENTIAL VOLTMETER: A voltmeter that operates on the potentiometric principle. The unknown voltage is compared to an adjustable calibrated voltage developed within the differential voltmeter.

DIFFERENTIATING CIRCUIT: A circuit in which the output voltage is proportional to the rate of change of the input voltage.

DIFFRACTION: When light passes sharp edges or goes through narrow slits the rays are deflected and produce fringes of light and dark bands.

DIGITAL VOLTMETER:An electronic voltmeter that gives readings in digits.

DIOPTER: A unit of measurement of the refractive power of a lens which is equal to the reciprocal of the focal length measured in meters.

DIRECT CURRENT (DC): A current with a constant polarity.

DISPOSITION OF NONCONFORMITY: Action to be taken to deal with an existing nonconforming entity in order to resolve the nonconformity.

DISTORTION: Any deviation from the desired waveform.

DOUBLE-POLE, DOUBLE-THROW (DPDT): A term used to describe a switch or relay output contact form. Two separate switches that operate simultaneously each with a normally open and normally closed contact and a common connector.

DRIFT: Slow change of a metrological characteristic of a measuring instrument.

DYNE: The unit of force which, when acting upon a mass of 1 gm, will produce an acceleration of 1 cm/sec/sec.

EFFECTIVE MASS: The mass of a body which is being acted upon by the buoyant forces of air. The effective mass of a weight is its true mass minus the buoyant force of air displaced by the weight.

EFFECTIVE VALUE (RMS): The alternating current value that will produce the same amount of heat in a resistance as the corresponding direct current value.

EFFICIENCY: The ration of useful output energy, usually expressed as a percentage.

EFFLUENT: Liquid that has passed through a processing operation.

ELASTIC ELEMENT: The material of which the transducer is constructed, generally selected for its good elastic properties.

ELECTRICAL RELAY: Employs a solenoid to provide mechanical action to move a varying number of electrical contacts back and forth or on and off.

ELECTRONIC SWITCH: An electric circuit designed to cause a start and stop action or a switching action.

ELECTROSTATIC FIELD: The region surrounding an electric charge in which another charge experiences a force.

ELEMENT: A quality of product, material, or service forming a cohesive entity on which a measurement or observation may be made.

EMPIRICAL: Based on actual measurement, observation, or experience without regard to science and theory.

ENDOERGIC REACTION: A reaction which absorbs energy.

ERG: A CGS unit of work or energy.

ERROR (OF MEASUREMENT): The result of a measurement minus the true value of the measurnand.

EXOERGIC REACTION: The reaction which liberates energy.

EXPLOSION-PROOF (XPRF) MOTOR: A totally enclosed motor that will withstand an explosion of a specific vapor or gas within its housing, or will prevent sparks or flashes generated within its housing from igniting surrounding vapor or gas.

FACTORY CALIBRATION: The tuning or altering of a control device by the manufacture to bring it into specification.

FAHRENHEIT SCALE: A temperature scale that defines the freezing point of water as 32 degrees and the boiling point of water a 212 degrees.

FIXED POINT: The point where all heat energy applies or removed is used to change the state of a substance.

FLUX: (1) A material used to promote fusion or joining of metals in soldering, welding, or smelting. (2) A general term used to designate collectively all the electric or magnetic lines of force in a region.

FORCE: A push or pull which produces or prevents motion or has a tendency to do so.

FORCE MEASUREMENT DEVICE: Any device which a quantitative determination of an applied force can be made.

FORCED VIBRATION: The motion caused by some mechanical excitation.

FREE VIBRATION: Vibration that occurs without forcing.

FREQUENCY: The number of recurrences of a periodic phenomenon.

FREQUENCY METER: An instrument for measuring the frequency of an AC signal.

FULL SCALE OUTPUT (FSO): The output at rated capacity minus the output at zero applied force.

FUNDAMENTAL METHOD OF MEASUREMENT: The method of measurement in which the value of a measurand is obtained by measurement of the appropriate base quantities.

FUNDAMENTAL MODE OF VIBRATION: The lowest natural frequency.

FUNCTION TESTS: Function test often duplicates unit test activities sincefunction testers do not assume that unit test is adequately done.

GAGE: A measuring instrument for measuring and indicating a quantity.

GAGE BLOCK: A block of alloy steel with two gaging surfaces.

GAIN: Ratio of output voltage, current, or power to input voltage current or power.

GALVANOMETER: Meter for detecting or comparing or measuring small electric currents.

GAMMA RAY: Electromagnetic radiation emitted during radioactive decay and having an extremely short wavelength.

GAS: The state of matter that has no definite shape of volume.

GAUGE FACTOR: The sensitivity of the strain gauge.

GAUGE PRESSURE (PSIG): A measure of the force per area exerted by a fluid using atmospheric pressure as a zero reference.

GAUSS: Unit of magnetic induction.

GO AND NO-GO GAGES: Gages that do not measure actual size but merely determine whether parts are within specified limits.

GRAIN: A measure of mass in the English gravitational system equal to 1/7000th pound.

GRAM: A metric unit of weight equal to one thousandth of a kilogram.

GRAM-ATOMIC WEIGHT: The quantity of an element whose weight in grams is numerically equal to the atomic weight of the element.

GRAM-MOLECULAR WEIGHT (GRAM-MOLE): The relative molecular weight of a compound, expressed in grams.

GRATICULE: A network of fine lines, dots, cross hairs, or wires in the focal plane of the eyepiece of an optical instrument.

GRAVITATIONAL ACCELERATION: The acceleration due to the force of gravity in the product is counted, with respect to a given requirement or set of requirements.

INSTABILITY: An undesired change over a period of time, which change is unrelated to input, operating conditions, or load.

INTERFEROMETER: Any measuring instrument that uses interference patterns to make accurate measurements of waves.

INTERPOLATION: Calculation of the value of a function

INOP: C1. Inoperative. 2. Slang. Broken.

ISO: International Organization for Standardization.

INVERSION: The condition that exists when both axes of an image are reversed.

INVERTER: Any mechanical or electrical device for converting direct current into alternating current.

JITTER: Small, rapid variations in a waveform due to mechanical disturbances.

JOULE: A unit of electrical energy equal to the work done when a current of one ampere passes through a resistance of one ohm for one second.

KELVIN TEMPERATURE SCALE: The absolute temperature scale in the CGS system. Kelvin is equal to degrees Celsius plus 273.15.

KILOGRAM: One thousand grams.

KINETIC ENERGY: Energy due to motion.

LEVEL: Perpendicular to the force of gravity.

LIMS (LABORATORY INFORMATION MANAGEMENT SYSTEM): A system that manages operations of a testing laboratory.

LINEARITY: The degree to which performance or response approaches the condition of being linear.

LINEAR METER: The deflection of the pointer is proportional to the quantity measured.

LOAD CELL: A type of force transducer designed primarily for the measurement of load or weight.

LOADING EFFECTS: An error of measurement resulting in a change of the system under test caused by insertion of the test instrument.

LUMEN: Unit of luminous flux.

MAGNETIC DEFLECTION: Method of bending electrons in a CRT by means of the magnetic field produced by coils placed outside the tube.

MANAGEMENT REVIEW: Formal evaluation by top management of the status and adequacy of the quality system in relation to quality policy and objectives.

MASS: Measure of the quantity of matter that a body contains.

MASS DENSITY: Mass per unit volume.

MASS NUMBER: The number of protons and neutrons in the atomic nucleus of an element.

MASS UNIT: A unit of measurement for mass.

MCLEOD GAGE: A primary instrument for the measurement of pressure in a vacuum system.

MEGOHM: 1,000,000 ohms of resistivity.

MEASURAND: A particular quantity subject to measurement.

MEASUREMENT: The act or process of measuring.

MEASUREMENT STANDARDS: A material measure, measuring instrument, reference material, or system intended to define, conserve, or reproduce a unit or one or more values of a quantity in order to transmit them to other measuring instruments by comparison.

MEASUREMENT UNCERTAINTY: The estimated amount by which the measured quantity may depart from the true value.

MEASURING EQUIPMENT: All of the measuring instruments, measurement standards, reference materials, auxiliary apparatus, and instructions that are necessary to carry out a measurement. This includes measuring equipment used in the course of that used in calibration.

METER: The basic unit of length adopted under the Systeme International d’Unites (approximately 1.094 yards)

METROLOGY: The science of measurement.

MEV: The abbreviation for a million electron volts.

MHO: A unit of conductance.

MICRO: Equivalence of one millionth.

MICRON: A metric unit of length equal to one millionth of a meter.

MILLI: Equivalence of one thousandth.

MINUTE: 1/60th of a degree.

MKS SYSTEM: The meter-kilogram-second system.

MODEL FOR QUALITY ASSURANCE: Standardized or selected set of quality system requirements situation.

MOMENT ARM: The length of a torque wrench from the center of pivot to the point where force is applied.

MOMENTUM: The product of the mass of a body and its velocity.

NATIONAL INSTITUTE OF SCIENCE AND TECHNOLOGY: An independent agency of the U.S. Department of Commerce charged with the improvement and maintenance of standards.

NEON: An inert element which is a gas at room temperature.

NEUTRON: An elementary particle with 0 charge and mass about equal to a proton.

NEUTRINO: An elementary particle with zero charge and zero mass.

NEWTON: A unit of force equal to the force that imparts an acceleration of 1 m/sec/sec to a mass of 1 kilogram.

NEWTONIAN FLUID: A fluid whose absolute viscosity is the same for all values of shear stress.

NOMINAL VALUE: This is normally the value indicated by the manufacture.

NONCONFORMITY: Nonfulfillment of a specified requirement.

NONLINEAR: Pertaining to a response which is not directly or inversely proportional to a given variable.

NORMALLY CLOSED (NC) SWITCH: A switch in which the contracts are closed without any external force acting upon it.

NORMALLY OPEN (NO) SWITCH: A switch in which contracts are open when no external forces act upon the switch.

NULL METHOD: Any method of measurement in which the reading is taken at zero.\

OBJECTIVE EVIDENCE:Information that can be proved true, based on facts obtained through observation, measurement, tests or other means.

OHM: A unit of electrical resistance equal to the resistance between two points on a conductor when a potential difference of one volt between them produces a current of one ampere.

OHMMETER: A instrument for measuring resistance.

OPTICAL PYROMETER: An instrument designed to estimate the temperature of glowing surfaces.

OPTICAL TOOLING: The geometric method of optically establishing a precise line and/or reference plane.

ORGANIZATION: Company, corporation, firm, enterprise, or institution or part, thereof, whether incorporated or not, public or private, that has its own functions and administration.

ORGANIZATIONAL STRUCTURE: Responsibilities, authorities, and relationships, arranges in a pattern, through which an organization performs its functions.

OUT OF PHASE: Having waveforms that are of the same frequency but not passing through corresponding values at the same instants.

OUT-OF-ROUND: The high and low spots in a true circle.

OVERSHOOT: The initial transient response to an unidirectional change in input which exceeds the steady stat response.

PACKING FRACTION: The difference between the atomic weight in mass units and the mass number of an element divided by the mass number and multiplied by 10,000.

PARALLAX: The apparent displacement of an object as seen from two different points that is not on a line with the object.

PARALLEL TRANSMISSION: Transmission of data bits over different lines, as opposed to serial transmission.

PEAK-TO-PEAK AMPLITUDE: The amplitude of an alternating quantity measured from positive to negative peak.

PH: A indication of the acidity or alkalinity of a solution.

PID CONTROL: Control in which the control signal is a linear combination of the error signal, its integral, and its derivative.

POINTER: The needle-shaped rod that moves over the scale of a meter or dial.

POTENTIAL: The amount of voltage or change between a point and a zero reference point.

POTENTIAL DIFFERENCE: The difference in potential between any two points in a circuit.

POTENTIAL ENERGY: Energy due to position.

PONTENTIOMETER: A measuring instrument for measuring direct current electromotive forces.

POTENTIOMETRIC MEASUREMENT: Comparing the unknown voltage with a known voltage from a calibrated potentiometer.

PRECISION: The closeness of agreement between randomly selected individual measurements or test results.

PRESSURE: Force exerted per unit area.

PREVENTIVE ACTION: Action taken to eliminate the causes of a potential nonconformity defect or other undesirable situation in order to prevent recurrence.

PRIMARY STANDARD: A unit established by some authority or developed through practical application of a formula.

PROBABILITY: A measure of how likely it is that some event will occur.

PROPORTIONAL CONTROL: Control in which the amount of corrective action is proportional to the amount of error.

PSYCHROMETER: A instrument for measuring relative humidity.

PYROMETER: A device for measuring high temperatures.

QUALIFICATION PROCESS: Process of demonstrating whether an entity is capable of fulfilling specified requirements.

QUALIFIED: Status given to an entity when capability of fulfilling specified requirements has been demonstrated.

QUALITY: The totality of features and characteristics of a product or service that bears on its ability to satisfy given needs.

QUALITY ASSURANCE: All those planned or systematic action necessary to provide adequate confidence that adequate or service will satisfy given needs.

QUALITY AUDIT: A systematic and independent examination to determine whether quality activities and related results comply with planned arrangements and whether these arrangements are implemented effectively and are suitable to achieve objectives.

QUALITY AUDIT OBSERVATION: Statement of fact during a quality audit and substantiated by objective audience.

QUALITY CONTROL: The operational techniques and the activities that sustain a quality of product or service that will satisfy given needs; also, the use of such techniques and activities.

QUALITY EVALUATION: Systematic examination of the extent to which an entity is capable of fulfilling specified requirements.

QUALITY LOSSES: Losses caused by not realizing the potential of resources in processes and activities.

QUALITY MANAGEMENT: The totality of functions involved in the determination and achievement of quality.

QUALITY MANUAL: Document stating the quality policy and describing the quality system of an organization.

QUALITY PLAN: Document setting out the specific quality practices, resources, and sequence of activities relevant to a particular product, project, or contract.

QUALITY POLICY: Overall intentions and direction of an organization with regard to quality, as formally expressed by top management.

QUALITY-RELATED COSTS: Those costs incurred in ensuring satisfactory quality, as well as the losses incurred when satisfactory quality is not achieved.

QUALITY SURVEILLANCE: Continued monitoring and verification of the status of an entity and analysis of records to ensure that specification requirements are being fulfilled.

QUALITY SYSTEM: Organizational structure procedures, processes and resources needed to implement quality management.

REQUIREMENT FOR QUALITY: Expression of the needs or their translation into a set of quantitatively or qualitatively stated requirements for the characteristics of an entity to enable its realization and examination.

RADIATION: A method of transmission of energy.

RANGE: (1) Extent of coverage of effectiveness. (2) Measure of distance.

RATIO BRIDGE: A bridge circuit that uses a calibrated resistive or calibrated inductive voltage divider for one side of the bride.

REFERENCE LINE: A line from which all other measurements are taken.

REFERENCE PLANE: A reference lie that has been rotated through 360 degrees.

REPEATABILITY: Same reading each time for the same solution.

RESONANCE: An excited state of a stable particle causing a sharp maximum in the probability of absorption of electromagnetic radiation.

RESTORING FORCE: The constant mechanical force provided.

RHO: The magnitude of the reflection coefficient.

SCALE: (1) Something graduated when used as a measure or rule. A series of spaces marked by lines to indicate the magnitude of some quantity. (2) A weighing device.

SCINTILLATION COUNTER: A device used for the detection of radioactivity.

SECONDARY EMISSION: Electron emission that is the direct result of the impact of electrons against a surface.

SEEBACK EFFECT: The EMF produced in a circuit containing two contacting conductors of different metals having two junctions at different temperatures.

SENSITIVITY: Full scale output divided by the rated capacity of a given transducer / load cell.

SENSOR: Element of measuring instrument or measuring chain that is directly or indirectly affected by the measurand.

SERVO SYSTEM: A electromechanical system which is used for positioning one element of a system in relation to another.

SHEAR: A deformation of an object in which parallel planes remain parallel but are shifted in a direction parallel to themselves.

SOLENOID VALVE: A valve actuated by a solenoid for controlling the flow of gases or liquid in pipes.

SPAN: Modules of the difference between the two limits of a normal range.

SOLID: The state in which a substance has no tendency to flow under moderate stress.

SPECIFICATIONS: The range of values or numerical value which ties the performance of the product parameter.

SPECTRUM: (1) The entire range of wavelengths within which electromagnetic radiations occur. (2) A segment of wavelengths which has a special function or possesses special properties.

STABILITY: The ability of a measuring instrument to maintain constant metrological characteristics with time.

STANDARD: (1) Conforming to or constituting a standard of measurement or value. (2) a basis for comparison. (3) the ideal in terms of which something can be judged

STANDARD DEVIATION: A mathematical quantity used to characterize the dispersion of results.

STANDARD OPERATING CONDITIONS, STANDARD TEMPERATURE AND PRESSURE (STP): Defined temperature and pressure to which all values are referenced for comparison.

STANDARD PRESSURE: The pressure exerted by a column of mercury exactly 760 mm high.

STANDARD UNCERTAINTY: Uncertainty of the result of a measurement expressed as a standard deviation.

STRAIN: Deformation of a material body under the action of applied forces.

STRAIGHTNESS: The uniformity of direction throughout the extent of that feature.

STRESS: Force that produces strain on a physical body.

STROBOSCOPE: Scientific instrument that provides a flashing light synchronized with the periodic movement of an object.

SUBCONTRACTOR: Organization that provides a product to the supplier.

SUPPLIER: Organization that provides a product to a customer.

SURFACE TENSION: The tendency of the surface of a liquid to contract.

TACHOMETER: An instrument for measuring rotational speed in revolutions per minute.

TEMPERATURE COEFFICIENT: The change in measured value per unit change in temperature.

TEMPERATURE COMPENSATION: The method of reducing the effect of a change in temperature on a force measuring instrument.

TERMINAL LINEARITY: Ratio of the actual error voltage in the output to the total input voltage.

TERMINATION: The load connected to the output end of a circuit or transmission line.

TESTING: A means of determining the capability of an item to meet specified requirements by subjecting the item to a set of physical, chemical, environmental or operation actions and conditions

TEST INSTRUMENT: The device being compared with the calibration standard.

TEST LINE LIMIT: The pass or fail limit.

THEODOLITE: A optical instrument used for measuring horizontal or vertical angles.

THERMISTOR: A semiconductor device made of materials whose resistance varies as a function of temperature.

TILT GRATICULE: A graduate reticule used in Collimators for measuring vertical and horizontal tilt, or angular deviation.

TIME: Measurement of duration.

TORQUE: Cause of rotary motion. It is equal to the applied force multiplied by the distance from the center of rotation.

TORR: 1/760 of and atmosphere.

TOTAL QUALITY MANAGEMENT: Management approach of an organization, centered on quality based on the participation of its member and aiming at long-term success through customer satisfaction and benefits of all members of the organization and to society.

TRACEABILITY: Ability to trace the history, application, or location of an entity by means of recorded identification

TRANSDUCER: A device that provides an output quantity having a determined relationship to the force.

TRANSFER: Standard used as in intermediary to compare standards.

TRUE MASS: Mass as measured in a vacuum.

UNCERTAINTY: A parameter, associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measured.

UNIT: A value, quantity, or magnitude of which other values, quantities, or magnitudes are expressed.

VACUUM: Any pressure below atmospheric.

VELOCITY: The time rate of change of position.

VELOCITY CONSTANT: The ratio of the velocity of propagation in a transmission line to the velocity of light.

VERIFICATION: Confirmation by examination and provision of objective evidence that specified requirements have been filled.

VIBRATION: Mechanical oscillations or motion about a reference point or equilibrium.

VISCOSITY: Resistance of a liquid to sheer forces (and hence to flow).

VSLI: Very Large Scale Integration.

VOLATILE: Readily vaporizable at a relatively low temperature.

VOLUME: The amount of space which matter occupies.

WAVE FRONT: A surface composed at any instant of all the points just reached by a vibration disturbance in its propagation through a medium.

WEIGHT: The force of gravity acting on an object.

Common Conversion Factors

Property English To Metric Metric To English
Multiply By To Obtain Multiply By To Obtain
LENGTH inches 25.4 mm mm .03937 inches
THICKNESS inches 25400 um um 3.937x10-5 inches
AREA inches² 645.16 mm² mm² .00155 inches²
FORCE pounds
4.448 Newtons
.2248 pounds
TORQUE inch-
.113 Newton-
8.851 inch-
STRESS PSI .006895 MPa MPa 145.04 PSI
STRESS KSI 6.895 MPa MPa .14504 KSI

Calibration Metric / Inch Conversion Chart

Millimeters Fractions Inches
.397 1/64 .015625
.794 1/32 .03125
1.191 3/64 .046875
1.588 1/16 .0625
1.984 5/64 .078125
2.381 3/32 .09375
2.778 7/64 .109375
3.175 1/8 .125
3.572 9/64 .140625
3.969 5/32 .15625
4.366 11/64 .171875
4.762 3/16 .1875
5.159 13/64 .203125
5.556 7/32 .21875
5.953 15/64 .234375
6.350 1/4 .25
6.747 17/64 .265625
7.144 9/32 .28125
7.541 19/64 .296875
7.938 5/16 .3125
8.334 21/64 .328125
8.731 11/32 .34375
9.128 23/64 .359375
9.525 3/8 .375
9.922 25/64 .390625
10.319 13/32 .40625
10.716 27/64 .421875
11.112 7/16 .4375
11.509 29/64 .453125
11.906 15/32 .46875
12.303 31/64 .484375
12.700 1/2 .5
13.097 33/64 .515625
13.494 17/32 .53125
13.891 35/64 .546875
14.288 9/16 .5625
14.684 37/64 .573125
15.081 19/32 .59375
15.478 39/64 .609375
15.875 5/8 .625
16.272 41/64 .640625
16.669 21/32 .65625
17.066 43/64 .671875
17.462 11/16 .6875
17.859 45/64 .703125
18.256 23/32 .71875
18.653 47/64 .734375
19.050 3/4 .75
19.447 49/64 .765625
19.844 25/32 .78125
20.241 51/64 .796875
20.638 13/16 .8125
21.034 53/64 .828125
21.431 27/32 .84375
21.828 55/64 .859375
22.225 7/8 .875
22.622 57/64 .890625
23.019 29/32 .90625
23.416 59/64 .921875
23.812 15/16 .9375
24.209 61/64 .953125
24.606 31/32 .96875
25.003 63/64 .984375
25.400 1 1.000

Sheet Metal

Gage No. Steel Stainless Steel Aluminum
7 .179 - -
8 .164 .172 -
9 .150 .156 -
10 .135 .141 -
11 .120 .125 -
12 .105 .109 -
13 .090 .094 .072
14 .075 .078 .064
15 .067 .070 .057
16 .060 .063 .051
17 .054 .056 .045
18 .048 .050 .040
19 .042 .044 .036
20 .036 .038 .032
21 .033 .034 .028
22 .030 .031 .025
23 .027 .028 .023
24 .024 .025 .020
25 .021 .022 .018
26 .018 .019 .017
27 .016 .017 .014
28 .015 .016 -
29 .014 .014 -
30 .012 .013 -
31 - .011 -

Calibration Basics

The following is a presentation from National Instrument’s Test Equipment Summit that serves as a good primer on calibration.  It explains all the basic concepts and terms in respect to incorporating calibration in best practices and ensuring product quality.

What is Calibration?

Definition: Calibration is the comparing of a measurement device (an unknown) against an equal or better standard. A standard in a measurement is considered the reference; it is the one in the comparison took to be the more correct of the two. One calibrates to find out how far the unknown is from the standard.

Typical Calibration: A “typical” commercial calibration references a manufactures calibration procedure and is performed with a reference standard at least four times more accurate than the instrument under test.

Why Calibrate?

Calibration is an Insurance Policy.

Some people consider calibration a necessary annoyance to keep the auditor off their back.  In fact, out of tolerance (OOT) instruments may give false information leading to unreliable product, customer dissatisfaction and increased warranty costs.  In addition, OOT conditions may cause good products to fail tests, which ultimately results in unnecessary rework costs and production delays.

Common calibration terms

Out of Tolerance Conditions: If the results are outside the instrument’s performance specifications it is considered an OOT (Out of Tolerance) condition and will result in the need to adjust the instrument back into specification.

Optimization: Adjusting a measuring instrument to make it more accurate is NOT part of a “Typical” calibration and is frequently called “Optimizing” or “Nominalizing” an instrument. (this is a common misconception) Only reputable and experienced calibration providers should be trusted to make adjustments on critical test equipment.

As Found Data: The reading of the instrument before it is adjusted delays.

As Left Data: The reading of the instrument after adjustment or “Same As Found” if no adjustment was made.

Without Data: Most calibration labs charge more to provide the certificate with data and will offer a “No-Data” option. In any case “As-Found” data must be provided for any OOT condition.

Limited Calibration: Sometimes certain functions of an instrument may not be needed by the user. It may be more cost effective to have a limited calibration performed (This can even include a reduced accuracy calibration).

TUR – Test Uncertainty Ratio: The ratio of the accuracy of the instrument under test compared to the accuracy of the reference standard.

ISO/IEC 17025 Calibration: As a general rule 17025 calibrations are required by anyone supplying the automotive industry and it has also been voluntarily adapted by numerous companies in FDA regulated industries.

ISO/IEC 17025 is an international standard that assesses the technical competency of calibration laboratories. ISO/IEC 17025 covers every aspect of laboratory management, ranging from testing proficiency to record keeping and reports. It goes several steps beyond a ISO 9001:2000 certification.

A “17025” calibration is a premium option that provides additional information about the quality of each measurement made during the calibration process by individually stating the uncertainty calculation of each test point.

How Calibration Intervals are Determined

Calibration intervals are to be determined by the instrument “owner” based on manufacture recommendations. Commercial calibration laboratories can suggest intervals but usually they are not familiar with the details of the instrument’s application.

The OEM intervals are typically based on guidelines like mean drift rates for the various components within the instrument. However, when determining calibration intervals as an instrument “owner” several other factors should be considered such as: the required accuracy vs. the instrument’s accuracy, the impact an OOT will have on the process, and the performance history of the particular instrument in your application.

How to Implement or Improve a Calibration Program

Any successful calibration program must begin with an accurate recall list of your test, measurement and diagnostic equipment.

  • The recall list should contain a unique identifier which tracks the instrument, the location, and the instrument’s custodian (Often asset management software, bar-coding systems, and physical inventories are used to help establish accurate recall lists).
  • It is important when assembling a recall list that module, plug-ins, and small handheld tools are not overlooked. Also, you may have several “home-made” measuring devices (e.g. Test Fixtures) which will also need to be captured on your equipment list for a reliable calibration program.
  • The next step is to identify all the instruments on your recall list which may not require calibration due to redundancies in your testing process (A commercial calibration laboratory should be able to aid you in identifying these instruments).
  • After creating an accurate recall list procedures must be established for adding new instruments, removing old or disposed instruments, or making changes in instrument custodianship. Recall reports should be run with sufficient time for both the end user and the service provider to have the unit calibrated with a minimal impact on production.
  • A late report identifying any units about to expire or already expired will ensure 100% conformity. A full service calibration laboratory will supply these recall reports and will provide special escalation reporting when equipment is not returned for service.

(Some calibration labs offer the choice of web-based equipment management systems that allow their customer to perform recall reports, late reports and keep electronic versions of their calibration certificates.)

Avoiding Production Delays

Obtain timely equipment calibrations without shutting down a line for days.

  • Look for a calibration service provider that can performs onsite (or in-place) calibrations at your facility. Often when your volume is more than 20 calibrations, scheduling onsite calibration saves time and lowers cost.
  • Make sure you find a “one-source” calibration provider that has sufficient capabilities to calibrate nearly all your equipment during the onsite, reducing the delays and the expense of using an additional subcontractor.
  • Other options for reducing downtime include mobile Calibration lab services, scheduled depot calibrations, calibrations during shutdowns, scheduled pick-up and delivery, and weekend or nightshift calibrations.

Should We Calibrate Ourselves?

Most companies discover they cannot effectively perform their own calibrations for many reasons.  The most frequent issues with internal calibrations are:

Cost of standards:  Often, the cost of the assets with the required accuracy to perform the calibration is prohibitive (It could take years of calibrations to pay for one standard).

Developing Procedures:  Many manufacture’s procedures are not readily available. Sometimes they require research and development. This can cost hundreds of hours of labor.

Productivity of Technicians:  Often a non-commercial calibration laboratory’s productivity per employee is only a fraction of what can be obtained through an external commercial calibration laboratory who specializes in automation, efficient procedures and experienced management.

Cost of Management:  Managing the employees, assets, maintenance and processes of a calibration lab can be burdensome on existing management staff.

Not a core competency: The overall management burden of the operation distracts from the core competency of the company.

Calibration Terminology

The field of calibration has a huge vocabulary describing the methods and processes used to verify the measurement accuracy of masters, gages and other measuring instruments. The following definitions are for the most commonly used terms.


A2LA are the initials of American Association for Laboratory Accreditation, a non-profit accrediting agency specializing in the accreditation of calibration and testing laboratories.

Accreditation is a process used by a qualified independent agency to verify the quality system and technical capability of a calibration laboratory to a recognized standard such as ISO 17025.

Accuracy defines how close a measured value is to the true value of the dimension.

Calibration is the set of operations which establish, under specified conditions, the relationship between values of quantities indicated by a measuring instrument or measuring system, or values represented by a material measure or a reference material and the corresponding values realized by standards.

Calibration Certificate or Report is the document that presents calibration results and other information relevant to a calibration.

Calibration Frequency is the time intervals at which instruments, gages and masters are calibrated. These intervals are determined by their user based on the conditions of their use to ensure their performance or size remain within acceptable limits.

Calibration Limits is a tolerance applied to gages and instruments beyond which they are not considered suitable for use.

International (Measurement) Standard is a standard recognized by an international agreement to serve internationally as the basis for fixing the value of all other standard of the quantity concerned.

Limits of Permissible Error (of a measuring instrument) are the extreme values of an error permitted by specifications, regulations, etc. for a given measuring instrument.

Measurement Assurance is the technique that may include, but is not limited to: 1) use of good experimental design principles so the entire measurement process, its components, and relevant influence factors can be well characterized, monitored and controlled; 2) complete experimental characterization of the measurement process uncertainty including statistical variations, contributions from all known or suspected influence factors, imported uncertainties, and the propagation of uncertainties throughout the measurement process; and 3) continuously monitoring the performance and state of statistical control of the measurement process with proven statistical process control techniques including the measurement of well characterized check standards along with the normal workload and the use of appropriate control charts.

Measuring and Test Equipment includes all of the measuring instruments, measurement standards, reference materials, and auxiliary apparatus that are necessary to perform a measurement. This term includes measuring equipment used in the course of testing and inspection, as well as that used in calibration.

Quality System is the organizational structure, responsibilities, procedures, processes and resources for implementing quality management.

Resolution represents the smallest reading unit provided by an instrument.

Traceability is the path by which a measurement can be traced back to the source from which it is derived, such as NIST in the United States. Direct traceability implies that the laboratory has its primary masters calibrated directly by such an agency for reduced measurement uncertainty.

Uncertainty of Measurement is a parameter associated with the result of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurement.


Digits to the right of the decimal point represents the fractional part of the decimal number. Each place value has a value that is one tenth the value to the immediate left of it.

Number Name Fraction
.1 tenth 1/10
.01 hundredth 1/100
.001 thousandth 1/1000
.0001 ten thousandth 1/10000
.00001 hundred thousandth 1/100000


0.234 = 234/1000 (said – point 2 3 4, or 234 thousandths, or two hundred thirty four thousandths)

4.83 = 4 83/100 (said – 4 point 8 3, or 4 and 83 hundredths)

Number Prefix Symbol
10 1 deka- da
10 2 hecto- h
10 3 kilo- k
10 6 mega- M
10 9 giga- G
10 12 tera- T
10 15 peta- P
10 18 exa- E
10 21 zeta- Z
10 24 yotta- Y
10 -1 deci- d
10 -2 centi- c
10 -3 milli- m
10 -6 micro- u (greek mu)
10 -9 nano- n
10 -12 pico- p
10 -15 femto- f
10 -18 atto- a
10 -21 zepto- z
10 -24 yocto- y
I=1 (I with a bar is not used)
V=5 _
X=10 _
L=50 _
C=100 _
D=500 _
M=1,000 _

There is no zero in the roman numeral system.

The numbers are built starting from the largest number on the left, and adding smaller numbers to the right. All the numerals are then added together.

The exception is the subtracted numerals, if a numeral is before a larger numeral, you subtract the first numeral from the second. That is, IX is 10 – 1= 9.

This only works for one small numeral before one larger numeral – for example, IIX is not 8, it is not a recognized roman numeral.

There is no place value in this system – the number III is 3, not 111.


1 = I
2 = II
3 = III
4 = IV
5 = V
6 = VI
7 = VII
8 = VIII
9 = IX
10 = X

11 = XI
12 = XII
13 = XIII
14 = XIV
15 = XV
16 = XVI
17 = XVII
18 = XVIII
19 = XIX
20 = XX
21 = XXI

25 = XXV
30 = XXX
40 = XL
49 = XLIX
50 = L
51 = LI
60 = LX
70 = LXX
80 = LXXX
90 = XC
99 = XCIX

Decimal(10) Binary(2) Ternary(3) Octal(8) Hexadecimal(16)
0 0 0 0 0
1 1 1 1 1
2 10 2 2 2
3 11 10 3 3
4 100 11 4 4
5 101 12 5 5
6 110 20 6 6
7 111 21 7 7
8 1000 22 10 8
9 1001 100 11 9
10 1010 101 12 A
11 1011 102 13 B
12 1100 110 14 C
13 1101 111 15 D
14 1110 112 16 E
15 1111 120 17 F
16 10000 121 20 10
17 10001 122 21 11
18 10010 200 22 12
19 10011 201 23 13
20 10100 202 24 14
+ 0 1 2 3 4 5 6 7 8 9 10
0 0 1 2 3 4 5 6 7 8 9 10
1 1 2 3 4 5 6 7 8 9 10 11
2 2 3 4 5 6 7 8 9 10 11 12
3 3 4 5 6 7 8 9 10 11 12 13
4 4 5 6 7 8 9 10 11 12 13 14
5 5 6 7 8 9 10 11 12 13 14 15
6 6 7 8 9 10 11 12 13 14 15 16
7 7 8 9 10 11 12 13 14 15 16 17
8 8 9 10 11 12 13 14 15 16 17 18
9 9 10 11 12 13 14 15 16 17 18 19
10 10 11 12 13 14 15 16 17 18 19 20
12 0 12 24 36 48 60 72 84 96 108 120 132 144
11 0 11 22 33 44 55 66 77 88 99 110 121 132
10 0 10 20 30 40 50 60 70 80 90 100 110 120
9 0 9 18 27 36 45 54 63 72 81 90 99 108
8 0 8 16 24 32 40 48 56 64 72 80 88 96
7 0 7 14 21 28 35 42 49 56 63 70 77 84
6 0 6 12 18 24 30 36 42 48 54 60 66 72
5 0 5 10 15 20 25 30 35 40 45 50 55 60
4 0 4 8 12 16 20 24 28 32 36 40 44 48
3 0 3 6 9 12 15 18 21 24 27 30 33 36
2 0 2 4 6 8 10 12 14 16 18 20 22 24
1 0 1 2 3 4 5 6 7 8 9 10 11 12
0 0 0 0 0 0 0 0 0 0 0 0 0 0
x 0 1 2 3 4 5 6 7 8 9 10 11 12

Important Note: any span of numbers that is underlined signifies that those numbers are repeated. For example, 0.09 signifies 0.090909….

Only fractions in lowest terms are listed. For instance, to find 2/8, first simplify it to 1/4 then search for it in the table below.

fraction = decimal      
1/1 = 1      
1/2 = 0.5      
1/3 = 0.3 2/3 = 0.6    
1/4 = 0.25 3/4 = 0.75    
1/5 = 0.2 2/5 = 0.4 3/5 = 0.6 4/5 = 0.8
1/6 = 0.16 5/6 = 0.83    
1/7 =  0.142857 2/7 =  0.285714 3/7 =  0.428571 4/7 =  0.571428
  5/7 =  0.714285 6/7 =  0.857142  
1/8 = 0.125 3/8 = 0.375 5/8 = 0.625 7/8 = 0.875
1/9 = 0.1 2/9 = 0.2 4/9 = 0.4 5/9 = 0.5
  7/9 = 0.7 8/9 = 0.8  
1/10 = 0.1 3/10 = 0.3 7/10 = 0.7 9/10 = 0.9
1/11 = 0.09 2/11 = 0.18 3/11 = 0.27 4/11 = 0.36
  5/11 = 0.45 6/11 = 0.54 7/11 = 0.63
  8/11 = 0.72 9/11 = 0.81 10/11 = 0.90
1/12 = 0.083 5/12 = 0.416 7/12 = 0.583 11/12 = 0.916
1/16 = 0.0625 3/16 = 0.1875  5/16 = 0.3125 7/16 = 0.4375
  11/16 = 0.6875 13/16 = 0.8125 15/16 = 0.9375
1/32 = 0.03125 3/32 = 0.09375 5/32 = 0.15625 7/32 = 0.21875
  9/32 = 0.28125 11/32 = 0.34375 13/32 = 0.40625
  15/32 = 0.46875 17/32 = 0.53125 19/32 = 0.59375
  21/32 = 0.65625 23/32 = 0.71875 25/32 = 0.78125
  27/32 = 0.84375 29/32 = 0.90625 31/32 = 0.96875

Need to convert a repeating decimal to a fraction? Follow these examples:
Note the following pattern for repeating decimals:

0.22222222… = 2/9
0.54545454… = 54/99
0.298298298… = 298/999
Division by 9’s cause the repeating pattern.

Note the pattern if zeros proceed the repeating decimal:

0.022222222… = 2/90
0.00054545454… = 54/99000
0.00298298298… = 298/99900
Adding zero’s to the denominator adds zero’s before the repeating decimal.

To convert a decimal that begins with a non-repeating part, such as 0.21456456456456456…, to a fraction, write it as the sum of the non-repeating part and the repeating part.

0.21 + 0.00456456456456456…
Next, convert each of these decimals to fractions. The first decimal has a divisor of power ten. The second decimal (which repeats) is converted according to the pattern gave above.

21/100 + 456/99900
Now add these fraction by expressing both with a common divisor

20979/99900 + 456/99900
and add.

Finally simplify it to lowest terms

and check your calculator or with long division.
= 0.2145645645…

A note on the metric system:
Before you use this table, convert to the base measurement first. For example, convert centimeters to meters, convert kilograms to grams.

The notation 1.23E – 4 stands for 1.23 x 10-4 = 0.000123.

from \ to = __ feet = __ inches = __ meters = __ miles = __ yards
foot   12 0.3048 (1/5280) (1/3)
inch (1/12)   0.0254 (1/63360) (1/36)
meter 3.280839... 39.37007...   6.213711...E - 4 1.093613...
mile 5280 63360 1609.344   1760
yard 3 36 0.9144 (1/1760)  

To use: Find the unit to convert from in the left column, and multiply it by the expression under the unit to convert to.
Examples: foot = 12 inches; 2 feet = 2×12 inches.
Useful Exact Length Relationships
mile = 1760 yards = 5280 feet
yard = 3 feet = 36 inches
foot = 12 inches
inch = 2.54 centimeters

A note on the metric system:

Before you use this table convert to the base measurement first. For example, convert centimeters to meters, convert kilograms to grams.

from \ to = __ acres = __ feet2 = __ inches2 = __ meters2 = __ miles2 = __ yards2
acre   43560 6272640 4046.856... (1/640) 4840
foot2 (1/43560)   144 0.09290304 (1/27878400) (1/9)
inch2 (1/6272640) (1/144)   6.4516E - 4 3.587006E - 10 (1/1296)
meter2 2.471054...E - 4 10.76391... 1550.0031   3.861021...E - 7 1.195990...
mile2 640 27878400 2.78784E + 9 2.589988...E + 6   3097600
yard2 (1/4840) 9 1296 0.83612736 3.228305...E - 7  

To use: Find the unit to convert from in the left column, and multiply it by the expression under the unit to convert to.
Examples: foot2 = 144 inches2; 2 feet2 = 2×144 inches2.

Useful Exact Area & Length Relationships
acre = (1/640) miles2
mile = 1760 yards = 5280 feet
yard = 3 feet = 36 inches
foot = 12 inches
inch = 2.54 centimeters

Note that when converting area units:
1 foot = 12 inches
(1 foot)2 = (12 inches)2 (square both sides)
1 foot2 = 144 inches2
The linear & area relationships are not the same!

A note on the metric system:
Before you use this table, convert to the base measurement first. For example, convert centimeters to meters, kilograms to grams, etc.

The notation 1.23E – 4 stands for 1.23 x 10-4 = 0.000123.

from \ to = __ feet3 = __ gallons = __ inches3 = __ liters = __ meters3 = __ miles3 = __ pints = __ quarts = __ yards3
7.480519... 1728 28.31684... 0.02831684... 6.793572E - 12 59.84415... 29.92207... (1/27)
gallon 0.1336805...
231 3.785411... 0.003785411... 9.081685...E - 13 8 4 0.004951131...
inch3 (1/1728) (1/231)
0.01638706... 1.638706...E - 5 3.931465...E - 15 (1/28.875) (1/57.75) (1/46656)
liter 0.03531466... 0.2641720... 61.02374...
(1/1000) 2.399127...E - 13 2.113376... 1.056688... 0.001307950...
meter3 35.31466... 264.1720... 61023.74... 1000
2.399127...E - 10 2113.376... 1056.688... 1.307950...
mile3 1.471979...E + 11 1.101117...E + 12 2.543580E + 14 4.168181...E + 12 4.168181...E + 9
8.808937...E + 12 4.404468...E + 12 5.451776...E + 9
pint 0.01671006... (1/8) 28.875 0.4731764... 4.731764...E - 4 1.135210...E - 13
(1/2) 6.188914...E - 4
quart 0.03342013... (1/4) 57.75 1.056688... 9.463529...E - 4 2.270421...E - 13 2
yard3 27 0.004951131... 46656 0.001307950... 0.7645548... 1.834264...E - 10 1615.792... 807.8961...

To use: find the unit to convert from in the left column, and multiply it by the expression under the unit to convert to.
Examples: foot3 = 1728 inches3; 2 feet3 = 2×1728 inches2.

Useful Exact Volume Relationships

fluid ounce = (1/8) cup = (1/16) pint = (1/32) quart = (1/128) gallon
gallon = 128 fluid ounces = 231 inches3 = 8 pints = 4 quarts
quart = 32 fluid ounces = 4 cups = 2 pints = (1/4) gallon

Useful Exact Length Relationships

cup = 8 fluid ounces = (1/2) pint = (1/4) quart = (1/16) gallon
mile = 63360 inches = 5280 feet = 1760 yards
yard = 36 inches = 3 feet = (1/1760) mile
foot = 12 inches = (1/3) yard = (1/5280) mile
pint = 16 fluid ounces = (1/2) quart = (1/8) gallon
inch = 2.54 centimeters = (1/12) foot = (1/36) yard
liter = 1000 centimeters3 = 1 decimeter3 = (1/1000) meter3

Note that when converting volume units:
1 foot = 12 inches
(1 foot)3 = (12 inches)3 (cube both sides)
1 foot3 = 1728 inches3
The linear & volume relationships are not the same! 

Number Prefix Symbol
10 1 deka- da
10 2 hecto- h
10 3 kilo- k
10 6 mega- M
10 9 giga- G
10 12 tera- T
10 15 peta- P
10 18 exa- E
10 21 zeta- Z
10 24 yotta- Y
10 -1 deci- d
10 -2 centi- c
10 -3 milli- m
10 -6 micro-
10 -9 nano- n
10 -12 pico- p
10 -15 femto- f
10 -18 atto- a
10 -21 zepto- z
10 -24 yocto- y

Hierarchy of Decimal Numbers

To divide decimal numbers:

  • If the divisor is not a whole number:
  • Move the decimal point in the divisor all the way to the right (to make it a whole number).
  • Move the decimal point in the dividend the same number of places.
  • Divide as usual. If the divisor doesn’t go into the dividend evenly, add zeroes to the right of the last digit in the dividend and keep dividing until it comes out evenly or a repeating pattern shows up.
  • Position the decimal point in the result directly above the decimal point in the dividend. [Show Me. Show and highlight the decimal point in the quotient, among the 4 and 9]
  • Check your answer: Use the calculator and multiply the quotient by the divisor. Does it equal the dividend?
  • Let’s work through an example.

Request A Quote

All of your test equipment calibration needs in one place.