Confirmation bias when measuring
Close
Login to Your Account
Likes Likes:  0
Results 1 to 14 of 14
  1. #1
    Join Date
    Jun 2007
    Location
    New Zealand
    Posts
    2,673
    Post Thanks / Like
    Likes (Given)
    0
    Likes (Received)
    28

    Default Confirmation bias when measuring

    One issue I've not heard or seen discussed which, to my mind, bears sober reflection, particularly with dial and digital slide calipers and height gauges (but not vernier) is the impact of what I think of as "confirmation bias".

    This applies in cases when we have a target reading in mind, as when calibrating an instrument, or 'shooting' for a particular size (say when turning to a diameter using a caliper, or checking a series of components are within tolerance).

    Because the effective rigidity of the jaws of any slide caliper is relatively low, and the reading is evident while we are taking the measurement, there is a strong element of discretion in the result we choose. Another way of saying we can make the caliper read pretty much whatever we want, within reason.

    The collateral damage from this is that if we know what the instrument 'should' read, we are likely to unconsciously adjust the variables (force, angle) so as to hit the target ... or to be slightly closer to it.

    This can be checked by calibrating 'blind' - holding the caliper so we can't see the display or dial, and calling when we believe we have the correct angle and force, so that someone in a position to see the dial or display can note the reading.

    IF you try this with some length standards, I think you might be surprised by the increased variation when calibrating 'blind'.

    Unless .... a force measurement device is used (and a 'datum' is established using a length standard at the same distance from the main beam as the item we're measuring, rather than simply zeroing on closed jaws)

    OR the caliper is of very stable design with respect to guideways and is closed by pushing on the jaws themselves, in line with the length standard.

    I personally do the latter whenever taking accurate measurements with such a caliper (which I mainly do when I can't get a mic into position)

    My favourite, though, (if I have to use a slide caliper at all) is a heavily built vernier, with a separate sliding clamp and a fine adjustment screw (more like the usual setup for, say, a height gauge):

    ... firstly because it's simply more rigid (and the jaws being wider, there is less problem holding them dead square),

    ... secondly I can adjust until I get the sliding feel on the workpiece I want, more like with a mic

    .... thirdly because with a vernier, you have to decide purely on feel when you consider you're at 'size' ... without your perceptions being potentially skewed by metaphorically "flicking to the answers at the back of the book" and reading the measurement. This is a sort of accidental benefit.

    The other thing I find pleasing about a good vernier caliper is that the intelligence is inherent to the mechanism and the geometry: there's no layer of extra fiddle-faddle to interpret the results for me, use up batteries, fade, or simply go wrong.

    Occam's razor.

  2. #2
    Join Date
    Jan 2005
    Location
    Toledo, OH
    Posts
    1,004
    Post Thanks / Like
    Likes (Given)
    2
    Likes (Received)
    34

    Default

    The human mind can operate in devious ways. Whenever I hear how someone has checked his Harbor Freight caliper against his Harbor Freight gage blocks and proved it to be "dead nuts accurate" I get a chuckle.

    It would be interesting to have one of these experts measure a set of ten unmarked samples that were close in size to each other. Somehow mix them up as they were dealt out such that each one was eventually measured ten times, but the subject was under the impression that he was measuring 100 unique samples.

  3. #3
    Join Date
    Jul 2006
    Location
    Clover Hill district, WI
    Posts
    2,297
    Post Thanks / Like
    Likes (Given)
    0
    Likes (Received)
    8

    Default

    Great that you brought this up.
    Validity of this topic is unquestioned.

    I've seen up to three 40 year guys, playing rock-paper-scizors
    with the largest vernier in the shop.

    Die rings that size pay the month's wages for all three.
    Conference measurement they used to call it.

    m1m

  4. #4
    Join Date
    Dec 2004
    Location
    SF Bay Area
    Posts
    2,188
    Post Thanks / Like
    Likes (Given)
    0
    Likes (Received)
    56

    Default

    I'm more than certain confirmation bias affects measuring things, though are you suggesting it's not happening with a vernier? Or verniers just have less fiddle range, hence less range in which to bias?

    I don't have any digital measuring devices, so I can't speak to that, but when I was first taught to use a mic (age 13, measuring crank journals) I was taught by an old timer that it is done "blind": first you fit the mic by feel, secondly you read it. Or conversely, you lock mic to a size, then you feel its fit on the journal. (That was also my introduction to the fact that "feel" is key to highly accurate measuring, which is not intuitive to the uninitiated.) Measuring blind wasn't presented to me as being about confirmation bias, but rather that you had to separate feeling the mic from reading it. But not long afterward I discovered confirmation bias in myself.

    For a while during my teenage years I had a mic in my desk drawer and I measured things obsessively: various paper, hairs (cat, dog, hamster, people), insect wings, pieces of wire, bits of wood, you name it. Some of these I kept, measured repeatedly, kept notes and compared over time. I did things like try to see how repeatably I could measure the taper of a cat hair. I also tried to train myself to measure by direct feel; I would always guess dimension of something new before measuring it.

    This tinkering revealed what to me at the time was an unexpected degree of variability, and I hunted for explanations for that. That in turn taught me a lot about the effects of thermal expansion, humidity, the repeatability of my "feel", et. al. I found measuring blind to be essential to develop a repeatable feel. But in this process I also discovered that I got more consistent and accurate measurements when I knew in advance what the size of thing was. Eureka! The effect was plain as day to me at the time, and I suspect I am in no way unusual. I bet controlled experiments would easily confirm this (in fact, probably been done)

  5. #5
    Join Date
    Jul 2006
    Location
    Clover Hill district, WI
    Posts
    2,297
    Post Thanks / Like
    Likes (Given)
    0
    Likes (Received)
    8

    Default

    Quote Originally Posted by precision tools View Post
    It would be interesting to have one of these experts measure a set of ten unmarked samples that were close in size to each other. Somehow mix them up as they were dealt out such that each one was eventually measured ten times, but the subject was under the impression that he was measuring 100 unique samples.
    Bet some surprising results would also occure about just how
    accurate he who measures is as well.
    Not discounting the customer's wishes............ tenths are quite small
    in the scope of things. Be it human nerves , axis lash , even the actual
    finished function of a tool or assembly.

    My favorite mentors would abrade the shop's "tenth-tigers".
    Chasing ghosts they would say.
    I decided who to emulate by seeing the finished tooling.
    The tool-makers that lived and breathed mechanics made consistantly
    finer tooling than the guys who had tenths on their minds all the time.
    m1m

  6. #6
    Join Date
    Jan 2005
    Location
    Toledo, OH
    Posts
    1,004
    Post Thanks / Like
    Likes (Given)
    2
    Likes (Received)
    34

    Default

    The likelyhood of this occurring is more prevalent with a dial or digital instrument than a vernier caliper or mechanical micrometer as it is easier to see and force the measurement.

    I have no doubt that I can measure an item a number of times and produce a number of different readings. Ideally, they would be clustered closely about the true dimension within the known +- accuracy of the instrument.

  7. #7
    Join Date
    Jul 2006
    Location
    Clover Hill district, WI
    Posts
    2,297
    Post Thanks / Like
    Likes (Given)
    0
    Likes (Received)
    8

    Default

    What is kind of fun when involved in a conference measurement.
    (Providing a tiebraker) for two co-workers is to calmly read
    and re-read the vernier...... pause the perfect amount ........... .
    and declair a figure a full TWO thou lower than either.

    I would only do this to best fiends. And only rarely.
    I would expect revenge.
    m1m

  8. #8
    Join Date
    Jul 2007
    Country
    UNITED KINGDOM
    Posts
    4,668
    Post Thanks / Like
    Likes (Given)
    2443
    Likes (Received)
    2246

    Default

    I will admit that my personal practice is always to take the measurement and then look at the numbers. It doesn't matter whether you are using a mike, a verynear caliper or a digital one, you can always force it towards the reading you want if you know what it's showing as you take the measurement.

  9. #9
    Join Date
    Jun 2007
    Location
    New Zealand
    Posts
    2,673
    Post Thanks / Like
    Likes (Given)
    0
    Likes (Received)
    28

    Default

    Quote Originally Posted by mark thomas View Post
    I'm more than certain confirmation bias affects measuring things, though are you suggesting it's not happening with a vernier? Or verniers just have less fiddle range, hence less range in which to bias?
    My contention as to the difference was originally sparked by the realisation that a high-resolution vernier does not announce or broadcast the reading, to the last significant digit, in the way a dial or digital does.

    To home in on that last digit, you have to hold the whole deal at an optimal angle for reading, which is usually not the same as the optimal angle for feel.

    Secondly with fine verniers there's a time involved for reflection and comparison to arrive at the best match, so there's no real-time feedback loop with applying the measurement force/ manipulating the angular relationships, unless you consciously (and rather laboriously) cheat.

    Thirdly because in many cases there's a small arithmetical barrier between you and the result, (eg adding .025, .050 or .075, in the inch case) creating another accidental decoupling.

  10. #10
    Join Date
    Jun 2007
    Location
    New Zealand
    Posts
    2,673
    Post Thanks / Like
    Likes (Given)
    0
    Likes (Received)
    28

    Default

    Quote Originally Posted by precision tools View Post
    .....

    It would be interesting to have one of these experts measure a set of ten unmarked samples that were close in size to each other. Somehow mix them up as they were dealt out such that each one was eventually measured ten times, but the subject was under the impression that he was measuring 100 unique samples.
    Interesting idea. I'm also reminded that unmarked workpieces is a preferable scenario for a blind test than the awkward one I proposed.

    it would be interesting to explore confirmation bias by marking all the samples with their nominal dimension, but in a few cases fudging the marked dimension by one "graduation" in a random direction.
    Then do some stats on the results to see if that fudging had 'pulled' the results (and whether some operators were pulled less than others!)

  11. #11
    Join Date
    Jul 2006
    Location
    Clover Hill district, WI
    Posts
    2,297
    Post Thanks / Like
    Likes (Given)
    0
    Likes (Received)
    8

    Default

    Quote Originally Posted by Troup View Post
    Interesting idea. I'm also reminded that unmarked workpieces is a preferable scenario for a blind test than the awkward one I proposed.

    it would be interesting to explore confirmation bias by marking all the samples with their nominal dimension, but in a few cases fudging the marked dimension by one "graduation" in a random direction.
    Then do some stats on the results to see if that fudging had 'pulled' the results (and whether some operators were pulled less than others!
    )
    Yes indeed;
    But it will have to happen in new Zealand.
    Around here first you have to find a shop that's still in buisness.
    Then if your machine center can do with-out you for 12 minutes,
    they find some other trouble for you to get into.
    Not even tiddy up from the last job, or get ready for the next.

    Ah for the good old days when a big deep manual cut let you clean
    and wipe down your paralells. Knock large drill out of the sleaves,
    Maybe walk behind the column and have a look-see how the Ol' girl
    is holding up these days.
    m1m

  12. #12
    Join Date
    Nov 2005
    Location
    Maryland
    Posts
    8,201
    Post Thanks / Like
    Likes (Given)
    746
    Likes (Received)
    1103

    Default

    User bias is certainly a factor in measurements of all types, not just dimensional.

    Modern digital instruments have mitigated this problem to a great extent. Apply proper gauging pressure and activate the Hold button without looking at the reading.

    One common problem that contributes significantly to measurement error and inconsistency with calipers is applying pressure at the slide. The proper method is to press the jaws against the work with your fingers on the jaws, inline with the points of contact in the case of round work pieces.

    - Leigh

  13. #13
    Join Date
    Dec 2000
    Location
    Bremerton WA USA
    Posts
    10,661
    Post Thanks / Like
    Likes (Given)
    40
    Likes (Received)
    4170

    Default

    It's amazingly easy to "wish on" a thou or two so the part size agrees with the plan. Thing is to never look at the mike thimble. vernier, dial, or display when you're actually taking the measurement. You look after you have a good feel and can read the undisturbed size.

    It's difficult to prevent your desires and emotions from clouding objective evidence. Many a time I've had the next guy re-take and verify my measurements. Cheap insurance when the work is pricey and the stock yet to be removed is scant.

  14. #14
    Join Date
    May 2002
    Location
    South Central PA
    Posts
    13,425
    Post Thanks / Like
    Likes (Given)
    2159
    Likes (Received)
    3351

    Default

    What Forrest said.


Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •