Sleeping Bag Temperature Ratings: Fact or Fantasy?

 The E13537 Dummy

Introduction

The European Sleeping Bag Temperature Rating Standard (EN13537) was supposed to be a great thing for outdoor manufacturers, retailers, and consumers. Finally, we’d be able to compare sleeping bag temperature ratings in a reliable and objective manner.

Ah, I have some bad news. I was browsing the Outdoor Industry Association’s web site last week and came across a June 2009 report that shows that the EN13537 standard is potentially misleading. I was shocked to come across this paper, so I contacted it’s author, Dr. Elizabeth McCullough, a noted textile scientist at the Kansas State University, and she explained the problems with the testing procedures to me in more detail.

Everyone’s Manikin is Different

But before I dive into those, it’s useful to understand how the temperature rating test is performed. REI has a good explanation of the EN 13537 standard in the experts tips section of their web site. Quoting them, “For EN 13537 temperature tests, a full-size mannequin with heaters and temperature sensors is dressed in one layer of long underwear and a hat. It is placed inside the sleeping bag being tested. The bag is laid atop an insulating sleeping pad inside a climate-controlled chamber. The mannequin is heated to simulate body warmth and measurements are taken of the air in the climate chamber and the “skin” surface of the mannequin. From these measurements, the insulation value of the sleeping bag is calculated.”

Currently, there are 5 different testing centers around the world that test sleeping bag temperature ratings using this setup, including Dr. McCullough’s labs at KSU. The problem, says Dr. McCullough is that every lab’s manikin is different and that there is no common definition of the thermal properties of the tracksuits, socks, hat (really a face mask), and sleeping pads, used in the test process.

Consequently,  the results of the different centers may vary significantly, and while the test itself is scientifically sound, she’s been advocating for a consistent definition of these non-standard testing variables. As it stands, inter-lab variability is unknown, making it impossible to compare temperature ratings from the different centers.

Sadly, this has not happened. As it stands, the European standards group responsible for the temperature rating standard is not allowing the Outdoor Industry Association (OIG) to attend the group’s standards meetings. Dr. McCullough has advocated the creation of a consensus document from the OIG to the EN standards body protesting the current test definition, but progress along these lines has been slow. Dead slow.

Who Can You Trust?

Adoption of EN13537 is now widespread in Europe, and several American sleeping bag manufacturers including REI and Marmot have started labeling their retail sleeping bags with it.

Unfortunately, Dr. McCullough advised me that highly insulated sleeping bags could vary as much as 20 degrees (F) in their ratings and that there are no common test procedures defined for non-mummy bags. In addition, other researchers have documented additional problems with the test procedures (see papers linked below).

Where does that leave consumers and why haven’t sleeping bag manufacturers that have adopted EN 13537 been more forthcoming about these issues?

Background Publications

http://www.outdoorindustry.org/pdf/EN13537Mccullough062209.pdf

http://www.outdoorindustry.org/pdf/EN13537TestLabComparisonMethod.pdf

Most Popular Searches

  • sleeping bag ratings explained
  • 13537
  • EN RATING

, , , ,

18 Responses to Sleeping Bag Temperature Ratings: Fact or Fantasy?

  1. Dave Hollin September 28, 2009 at 12:20 pm #

    food for thought and no mistake. I know there are differences in the testing parameters/protocols and in things like lofting and down rating etc but 20 degrees difference is quite a wide variance

  2. Jim Gibliln September 28, 2009 at 7:44 pm #

    Philip, this is an amazing site/blog congratulations and thank you for writing.

    Coming up with a standard to provide a single temperature rating number for an entire bag or “insulation system” has proven difficult. The variables are endless and that single number is an average based on doing everything possible to define and generalize humans and the environment as well as take into consideration sleeping bag design and sleep systems. Relying solely on a single number temp rating of a sleeping bag for survival (and/or comfort) would be “extremely uninformed”. The recent EN test method and standard tries to provide that single number for the consumer for product comparison but at the same time defaults to a “comfort range”. It also speaks to the average temperature differences between men and women. Check out the EN standard hang tags on sleeping bags they can help explain a lot. Also Google “Sleep Well Mammut” it is great at addressing both human and environmental variables. The EOG is currently rewriting the standard for release next year and claims to have moved from a 5 percent variable to a 3 percent in testing. Liz’s paper is both accurate and goes much deeper in regards to lab variables and testing. To really judge the standard as bad is ok if you understand the depth of lab testing data and your perspective is well informed and you can compare test methods. For the customer the EN standard is big step in the right direction for providing more accurate information for comparison and it is getting even better. Do you understand the difference Mr. Myers? Put down the axe Michael.

  3. Tom Murphy September 28, 2009 at 8:23 pm #

    I think you mis-stating her findings.

    From her summary:

    "Although there are some problems with the EN standard, it is a comprehensive document that is based on sound science, and the prediction results have been validated on human subjects."

    Although I have read the European Sleeping Bag Temperature Rating Standard (EN13537), I do not have enough experience in product testing to be able to directly challenge Dr. McCullough's paper. In fact, I think most of her suggestions have merit.

    I do know that there are always difficulties trying to harmonize American and European standards & that standards can always improve. Her suggestions seemed to be aimed at leveling the field so that one lab doesn't produce more generous results than others.

    American sleeping bag manufacturer's are not testing their bags to the ASTM standard that Dr. McCullough's paper references so an imperfect EN standard is still better than no standard.

    Dr. McCullough's main criticism is that there may be some inter-lab variability but she does not offer any evidence to support this.

    The other report referenced gives us the following paragraph:

    "The results imply that the test accuracy achieved is + 1.8° C on the T Comfort, and + 2.6° C on T Extreme. Put simply, these results suggest that the test is only half as accurate as it is specified. In other words, the “error bars” should be twice as big as stated."

    So their criticism was that the EN standard produced an accuracy of "ONLY" 1.8 deg C and not to the specified 1.0 deg C accuracy rhat the EN standard sates that it can achieve.

    5 deg F is accurate enough for me and far better than the fantasy ratings that the American sleeping bag manufacturer's have been giving us.

    The real question is why haven't the other sleeping bag manufacturuers applied the standard? Hiding behind trivial issues with the standard is a smoke screen. They could always have their bags tested at multiple labs and then issue the best rating. Ibelieve that their already know what temp rating their bags would achieve per the EN standard and do not want to improve their product.

    As anecdotal evidence, I know that my MARMOT bag which is certified to 15 deg F works well below that rating for me. I also respect the fact that MARMOT doesn't hide the fact that their -20 and -40 bags are not certifed.

    TJM

  4. Earlylite September 28, 2009 at 11:51 pm #

    Tom – Honestly, my intent behind the post was to bring the issues to light not condemn the test outright. Liz just wants to nail down the variables more so that the test can be consistently and repeatedly implemented and a comparison between labs has some meaning.

    Consider this problem: 8 sleeping bags were used to calibrate the original test. Then they were packed up and sent to another lab to be used as calibration bags. But guess what, they lost some loft after being transferred between centers. Those same 8 bags are in tatters now after being tested so many times. Substituting newly manufactured replacements doesn't work because they use slightly different materials and don't generate the same results. It's not like they're testing a kilogram of water, whose weight doesn't change between tests.

    Moreover, as it is being marketed (with the exception of Mammut), the test is potentially misleading because it measures the sleep system in its entirety and not the bag alone. This isn't bad, but anyone buying a 20 degree bag will be surprised when they learn that they have to sleep on a 1.5 inch pad, wear long underwear, socks, and a hat to achieve the temperature rating that they thought the bag was rated for.

  5. Earlylite September 28, 2009 at 11:59 pm #

    Jim – thanks for the update on the European Outdoor Groups revisions. That is big news and not something that is well known, even among industry insiders like Liz.

    I agree that Mammut (a brand most Americans have never heard of) does an excellent job about explaining the importance of the environmental variables – and I think their inclusion in the standard is actually a good thing – since it emphasizes the need to think about a sleep system as a whole and not just one's sleeping bag. I just question whether American and European consumers can ever understand that and if sleeping bag marketers are willing to complicate their messages with that info. I'm a product manager by trade, and know all too well how technical product features get smoothed over and simplified in product packaging and in the distribution channel.

  6. Earlylite September 29, 2009 at 12:15 am #

    Dave – the variances are greatest in high loft bags, and hopefully the people buying those "know better" than to buy a bag on it's rating alone and understand the need to build a complete sleep system around it. For example, "how to extend a 20 degree bag down to 0 with a sleeping bag cover/bivy and a thicker pad." But, that's probably wishful thinking.

  7. niall maplesden September 29, 2009 at 12:34 am #

    Dear Earlylite,

    you are partly right and are partly missing the point. You mention that the sleep system as a whole is measured, that is true, but if you always use the same clothing ensemble, then that is a fixed amount, not a variable. This leaves you with a single amount that is added, it is also based around a type examination and a part personal protective equipment standard. That is why the comfort and extreme levels are given.

    I will not disagree that the test is variable, but its a lot better than buying a bag on fill weight alone. The main problem is that sleeping bag manufacturers have had many years to build up a business without much testing needed. There were standards from 6 different countries all of which will give different results. This is a ridiculous situation and hence CEN got involved and developed this standard. One of the most contentious issues regarding this standard is that the data generated is based on data taken from the German military, hence the comfort and extreme ratings are based around very fit people, but alternatively where else can you get a series of volunteers to sleep in extreme conditions with sensors attached to them?

    The data on comparisons that is claimed in the article not to exist, does exist and is in the public domain. You simply have to write to the committee chairperson and request it as an interested party. The european outdoor group and the outdoor industry association have not been blocked from joining the committee, they are interested parties and have every right to join the technical committee, they may have been blocked from joining the working group, but that may be because a large working group is more cumbersome whereas a small working group can get the job done in a smaller timescale. If you join the technical committee, you still get to comment on the work done and get to see the results of the work done.

    The report from Dr. Mc Collough is very informative, but misses one of the salient points. In standards created for America, you can stipulate a specific piece of equipment for a specific test, this has the advantage that you should get a good inter laboratory repeatability, but has the commercial disadvantage that you have to buy the equipment from a specific supplier. EN standards and ISO standards are not allowed to stipulate a specific manufacturer, they have to describe the equipment in detail so that it could be built by anyone. This always leads to a larger variability, hence the need for a set of standard materials to help 'calibrate' the equipment. You mention in your post that these bags lost some loft when transported between labs, but ask yourself this what is the variability of bags on a production line? Sleeping bags are reasonably complicated and thermal resistance, which is actually what is measured, is actually a measure of the amount of trapped air in a product. Hence the loftier the bags, the higher the thermal resistance. Overfilling a bag will result in a higher fill weight, but may result in a lower thermal resistance because there is less trapped air. Knowing this I ask again what is the variability of a sleeping bag production line? Sleeping bag manufacturers simply don't want that question asked.

    I also read the sleep well paper from mammut, I have even seen that presented by a member of mammuts management team. The data that is missing from the online paper but was present in the presentation was that although they disagree with the standard, and tried to prove that it is inconsistent, their customer complaints have dropped to nearly zero since having them tested and providing the information.

    In real terms, this standard is not perfect, no standard is, and people will have tests done on three criteria:

    1 cost

    2 result, if i get the result I want, I will pay more

    3 Wheteher my clients demand it, no testing would be performed if the custoomers didn't demand it.

  8. Earlylite September 29, 2009 at 12:50 am #

    Niall – thank you for the lengthy comment and for bringing to light so much more additional information. I've spearheaded industry standards myself and know all of the compromises that must be made to get something out.

    Interesting data point on the benefits to Mammut in terms of complaint reduction. Can I ask what your relationship to them was that let you obtain this information? Fascinating background.

  9. niall maplesden September 29, 2009 at 1:07 am #

    I have no relationship with mammut, I was at a lecture where they were offering a presentation. Its odd what you remember isn't it

  10. Tom Murphy September 29, 2009 at 8:48 am #

    Phil,

    Great topic with many excellent and informed comments.

    This might be a tangential topic, but you mentioned bivys and perhaps you could expand on this topic by discussing how bivys extend the sleeping system.

    I have read postings on the Winter Trekking website discussing bivys [http://wintertrekking.com/index.php?action=article_view&a_id=45] but I am still not sure if it is applicable to New England winters.

    From what I have read, the bivy's function is to increase the temperature at the outer shell of your sleeping bag so that moisture condenses on the inner liner of the bivy rather than inside the sleeping bag. However, I assume that bivys are not very breathable so you are trapping that moisture inside the bivy. So if it is deep cold the trapped moisture turns to frost on the inside of the bivy. But what about in New England, will the bivy just trap moisture and soak the outer layer of my sleeping bag?

    Thanks,

    Tommy

  11. Earlylite September 29, 2009 at 9:56 am #

    Yes, I love it when we get a good comment thread going with such erudite guests.

    Regarding bivies. Think of them as a vapor barrier like a rain coat or shell. They naturally trap heat. Some are made with highly breathable fabrics like eVent and vent moisture. For example, I just picked up a <a hred="http://www.backcountry.com/outdoorgear/MontBell-Breeze-Dry-Tec-UL-Sleeping-Bag-Cover/MTB0092M.html?CMP_ID=SH_FRO001&CMP_SKU=MTB0092&mv_pc=r126&quot; rel="nofollow">Montbell Breeze which I plan on using a few weeks in the Whites with a Thermarest Neoair Pad and a Golite 20 quilt. I got the breeze – which is technically a sleeping bag cover and not a pure mountaineering bivy – for keeping my bag dry when sleeping under a tarp (rain-splatter), but expect it to provide some additional warmth, as well. It's made form something called Dry-tec, which is purported to be very breathable.

    But you are correct in being concerned about using bivies in New England. Depending on the humidity and their breathability, they can make you wet. So, test, test.

  12. Dave Hollin September 29, 2009 at 12:15 pm #

    Like you say, I always go for a combined sleep system and carefully look at not just the sleeping bag but also the pad and the clothing I wear. I think there is quite a lot of fun to be had carefully working out what works well for a given set of conditions.

  13. Earlylite September 29, 2009 at 12:49 pm #

    Completely agree. It's an art form.

  14. Bill McGrane December 10, 2009 at 6:17 am #

    Quick questions. I have an Alps Clearwater 20 deg. bag plus a poly liner & a fleece liner. Approx. how much more warmth (in degrees) should those two liners afford? And how much more w/ a bivy? Many thanks. Best regards, Bill McGrane

  15. Earlylite December 10, 2009 at 6:26 am #

    A liner or a bivy is probably worth another 10 degrees (F), but I can't say how much 2 would contribute. Another alternative is use two twenty degree bags. You didn't say what your sleeping pad system is or how cold the ground is. This is actually very important since the part of the bag you sleep on does nothing to insulate you.

  16. Tom Murphy December 10, 2009 at 8:00 am #

    Bill,

    I did extensive searching on the Internet for this type of information.

    BIG AGNES has information on using two-bag systems and the expected improvement. From that information, I inferred the following rule of thumb.

    Combined Rating = Lower Rated Bag – [(75 - Higher Rated Bag) / 2]

    This is only an approximation and assumes that the two bags fit together without loss of loft for either bag.

    You should also use the Bag Rating that you have proven to yourself. By this I mean a bag is not a 0 deg F bag until you have actually slept in it at 0 deg F.

    For example, combining my 15 deg syn bag and my 15 deg down bag.

    § The 15 deg syn bag has only worked to 30 deg F for me.

    § The 15 deg F down bag has worked down to 20 deg F (it might be good to 15 deg F but I haven't used it alone below +20 deg F yet).

    § Therefore my guess of what my two bag system can do = 20 – [(75-30)/2] = – 2.5 deg F

    § This rating of minus 2.5 deg F already includes the benefit from my thin wool top/bottom base layer on since that is what I had on when I slept outside in the down bag at 20 deg F

    I found a table that compared Inches of Insulation vs. Sleeping Temperature that could be used as a general guide when trying to estimate a sleeping system’s performance.

    Remember, for a sleeping bag, half of the insulation is above you and half below, so the total bag loft should be at least twice the values in this table.

    US Army Quartermaster insulation table

    Temp sleeping

    40F 1.5"

    20F 2.0"

    . 0F 2.5"

    -20F 3.0"

    -40F 3.5"

    -60F 4.0"

    The heat source when sleeping outside is our metabolism and we all have different metabolisms so we all require different amounts of insulation for the same ambient temps.

    Our metabolisms vary day to day and during the day. Did you eat a large supper with a good amount of fats? Do you have a snack (chocolate or cheese) ready for just before going to bed? Are you hydrated? Are you exhausted?

    So my advice is to test your system at the temperature you are expecting but in a safe environment (backyard or car camping) before using it in the backcountry. And always have a safe margin.

    Finally, here is another table to use as a guide.

    Vendor Rating vs. Total Loft (top+bottom) from actual sleeping bags:

    20F 5"

    0F 6"

    -20F 9"

    -40F 12"

  17. Bill McGrane December 10, 2009 at 6:09 pm #

    Phil & Tom,

    Many thanks. Excellent info plus excellent site. Thank you.

  18. eddie s April 26, 2010 at 7:07 am #

    The early issues of "Backpacker" Magazine before the "old man died" were an excellent source of testing information. Real people in various sizes and shapes and clothing tested the equipment and used their own scales to weigh in the "claimed" weights versus real weights…Now I feel Backpacker is just a front for the Marketing manufacturers as noted with terms like "Reasonable Affordable" and other bogus terms and they no longer use "untrained" people to test equipment and they no longer list weights such as "claimed" and "Real" on the Backpacker scales. I hear they also fired all those people who objected to Ma and Theresa doing this…

    So again the Industry lacks any real creditable information and with so many Stores tightening up their "return policy" it is becoming more important than ever to find a true independent source of information….So sad, so sad,,Right now I am looking for a creditable source on the rating of the Marmot Pounder 40 Degree bag..thats why I stopped by here…

Leave a Reply