How We Test Grills And Smokers

"The one size fits all approach of standardized testing is convenient but lazy."James Dyson, British Designer

I research, test, review and rate grills and smokers full time.

You may think I have the best job in the world. I agree. It's a lot of fun, but I don't take it lightly. AmazingRibs.com is influential, and I understand that my conclusions can both help and hurt.

I do 99% of the grill and smoker reviews on this website. Occasionally I let Meathead or another member of our team test one. Here's how we do it.

I've been reviewing grills and smokers full time since 2012. To my knowledge, nobody else does this full time, and nobody else has cooked on as many grills and smokers as I have. I assemble products, measure  temperatures, slam lids, turn dials, kick casters, study warranties, take pictures, and cook a lot of food. I also consider look, feel, functionality, durability, and price. Given the enormous variety of products and designs, there is no one-size-fits-all test. I cannot simply plug a cooker into a computer and measure its myriad cooking qualities. My tests are both rigorous and thorough, objective and subjective. My overall goal is to provide useful and accurate information to help you select the best equipment for your needs.

AmazingRibs.com Product Testing Policy

We purchase many of the products we review, although manufacturers often send us samples. Before shipping anything, they must agree to our written Product Testing Policy which makes it clear that we will call 'em as we see 'em: "Manufacturers or merchants who send us products to test need to understand that our first obligation is to our readers and we will do our best to review your products honestly and fairly considering price. To make sure we get it right, we always give manufacturers an opportunity to respond to our reviews and address errors, omissions, and concerns. We put a lot of time and effort into testing; therefore we post all reviews whether good or bad."

Rest assured that when we recommend a product, it is because we like it, not because someone has paid us to say so or because the company is an advertiser or sponsor. We adhere to Federal Trade Commission regulations. Suppliers are never charged to have products reviewed or to be included in our listings. We do not make or sell any competitive products, and our advertising is sold by third party ad networks. We don’t do sponsored articles, paid articles, or accept junkets. We disclose all of our material relationships, policies, and business practices. Readers will always know the difference between editorial and advertising. We always present our honest opinions to the best of our ability.

After we are done with samples, if manufacturers do not want them back we often give them away. We never sell them. Due to this policy, many nearby fire departments have nice new grills and smokers. If I ever set my deck on fire, I’m hoping they remember my generosity and get here fast.

Four men loading a grill and a smoker into a pick up truck parked in a driveway. Some wear clothing labeled "FIRE". Red vehicles are parked in the background

Research

In addition to manuals and written materials that come with products, I gather information from various resources. Manufacturers’ websites can be helpful. Although biased, they are often a good source for specifications and company background information. If the product is available online, many e-tailers such as Amazon post buyer reviews, which sometimes provide useful insights, especially if they come from serious, credible reviewers. I always take note of comments from AmazingRibs.com readers and members of our Pitmaster Club who either own or are considering purchasing the tested product. 

The importance of price

Price is one of the primary factors in a consumer’s decision to buy a product. Folks looking for a new grill or smoker often ask what they should buy. My first question is, "What do you want to spend?” When testing, I don't expect a $200 gas grill to have the same quality and features as a $2,000 gas grill. For that reason, I consider the price category of each product and determine whether it is a good value at the price. AmazingRibs.com now has well over 500 products in our Searchable Equipment Reviews priced from $30 all the way up to $50,000. It's easy to pull up comparably priced products for reference, which helps me and the average consumer make an apples-to-apples evaluation of a specific grill or smoker.

About our award system

When I am done testing I rate the cooker on a scale, in this order: Pitmaster Award, Gold Medal, Silver Medal, Bronze Medal, Not Judged, and Not Recommended. Click here to read about what they mean.

Assembly

The best designs are well thought out and elegant, while others are awkward and often difficult to put together. Assembly reveals a lot about construction quality. Holding the various parts and fasteners in my hands gives a clear sense of what's solid and what's flimsy, what is well designed and what is not. Even before firing up the product, I assess design, construction, fit, and finish, with a consideration of price.

I also confirm the specs from the website and the manual. If they say the cooking surface is 36 x 24, I get out the tape measure, and surprisingly, sometimes, shall we say politely, their tape measure and mine are different?

A large black metal object with various parts packed in white protective foam inside.

Virtual Showroom Videos

After assembling the unit,  I usually make a "Virtual Showroom" video that attempts to show you what you might want to see if you walked up to the grill or smoker on a showroom floor. Most videos are shot before cooking tests begin, when the product is still new and shiny. You can find our Virtual Showroom Videos within with my reviews on AmazingRibs.com and on our YouTube Channel.

A word about thermometers

I ignore the built-in bi-metal dial "thermometers" on grills and smokers. They are rarely accurate. I call them “heat indicators”, not thermometers, and they are frequently mounted in the center, making them totally useless if you are using a 2-zone heat setup, which we strongly recommend for most cooking. To make matters worse, they are usually mounted in the lid, far above the food, so they experience much different temps than the food which is closer to the heat source. Click here to learn why you absolutely need digital thermometers to be the best you can be. And click here to read the recommendations of our electrical engineer and thermometer tester, Bill McGrath. I use his top rated precision thermometers to measure cooker and food temperatures. You should too.

For most temperature tests, I use the Fireboard digital thermometer, but the probes and cables cannot withstand temperatures exceeding 716°F. When testing sear burners or super powered main burners on gas grills, I use the Omega digital thermometer with Nextel Ceramic Insulation cables rated at 1,800°F. For high temp tests, I use aluminum discs with holes drilled in the side to accommodate temperature probes. Below is a gas grill with 22,000 BTU burners that hit 920°F.

A shiny steel rack with glowing red flame underneath. Shiny metal disks hooked to thin, white cable rest on the rack.

The importance of 2-zone cooking and searing

There are two functions that we think are essential for a good rating for a grill: 2-zone cooking, and a proper sear on a steak or burger.

1) 2-zone cooking. An essential technique for both charcoal and gas grills, 2-zone cooking is a matter of setting up the grill with one zone of moderate indirect convection heat and a second zone of hot direct infrared (IR) radiant heat. In most cases, you simply split the available grill space in half. This set-up allows you to gently bring up the internal meat temp on the indirect zone, then sear and crisp on the hot direct zone. We think a grill must be designed to allow 2-zone cooking. If not, we think it has a problem. The unique, rectangular shape of the PK charcoal grill pictured below makes it particularly well suited for 2-zone grilling.

A shiny rectangular trough outside at night. The right half is filled with red hot, glowing charcoal.

2) An all-over sear on a steak or burger. Contrary to grilling mythology, criss crossed sear marks on a steak are not the pinnacle of perfection. We want an all over brown surface from the Maillard reaction and caramelization as in the picture below. This creates a delicious brown crust. All-over searing requires intense infrared radiant heat. Grill marks are created by conduction heat from hot metal and they leave much of the surface pale in color. IR is different than air temp. A grill may be able to hit 900°F air temperature but still not be able to make a proper sear if the meat doesn't get exposed to IR.

perfect sear

Charcoal grills have no problem generating high temperature radiant heat. Gas grills are a different story. Most cannot do the job unless they have special sear or IR burners. Click here to read about the thermodynamics of cooking and different types of heat.

For burger tests on grills and smokers, I use Costco's Kirkland brand 1/2 inch thick, 1/3 pound, pre-formed ground sirloin patties to ensure consistency. We understand that these are not the world's greatest burgers, but they are extremely consistent and that is what we need to test.

A plastic bag labeled "Kirkland Ground Sirloin and Loin Beef Patties."

We found the quality of meat, fat content, size and thickness varied when making burgers by hand. Additionally, pre-formed patties are very popular and will likely be on the grills of a large percentage of our readers.

Hamburgers cooking on a bbq grill.

Most smokers are not designed for high temperature searing, but if a manufacturer claims their smoker can double as a grill, I test that function. Below is an example of a pellet “grill” that failed a sear test. Don't waste any tears: I finished searing these steaks indoors on a cast iron skillet. And we gave this smoker a high score as a smoker. But it is not a good grill.

Two thick steaks cooking on a bbq grill.

What we look for in charcoal grills

Good cooking requires temperature control. The hallmark of good charcoal grills is effective air control, which is essential for managing both heat and smoke quality. I look for tight construction and adjustable dampers on the intakes and exhausts. Cooking temperatures on the best charcoal grills are easy to set and maintain. Click here to learn more.

When it comes to searing, a red hot bed of charcoal, even in a low cost grill, can provide more than enough intense, IR radiant heat to sear anything you want. To eliminate variables, I use only fresh Kingsford Original Blue Bag charcoal briquets, known for their consistency and widely used throughout the world. Click here to read about the science of charcoal.

Many models do a pretty good job doubling as smokers. While smoker capability is not a requirement of a good charcoal grill, it is worth bonus points. I note if the stand or cart and wheels are solid. Many charcoal grills are enamel or powder coated. On some cheap units, coatings are already chipped and flaking right out of the box, even if not apparently damaged in shipping. Not all charcoal grills have ash removal systems, but that is a desirable feature and is noted. Some have ash receptacles placed under the charcoal grate that can't be accessed without removing the cooking grate, while others consist of canisters attached under the grill that can be emptied easily, even while cooking.

What we look for in gas grills

Many gas grills have low to moderate powered main burners that struggle to sear foods. Many mass market manufacturers address this either by using heavy cast iron cooking grates, which at least produce grill marks through conduction heat, or with sear burners, which produce more IR heat. They are often ceramic, infrared burners like the one below.

Steak getting searing over a hot, flaming burner.

The best gas grills have even heat across the entire cooking surface rather than extreme hot spots. There are many ways to test for even heat. I like to use chicken wings. I spread a flight of wings evenly across the cooking grate and cook with all burners on low, flipping them at the same time but not moving them around. This provides me with real life feedback on the overall heat profile of the cooking surface. Here is an example of even heat.

Golden brown chicken wings cooking on a bbq grill.

It is not uncommon for lower cost models like the one pictured below to be hotter in the back. On a grill like this, you have to move foods around and swap them front to back and side to side for even cooking and to get everything done at the same time. These grills get demerits. 

Chicken wings cooking on a bbq grill. The ones at the front of the grill are raw while those at the back are cooked.

Sometimes we use white bread to test for hot spots.

bread test

Gas grills usually have a limited low temperature range, requiring 2-zone set-ups with some burners shut off, and others dialed up to 225°F for a smoking zone or 325°F for a roasting zone. With the aid of a digital thermometer, 2-zone setups should not be difficult to achieve. Click here to learn the best setups for gas grills. It's a bit harder to do with a 2-burner gas grill, but some people have personal reasons for wanting them, such as limited space, so I don't knock off points.

Construction quality relative to price is always a factor. Most models under $200 are pretty poorly made, but some buyers prefer replacing a cheapo grill every few years to cleaning and maintaining a higher quality, higher priced model that will stay in service much longer. Disposable gassers are usually warrantied for one year, while better models have extended coverage. Gas grills have more moving parts than charcoal grills, hence more opportunity for failure. In addition to better build, higher priced gassers should have better burners, valves, electronics, heat retention and performance. Those that don't get dinged in my reviews. Conversely, some of my favorite finds are overachieving grills delivering features and performance that easily surpass competitively priced models. All gas grills have grease removal systems. The best designs make it easy to get that junk into a receptacle and get rid of it.

What we look for in smokers

If you really love ribs, pulled pork, brisket and smoked salmon, nothing beats a dedicated smoker.

Smoked meat sliced on a cutting board.

I apply several standard tests to grills and smokers, but unique products in the world of smokers makes standardization difficult. Wood burning offset smokers require a fair amount of skill, while thermostatically controlled pellet smokers are as easy to use as an indoor kitchen oven. Evaluating the former may require a detailed assessment of heat control, while heat control on the latter is completely automated. In between those two extremes are gas smokers, charcoal smokers, and electric models in all shapes and sizes. Some are easy to understand and use, while others present challenges.

Smoke quality is another issue. Sweet clean smoke produces the best barbecue while black bitter smoke bastes meats with nasty creosote. Unfortunately, some designs that produce the best quality smoke are the hardest to operate. Many smokers suffer dramatically from user error, so I pay extra attention to manufacturer's instructions to make sure I don't miss any idiosyncratic steps that would negatively affect performance. Click here for info about smoke quality.

As always, build quality in relation to price is important. Ash removal systems are common on some smokers, while others such as kamados and pellet smokers, rarely have them. Remote monitoring and temperature control is also on the rise with thermostatically controlled electric smokers and pellet smokers. For these products, my reviews also include connectivity and app design. All in all, when I find smokers that are affordable, easy to use, and produce great results, it's time to bring out our AmazingRibs.com Best Value Gold Medal Awards.

Food Donations

As you may imagine, my family can't eat all the food I cook. So I often bring grilled and smoked goodies to The Community Crisis Center in Elgin, IL. The Community Crisis Center was founded in 1975 by a group of dedicated volunteers to meet the needs of individuals and families in crisis. Employees provide comprehensive services to those in need due to domestic violence, sexual assault, or economic/financial difficulties. They provide Safety, Hope, Advocacy, Respite and Education for those impacted by crisis and violence in Illinois. AmazingRibs.com is happy to share barbecue with this wonderful organization. Click here to learn more.

A group of people wearing matching purple shirts, posing for a picture on a stairway.

Attention manufacturers, merchants, ad agencies, and others

If you have a product you want us to review, contact us. Max Good is our full time Grill and Smoker Tester. Bill McGrath, an electrical engineer, is our Thermometer Tester. If you have earned an "AmazingRibs.com Best Value Medal", there is no licensing fee to use copies of the artwork, but you must contact us for permission first. If you need them, high resolution color and black & white versions are also available. If you refer to your medal in publicity or on your website, please call it by its proper name: "The AmazingRibs.com Pitmaster Award For Best Value", "The AmazingRibs.com Best Value Gold Medal", "The AmazingRibs.com Best Value Silver Medal", or "AmazingRibs.com Best Value Bronze Medal". On the internet, you should link to this page.

Red rectangular cabinet with shiny handles and two latched doors. Thin white cables from a small, yellow electronic device have been placed inside the box

bbq support

Many merchants pay us a small referral fee when you click our links and purchase from them. On Amazon it works on everything from grills to diapers, they never tell us what you bought, and it has zero impact on the price you pay, but has a major impact on our ability to improve this site! And remember, we only recommend products we love. If you like AmazingRibs.com, please save this link and use it every time you go to Amazonhttps://tinyurl.com/amazingribs

If you love barbecue and grilling you should join our Pitmaster Club. We can up your game.

  • FREE 30 day trial membership.
  • We block ads on the entire site.
  • The Pit forum brims with recipes and info.
  • A real community with no politics or flame wars.
  • Monthly newsletter.
  • Video seminars with famous pitmasters.
  • Weekly podcasts with Greg Rempe.
  • Weekly BBQ cartoons by Jerry King.
  • Comprehensive Temperature Guide Magnet ($10 retail).
  • Monthly giveaways of Gold Medal grills and smokers worth up to $2000.
  • Discounts on products we love.
  • Support for AmazingRibs.com!

 Click here to learn more and to join


Post comments and questions below

grouchy?

1) Please try the table of contents or the search box at the top of every page before you ask for help.

2) Try to post your question to the appropriate page.

3) Tell us everything we need to know to help such as the type of cooker and thermometer. Dial thermometers are often off by as much as 50°F so if you are not using a good digital thermometer we probably can't help you with time and temp questions. Please read this article about thermometers.

4) If you are a member of the Pitmaster Club, your comments login is probably different.

 Click to Show Comments or Add Your Own