How We Test Grills And Smokers

I research, test, review and rate grills and smokers full time.

You may think I have the best job in the world. I agree. It’s a lot of fun, but I don’t take it lightly. AmazingRibs.com is influential, and I understand that my conclusions can both help and hurt.

I do most of the grill and smoker reviews on this website. Occasionally I let Meathead or another member of our team test one. Here’s how we do it.

I’ve been reviewing grills and smokers full time since 2012. To my knowledge, nobody else does this full time, and nobody else has cooked on as many grills and smokers as I have. I assemble products, measure  temperatures, slam lids, turn dials, kick casters, study warranties, take pictures, and cook a lot of food. I also consider look, feel, functionality, durability, and price. Given the enormous variety of products and designs, there is no one-size-fits-all test. I cannot simply plug a cooker into a computer and measure its myriad cooking qualities. My tests are both rigorous and thorough, objective and subjective. My overall goal is to provide useful and accurate information to help you select the best equipment for your needs.

Here’s a key to our grill and smoker reviews

Cooked On It. This means we have hands-on experience testing this product. We have also gathered info from the manufacturer, owners, and other reliable sources.

Looked Closely At It. This means we have seen this product up close and we have also gathered info from the manufacturer, owners, and other reliable sources.

Researched It From Afar. This means we have researched this product from afar by gathering info from the manufacturer, owners, and other reliable sources, with our BS meters on high.

AmazingRibs.com Product Testing Policy

We purchase many of the products we review, although manufacturers often send us samples. Before shipping anything, they must agree to our written Product Testing Policy which makes it clear that we will call ’em as we see ’em: “Manufacturers or merchants who send us products to test need to understand that our first obligation is to our readers and we will do our best to review your products honestly and fairly considering price. To make sure we get it right, we always give manufacturers an opportunity to respond to our reviews and address errors, omissions, and concerns. We put a lot of time and effort into testing; therefore we post all reviews whether good or bad.”

Rest assured that when we recommend a product, it is because we like it, not because someone has paid us to say so or because the company is an advertiser or sponsor. We adhere to Federal Trade Commission regulations. Suppliers are never charged to have products reviewed or to be included in our listings. We do not make or sell any competitive products, and our advertising is sold by third party ad networks. We don’t do sponsored articles, paid articles, or accept junkets. We disclose all of our material relationships, policies, and business practices. Readers will always know the difference between editorial and advertising. We always present our honest opinions to the best of our ability.

After we are done with samples, if manufacturers do not want them back we often give them away. We never sell them. Due to this policy, many nearby fire departments have nice new grills and smokers. If I ever set my deck on fire, I’m hoping they remember my generosity and get here fast.

Research

In addition to manuals and written materials that come with products, I gather information from various resources. Manufacturers’ websites can be helpful. Although biased, they are often a good source for specifications and company background information. If the product is available online, many e-tailers such as Amazon post buyer reviews, which sometimes provide useful insights, especially if they come from serious, credible reviewers. I always take note of comments from AmazingRibs.com readers and members of our Pitmaster Club who either own or are considering purchasing the tested product.

The importance of price

Price is one of the primary factors in a consumer’s decision to buy a product. Folks looking for a new grill or smoker often ask what they should buy. My first question is, “What do you want to spend?” When testing, I don’t expect a $200 gas grill to have the same quality and features as a $2,000 gas grill. For that reason, I consider the price category of each product and determine whether it is a good value at the price. AmazingRibs.com now has well over 600 products in our Searchable Equipment Reviews priced from $30 all the way up to $50,000. It’s easy to pull up comparably priced products for reference, which helps me and the average consumer make an apples-to-apples evaluation of a specific grill or smoker.

About our award system

When I am done testing I rate the cooker on a scale, in this order: Platinum Medal, Gold Medal, Silver Medal, Bronze Medal, Not Judged, and Not Recommended. Click here to read about what they mean.

Assembly

The best designs are well thought out and elegant, while others are awkward and often difficult to put together. Assembly reveals a lot about construction quality. Holding the various parts and fasteners in my hands gives a clear sense of what’s solid and what’s flimsy, what is well designed and what is not. Even before firing up the product, I assess design, construction, fit, and finish, with a consideration of price.

I also confirm the specs from the website and the manual. If they say the cooking surface is 36 x 24, I get out the tape measure, and surprisingly, sometimes, shall we say politely, their tape measure and mine are different?

Virtual Showroom Videos

After assembling the unit,  I usually make a “Virtual Showroom” video that attempts to show you what you might want to see if you walked up to the grill or smoker on a showroom floor. Most videos are shot before cooking tests begin, when the product is still new and shiny. You can find our Virtual Showroom Videos within with my reviews on AmazingRibs.com and on our YouTube Channel.

A word about thermometers

I ignore the built-in bi-metal dial “thermometers” on grills and smokers. They are rarely accurate. I call them “heat indicators”, not thermometers, and they are frequently mounted in the center, making them totally useless if you are using a 2-zone heat setup, which we strongly recommend for most cooking. To make matters worse, they are usually mounted in the lid, far above the food, so they experience much different temps than the food which is closer to the heat source. Click here to learn why you absolutely need digital thermometers to be the best you can be. And click here to read the recommendations of our electrical engineer and thermometer tester, Bill McGrath. I use his top rated precision thermometers to measure cooker and food temperatures. You should too.

For most temperature tests, I use the Fireboard 6-Channel Digital Thermometer.

But the Fireboard probes and cables cannot withstand temperatures exceeding 716°F. When testing sear burners or super powered main burners on gas grills, I use the Omega digital thermometer with Nextel Ceramic Insulation cables rated at 1,800°F. For high temp tests, I use aluminum discs with holes drilled in the side to accommodate temperature probes. Below is a gas grill with 22,000 BTU burners that hit 920°F.

The importance of 2-zone cooking and searing

There are two functions that we think are essential for a good rating for a grill: 2-zone cooking, and a proper sear on a steak or burger.

1) 2-zone cooking. An essential technique for both charcoal and gas grills, 2-zone cooking is a matter of setting up the grill with one zone of moderate indirect convection heat and a second zone of hot direct infrared (IR) radiant heat. In most cases, you simply split the available grill space in half. This set-up allows you to gently bring up the internal meat temp on the indirect zone, then sear and crisp on the hot direct zone. We think a grill must be designed to allow 2-zone cooking. If not, we think it has a problem. The unique, rectangular shape of the PK charcoal grill pictured below makes it particularly well suited for 2-zone grilling.

2) An all-over sear on a steak or burger. Contrary to grilling mythology, criss crossed sear marks on a steak are not the pinnacle of perfection. We want an all over brown surface from the Maillard reaction and caramelization as in the picture below. This creates a delicious brown crust. All-over searing requires intense infrared radiant heat. Grill marks are created by conduction heat from hot metal and they leave much of the surface pale in color. IR is different than air temp. A grill may be able to hit 900°F air temperature but still not be able to make a proper sear if the meat doesn’t get exposed to IR.

Charcoal grills have no problem generating high temperature radiant heat. Gas grills are a different story. Most cannot do the job unless they have special sear or IR burners. Click here to read about the thermodynamics of cooking and different types of heat.

For burger tests on grills and smokers, I use Costco’s Kirkland brand 1/2 inch thick, 1/3 pound, pre-formed ground sirloin patties to ensure consistency. We understand that these are not the world’s greatest burgers, but they are extremely consistent and that is what we need to test.

We found the quality of meat, fat content, size and thickness varied when making burgers by hand. Additionally, pre-formed patties are very popular and will likely be on the grills of a large percentage of our readers.

Most smokers are not designed for high temperature searing, but if a manufacturer claims their smoker can double as a grill, I test that function. Below is an example of a pellet “grill” that failed a sear test. Don’t waste any tears: I finished searing these steaks indoors on a cast iron skillet. And we gave this smoker a high score as a smoker. But it is not a good grill.

What we look for in charcoal grills

Good cooking requires temperature control. The hallmark of good charcoal grills is effective air control, which is essential for managing both heat and smoke quality. I look for tight construction and adjustable dampers on the intakes and exhausts. Cooking temperatures on the best charcoal grills are easy to set and maintain. Click here to learn more.

When it comes to searing, a red hot bed of charcoal, even in a low cost grill, can provide more than enough intense, IR radiant heat to sear anything you want. To eliminate variables, I use only fresh Kingsford Original Blue Bag charcoal briquets, known for their consistency and widely used throughout the world. Click here to read about the science of charcoal.

Many models do a pretty good job doubling as smokers. While smoker capability is not a requirement of a good charcoal grill, it is worth bonus points. I note if the stand or cart and wheels are solid. Many charcoal grills are enamel or powder coated. On some cheap units, coatings are already chipped and flaking right out of the box, even if not apparently damaged in shipping. Not all charcoal grills have ash removal systems, but that is a desirable feature and is noted. Some have ash receptacles placed under the charcoal grate that can’t be accessed without removing the cooking grate, while others consist of canisters attached under the grill that can be emptied easily, even while cooking.

What we look for in gas grills

Many gas grills have low to moderate powered main burners that struggle to sear foods. Many mass market manufacturers address this either by using heavy cast iron cooking grates, which at least produce grill marks through conduction heat, or with sear burners, which produce more IR heat. They are often ceramic, infrared burners like the one below.

The best gas grills have even heat across the entire cooking surface rather than extreme hot spots. There are many ways to test for even heat. I like to use chicken wings. I spread a flight of wings evenly across the cooking grate and cook with all burners on low, flipping them at the same time but not moving them around. This provides me with real life feedback on the overall heat profile of the cooking surface. Here is an example of even heat.

It is not uncommon for lower cost models like the one pictured below to be hotter in the back. On a grill like this, you have to move foods around and swap them front to back and side to side for even cooking and to get everything done at the same time. These grills get demerits.

Sometimes we use white bread to test for hot spots.

Gas grills usually have a limited low temperature range, requiring 2-zone set-ups with some burners shut off, and others dialed up to 225°F for a smoking zone or 325°F for a roasting zone. With the aid of a digital thermometer, 2-zone setups should not be difficult to achieve. Click here to learn the best setups for gas grills. It’s a bit harder to do with a 2-burner gas grill, but some people have personal reasons for wanting them, such as limited space, so I don’t knock off points.

Construction quality relative to price is always a factor. Most models under $200 are pretty poorly made, but some buyers prefer replacing a cheapo grill every few years to cleaning and maintaining a higher quality, higher priced model that will stay in service much longer. Disposable gassers are usually warrantied for one year, while better models have extended coverage. Gas grills have more moving parts than charcoal grills, hence more opportunity for failure. In addition to better build, higher priced gassers should have better burners, valves, electronics, heat retention and performance. Those that don’t get dinged in my reviews. Conversely, some of my favorite finds are overachieving grills delivering features and performance that easily surpass competitively priced models. All gas grills have grease removal systems. The best designs make it easy to get that junk into a receptacle and get rid of it.

What we look for in smokers

If you really love ribs, pulled pork, brisket and smoked salmon, nothing beats a dedicated smoker.

I apply several standard tests to grills and smokers, but unique products in the world of smokers makes standardization difficult. Wood burning offset smokers require a fair amount of skill, while thermostatically controlled pellet smokers are as easy to use as an indoor kitchen oven. Evaluating the former may require a detailed assessment of heat control, while heat control on the latter is completely automated. In between those two extremes are gas smokers, charcoal smokers, and electric models in all shapes and sizes. Some are easy to understand and use, while others present challenges.

Smoke quality is another issue. Sweet clean smoke produces the best barbecue while black bitter smoke bastes meats with nasty creosote. Unfortunately, some designs that produce the best quality smoke are the hardest to operate. Many smokers suffer dramatically from user error, so I pay extra attention to manufacturer’s instructions to make sure I don’t miss any idiosyncratic steps that would negatively affect performance. Click here for info about smoke quality.

As always, build quality in relation to price is important. Ash removal systems are common on some smokers, while others such as kamados and pellet smokers, rarely have them. Remote monitoring and temperature control is also on the rise with thermostatically controlled electric smokers and pellet smokers. For these products, my reviews also include connectivity and app design. All in all, when I find smokers that are affordable, easy to use, and produce great results, it’s time to bring out our AmazingRibs.com Best Value Gold Medal Awards.

Food Donations

As you may imagine, my family can’t eat all the food I cook. So I often bring grilled and smoked goodies to The Community Crisis Center in Elgin, IL. The Community Crisis Center was founded in 1975 by a group of dedicated volunteers to meet the needs of individuals and families in crisis. Employees provide comprehensive services to those in need due to domestic violence, sexual assault, or economic/financial difficulties. They provide Safety, Hope, Advocacy, Respite and Education for those impacted by crisis and violence in Illinois. AmazingRibs.com is happy to share barbecue with this wonderful organization. Click here to learn more.

Attention manufacturers, merchants, ad agencies, and others

If you have a product you want us to review, contact us. Max Good is our full time Grill and Smoker Tester. Bill McGrath, an electrical engineer, is our Thermometer Tester. If you have earned an “AmazingRibs.com Best Value Medal”, there is no licensing fee to use copies of the artwork, but you must contact us for permission first. If you need them, high resolution color and black & white versions are also available. If you refer to your medal in publicity or on your website, please call it by its proper name: “The AmazingRibs.com Platinum Medal For Best Value”, “The AmazingRibs.com Best Value Gold Medal”, “The AmazingRibs.com Best Value Silver Medal”, or “AmazingRibs.com Best Value Bronze Medal”. On the internet, you should link to this page.