I have the same setup as you, the Cajun Bandit ring on a newish master touch. I used high temp flue tape to seal the extra set of holes.
https://www.amazon.com/3M-High-Temperature-Flue-15-Foot/dp/B00004Z4DS/I also put a gasket on the lid, more to decrease the tinny feeling of the lid than for leaks.
I have not had the high temp issues you have encountered. I think the issue is too much charcoal.
I use two small Weber chimneys. I don't quite fill them all the way. I dump the lit coals into the char baskets before they completely ash over. The char baskets are spread as far apart as they can be because I put a Costco disposable drip pan between the baskets and it is a smidge larger than the Weber ones. It gives me a perfect 375-425 every time from my now dwindling stash of Weber charcoal. On one occasion, I didn't get the color I was looking for on the bird, so the last few minutes I pulled the drip pan, pushed the baskets together and cooked direct.
Using two small chimneys, rather than a single large one, has been a game changer for me in charcoal grilling. Using a big chimney, the coals at the bottom are almost used up when the coals on top are just getting going, and two small chimneys are ready sooner than one large one. This is specially true when using a fast-lighting & fast-burning charcoal like K-Pro.
I cook lid on.
I highly recommend Jamie Purviance's buttermilk and hot sauce brine with fresh herbs. It's from one of his Weber cookbooks, New American Barbecue, I think. It is amazing. It's so good that my wife requested it for Valentine's Day dinner.