Back in college, I used JB weld to plug up a hole on a motorcycle engine that was leaking oil.
If I was being lazy & didn't want to deal with removing the old rivets, I'd probably put clamp or tie the bottom piece on & then add a bit of JB weld to the current rivet and glop it on a bit over the wood to basically try replacing the rivet head portion that holds it on. You might look through one of the other somewhat recent threads asking for tips on fixing a red grill that someone acquired, where
@Foster Dahlet mentions using some other high heat stuff on the outside of one of his bowls before.
But,
@michaelmilitello's suggestion is probably a better long term fix.