Skip the muffler and build a box around your fan itself. I recently built a plywood box around my fan and then sprayed expandable foam into the box, excluding the electrical box, and it did wonders for silencing my fan. MUCH moreso than the muffler ever did. Also, I mounted hooks to the ceiling and hung the fan from bungee cords instead of the mounting bracket being screwed to the ceiling/wall. Can barely hear it at all now, even with the house dead quiet.
Another suggestion, get a speed controller, as I've never needed my 6" fan on full speed to keep the room below 78-80 degrees. 75% speed does well even in the middle of summer.