I have read many helpful, but conflicting opinions over at HVACTalk.com so I'll relate mostly what I read in my sound construction book.
There are 2 things you can do to damp out sound transmission in an existing rectangular duct run. First, cut a piece out and replace it with a flexible coupling. A short section of a few inches or so is what is required to isolate mechanical noise/vibration. This should occur immediately after exiting the noisy space.
Second is linacoustc or similar product inside. Acoustically lined ducts are available for new construction, but I assume your post relates to a retrofit. If not possible to line the duct on all sides inside, insert a long piece from corner to corner diagonally with the ends cut tapered to improve airflow. Make sure it is material backed on both sides and put foil tape on leading and trailing edges to contain fibres.
This creates 2 smaller parallel ducts. As far as sound goes smaller is much better. Each of these "new" small ducts will share the acoustic damping material wall. Airflow will not be affected really.
I would not look at Dynamat or NoNoise products. The acoustic sound energy is most harmful inside the duct and enters/emits from other ports in the system. Assuming you are framing out and drywalling it in, I would just opt to line it on the outside with fiberglass insulation if anything.
If you had the option to bring a dedicated return into the room from the main trunk using flexduct, the sound would have to travel all the way around the "circuit" to affect other areas. Flexduct does not transmit sound like metal ducting. Best option is NOT to cut a takeoff into the return duct for a register right in the room.
