Smith's most obvious use as a kick returner isn't even really what would trouble me. Antonio Cromartie looked more than up to the task last week. Mike Westhoff has made a living with his blocking scheme of replacing talented return men well.
I wouldn't like losing Smith in the Wildcat package. I know it has been hit or miss lately, but it has been a tool for the Jets to pick up big gains and to diversify the run game. Perhaps even more, I am worried about losing Brad on coverage units. The Pats have some very good return men, Brandon Tate and Julian Edelman. Whenever there appears to be a breakdown, it always feels like Smith is the guy flying in to fill a hole and save a huge gain.
Losing Coleman would obviously be a hit to secondary depth. I know some have been critical of his play at the nickel, but he is the best guy the Jets have for that spot. He's also an excellent blitzer. Since the Pats like to spread the field and are most dangerous throwing the ball, the defensive backs are going to see a lot of time and play big roles in pressure packages.
It is also a lot to put on Kyle Wilson. He did look great in Week 17 so we will see how ready he is to take on a bigger role. Using that game to get backups reps could prove valuable.
On top of all of this, though, it takes options off the table in terms of mixing coverages. They activated 10 defensive backs against the Colts. It could be similar against Tom Brady. Losing Coleman takes away an option. For example, if Wilson has to move to play the slot receiver without Drew, the Jets will not be able to move him inside to play tight ends like he did with success in Week 2.