Depends on what converts you're playing. I think it's the inconsistency the turns most taiko players away, but I like the challenge in them. However, some of the 'challenge' comes from dumb things like 1/8 that's way too fast to hit with two fingers or unreadable SV changes.
In general, I think this is what people dislike about them the most
1. 1/8. As long as they're around 20 NPS or less, they're possible to hit (unless you play DDKK, then rip), but they're usually much faster than that. The 1/8 in Kokou no Sousei's converts are over 30 NPS, for example.
2. Slider velocity. There's a lot of songs where the slider velocity is way too high for any taiko player to possibly read without playing a lot of high SV converts (or maybe playing a ton of HR). Even if it's not fast, there are plenty of maps with constantly changing SV that make it impossible to read (see technonationalism). Sometimes it can be fun to try to learn it as a gimmick map or something similar, but most of the time it's annoying. Play enough converts and you get used to it, but normal taiko maps almost never do stuff like this, so it's a weird change to learn.
3. Weird/boring patterns. The patterns in converts aren't nearly as fun or interesting as normal taiko maps. They don't make much sense when they get tricky, either.
4. High OD. Some maps have it. Personally I don't mind, but I think it's gonna throw off a lot of taiko players, especially if there's any of the above problems in it. And if you're playing older maps... oh boy.
There's probably some other stuff, but that's all I can remember for now.
I still like playing them because they're unique and challenging in different ways from normal taiko maps, and they can be fun to mess around with. There's a lot more variety in them compared to taiko maps, too. But a lot of the time, they're dumb. Plus most maps don't have taiko maps, so it gives you a way to play the song.