If you're finding yourself tapping faster with one finger, and the aforementioned finger-switch doesn't seem to help, I would suggest you try slowing it down (or rather, dropping to easier/slower burst maps) and practice how you play streams. Intentionally force yourself to play them at a steady rhythm so you can spot when the mis-match starts and thus when you can correct it.
Stream practice maps aren't inherently bad for this, but I think learning how to do this alongside how to transition to other patterns/objects is also important. Plus, if you do switch to middle/ring anyways you'll have to learn that part regardless, so.