Yes, it is possible, to do so, follow those steps:
1. You can install osu!(lazer) first normally, and then open it, go to the Options.
2. Then click the "Change folder location" button

3. ...and direct it to your USB directory where you want the osu!(lazer) location to be.
4. Next, go to "C:\Users\(your pc name)\AppData\Local\osulazer" and cut the whole game folder to your USB (doesn't have to be under the same folder/path with the previous osu!(lazer) data folder).
You should be able to open osu!(lazer) directly from your USB now!
==============================
though, if you plug it into another pc, it may act like it's a fresh install, in that case, click below: