While I have yet to openly bawl while watching anime, these are the moments that left me in a sad state for a few days (in no particular order).
Cowboy Bebop, Ep. 24: After watching the Bebop crew grow closer through the series, seeing them split was tough and the music was just perfect for this moment.
Kotoura-san, Ep. 1: The first half of this episode is my new standard on grading how a show affects me.
Code Geass series: Lelouch erasing Shirley's memories, or Shirley's death in R2, or the ending of R2 (again, the music for this moment really made it memorable).
FMA: Hughes death, or Nina's transformation and death.