It's different based on what the series in general is trying to achieve, but I always say it's the same thing that makes something like, say, a standard film or miniseries actually good. I basically think my own guidelines are similar to Aristotle's The Poetics.
Writing/plot are of the utmost significance. If you have dialogue that is filled to the brim with exposition or characters that don't talk like how a real person would (and it is not treated with such oddity in-universe) or a plot that is brimming with holes and does not make sense when you fill everything in (or overly relies on contrivance), then it's going to fall apart.
Characters are just as important, but yet again, the writing/plot and characters/dialogue are on the same level depending on what the show is trying to do. You need flawed characters who are still sympathetic, nobody being overly powerful or contrived in order for the viewer to actually give a damn about them. A good character has a solid, developed, believable arc, fulfilling their character motivations.
Next is the actual animation, which is obviously extremely important given the medium of animation (duh). I don't JUST mean the animation quality here, either, but the cinematography and artistry and character designs, as well. These all combine to form the visual that the viewer sees, and if they fall flat, then they can harm the experience. This is easily the most overtly superficial element of anime and a lot of people can't see beyond the animation for what a show truly is (Aku no Hana, anything Masaaki Yuasa touches, etc.) and some even call a show perfect despite only being solid in its superficial elements (Project K, K-ON!, Guilty Crown, most Makoto Shinkai productions). The truth of the matter is: the animation does set the stage for what you are watching, and if a show doesn't look good it can hamper the experience. However, if you make your visual aesthetic the only driving factor in making your show intriguing, you're doing it wrong.
The lower end of the totem pole, despite it being my major interest in life, is reserved for music and sound. Good, convincing acting is absolutely necessary and should be considered part of the character section (or just as important as the animation), but unfitting sound effects and bad acting can really remove a show from someone's mind. A good score is the very lowest factor, being something that can definitely set the mood for a scene just as well (or in some shows' cases, better than) the scene itself. A good score is also very listenable outside of its intended context.
tl;dr: Writing/plot > Characters > Animation = Voice Acting > Sound Effects = Music. Focusing on the superficial elements of something may help in terms of making anime a commodity but fucking destroys it as an art form.
]
]
