Two responses.
The first is that you have, by the very construction of the example, moved out of the primary domain of usefulness of "evil." The term is, to some extent, inherently an abstraction; by demanding that we never speak in anything but singular, discrete, concrete realities, of course one can argue that "evil" must be useless, because we're not allowed to speak about the places where it is most useful. "Evil," as a label, bundles together a lot of things, in much the same way that "good" does; e.g. if someone says "I'm trying to be a good person," though they are speaking abstractly, that abstraction is still useful, because it signals that this person has realized that some of their past actions are blame-worthy and that they are putting in the work to stop their blame-worthy actions and commit to virtuous ones instead. Likewise, someone saying, "What I did was evil," is not just admitting that they committed some past action, but rather that they have come to understand and accept that they deserve blame and feel guilt and remorse about that action. It is, most certainly, the case that "X person is evil" can only be determined after observing something like "X person coerces others into abusive romantic relationships," "X person tortures prisoners," or "X person kills innocent people for their personal enjoyment." But by that same token, things like "X object has a color resembling the ocean and the sky" or "X object emits a sound like an avalanche or a bomb" must be observed before we can call objects "blue" or "loud," yet these abstractions linked to physically-observable states of behaviors are perfectly acceptable. (And certainly colors are abstractions, otherwise languages wouldn't have so much disagreement about how many of them there are. Categorizing sounds by loudness is more concrete, but still abstracted across many distinct instances with little in common: 100 dB is abstract unless it is represented as some specific pressure wave, and could be a roaring furnace, a concert, or a trumpet right in your ear.)