Here is an easy one-liner you can use when you have one or several conditions:
df['color'] = np.select(condlist=[df['Set']=="Z", df['Set']=="Y"], choicelist=["green", "yellow"], default="red")
Easy and good to go!
See more here: https://numpy.org/doc/stable/reference/generated/numpy.select.html