Living with PCOS, you should avoid caffeine for a number of reasons; including the fact that coffee increases your stress hormones which leads to an increase in your insulin levels. Another thing you should avoid with dairy, as it leads to unnecessary inflammation you don't need.
MY SOLUTION is an occasional cup of coffee with Oat milk! I mainly have herbal, oolong, and rooibos teas. When I'm extremely strict with my carb and calorie count, or when I want to fit other things into my daily intake all while following the 80/20 Rule, and I still want to have coffee I have it black ( not the best tasting, but it decreases how much of it I drink).