Doesn't that probably happen every year though? It stands to reason that Cy Young candidates are pitching for playoff teams and are going to make multiple starts in any playoff series.
Not really... Looking at ALCS and NLCS teams the last few years. If a Cy Young finalist played for the team in question, I went down 1 spot in the voting.
2017 Dodgers: 0 out of 15 games (although we don't know who is 4th in NL Cy Young voting, so not sure who to replace Kershaw with. If its Greinke, then 1 out of 15 games)
2017 Cubs: 5 out of 10 games (Strasburg x2, Scherzer, Kershaw x2)
2017 Astros: 5 out of 18 games (Sale, Severino x2, Kershaw x2)
2017 Yankees: 2 out of 12 games (Kluber x2 - Same as Dodgers, don't know who finished 4th)
2016 Dodgers: 6 out of 11 games (Scherzer x2, Lester x2, Hendricks x2)
2016 Cubs: 6 out of 17 games (Bumgarner, Kershaw x2, Kluber x3)
2016 Blue Jays: 2 out of 9 games (Kluber x2 - Kluber is the only pitcher in the Top 10 of voting that the Jays faced that year)
2016 Indians: 5 out of 15 games (Porcello, Lester x2, Hendricks x2)
2015 Mets: 5 out of 14 games (Kershaw x2, Greinke x2, Arrieta)
2015 Cubs: 1 out of 9 games (Cole)
2015 Royals: 3 out of 16 games (Kuechel, Price x2)
2015 Jays: 0 out of 11 games
2014 Giants: 2 out of 16 games (Wainwright x2)
2014 Cardinals: 4 out of 9 games (Kershaw x2, Bumgarner x2)
2014 Royals: 0 out of 15 games
2014 Orioles: 0 out of 7 games
So on average, a team faced a starter who is a Cy Young finalist 46 times in 204 games (23% of the time). The highest percentages in a run were the 2016 Dodgers (55%), 2017 Cubs (50%), , 2014 Cardinals (44%), 2015 Mets (36%) and 2016 Cubs (35%).
Of course its kind of limiting to draw the line at Top 3 Cy Young finishers as there are plenty of extremely good pitchers who didn't finish top 3 for whatever reason. BTW, in the games the Cubs didn't face Kershaw, Strasburg and Scherzer, they faced Alex Wood (2.72 ERA), Gio Gonzalez (2.96 ERA) x2, Rich Hill (3.32 ERA), Yu Darvish (3.44 ERA).
Edit: Correction, for some reason I missed 1 Hendricks start against the 2016 Dodgers. Dodgers faced one 55% of the time, most of any team over the last 4 postseasons.