One of the central challenges in identifying the causal effects of monetary policy is the inherent endogeneity of its conduct. This paper introduces a novel identification strategy that leverages LLMs to detect monetary policy shocks from newspaper coverage following European Central Bank (ECB) policy decisions. Based on a dataset of 7,620 articles from eleven major European newspapers, we classify each policy decision as unexpectedly restrictive, unexpectedly expansionary, or as expected. The resulting narrative-based surprise series captures immediate post-announcement perceptions and shows a close alignment with established High Frequency Identification (HFI) measures with notable exceptions during times of financial turmoil. We subsequently analyze the potential influence of the information effect on our series and find that the majority of identified surprises are unlikely to be driven by information effects.