Financial sentiment analysis requires understanding nuanced cues embedded in financial texts, especially numerical information. While large language models (LLMs) have shown strong performance in zero-shot tasks, it remains unclear whether they naturally attend to domain-specific indicators such as monetary values or percentages. In this study, we investigate whether expert-designed hints that highlight the importance of numbers can improve the zero-shot sentiment classification performance of LLMs. Using the Fin-SoMe dataset and evaluating four major LLMs (PaLM 2, Gemini Pro, GPT-3.5, and GPT-4), we compare standard zero-shot prompts, chain-of-thought (CoT) reasoning, and CoT with numerical focus hints. Our results show that models significantly benefit from expert-guided hints, especially on tweets containing numerical data or requiring deeper perspectivetaking. We further analyze performance across different number categories, revealing that monetary and temporal expressions benefit most from explicit prompting. These findings suggest that even advanced LLMs can benefit from targeted guidance in domain-specific tasks.

PDF