Enriching data for Cyber Threat Intelligence is essential for understanding trends, assessing risk, and supporting strategic decision-making. Among the many enrichment tasks, determining the country and economic sector of ransomware victims plays a central role due to its role in contextualizing incidents and uncovering broader patterns within the threat landscape. However, such enrichment remains far from straightforward: victim’s descriptions are quite ambiguous, ransomware groups publish information inconsistently, and sector labels rarely follow a unified taxonomy. To address this challenge, at Byron Labs we developed an AI-based enrichment plugin, firmly grounded in a clear, rigorous methodology. Rather than jumping directly into model selection, we first built a validated ground truth: a carefully reviewed dataset that would enable us to test approaches objectively. Of course, without such a foundation, any performance number would be little more than guesswork, and the system would lack a solid baseline for meaningful evaluation.
With the ground truth established, we designed a testing framework that would benchmark various LLMs under the exact same conditions, as shown in Figure 1. Comparing models side by side meant that we did not rely on a single “preferred” model; instead, we let the data drive our choice. Indeed, different models showed meaningful variations in accuracy, stability, and consistency across victim profiles, reinforcing the importance of systematic evaluation rather than vendor loyalty and highlighting how methodological rigor directly impacts the reliability of the final output.
Figure 1: Testing Framework schema
Using this methodology, our system achieved accuracy above 80% for the cases that were evaluated. More importantly, by applying this methodology to our historical dataset, we have since been able to successfully enrich 65% of previously collected ransomware victims with a validated sector classification, utilizing the NAICS classification system, thus considerably enhancing the quality and depth of our intelligence. In a world where AI is evolving at an incredible pace, this project underscores one basic fact: its business value does not necessarily originate with the use of LLMs; it does so from the way they are used. Good methodology, validated data, and disciplined evaluation remain cornerstones for building systems that produce trustworthy results that foster an in-depth understanding of the ransomware ecosystem.
Written by Juan García from Byron Labs
