https://ashwinviswamithiran.com/tags/evaluation/Recent content in Evaluation on Ashwin Viswamithiranhttps://ashwinviswamithiran.com/og-default.pnghttps://ashwinviswamithiran.com/og-default.pngHugoen-usTue, 29 Sep 2026 00:00:00 +0000https://ashwinviswamithiran.com/writing/attacks-that-worked/Mon, 28 Sep 2026 00:00:00 +0000https://ashwinviswamithiran.com/writing/attacks-that-worked/A technique survey of 18,479 successful prompt attacks where the correct MITRE ATLAS label is known in advance from how the competition was scored. Keyword attribution recovers 11.4% of it, and most of the remaining breakdown describes the challenges rather than the attackers.https://ashwinviswamithiran.com/writing/promptlsh-evaluation/Tue, 08 Sep 2026 00:00:00 +0000https://ashwinviswamithiran.com/writing/promptlsh-evaluation/Two organisations cannot compare prompt attacks by sharing the prompts: those are things users wrote. A compact derived fingerprint solves that, and this measures what each size actually buys — full float, 384-byte quantised, 32-byte digest, dependency-free lexical — on recall, on evasion resistance, and on what survives across independently collected feeds.https://ashwinviswamithiran.com/writing/twelve-ways-to-make-a-threat-intel-bot-lie/Thu, 11 Jun 2026 00:00:00 +0000https://ashwinviswamithiran.com/writing/twelve-ways-to-make-a-threat-intel-bot-lie/An evaluation of a retrieval-grounded threat-intelligence assistant against twelve adversarial prompts designed to induce fabrication, a twenty-four query quality set, and a three-model comparison — including what the tests failed to cover.