How do AI agents behave when given a goal and what happens when they find unexpected ways to achieve it?
INSAIT’s Director of AI Applications, Yasen Kiprov, explored these questions in an interview on NOVA News, discussing AI safety and the challenges of understanding increasingly capable AI systems.
Drawing on the attack involving OpenAI agents and Hugging Face, Yasen explained how agents tasked with testing software security found an unexpected way to communicate and collaborate. The experiment highlights why it is essential to develop reliable safeguards.
AI models are also becoming increasingly capable of identifying vulnerabilities in software and critical infrastructure. This creates opportunities to strengthen cybersecurity, but also raises concerns about how these capabilities could be misused.
Addressing these challenges is an important part of INSAIT’s work. Our researchers work to uncover vulnerabilities in AI models and improve their resilience, while also helping advance regulatory compliance to strengthen the overall reliability and safety of new models. As AI advances, this work is essential to ensuring that increasingly capable AI systems are developed responsibly.
Watch the full interview in Bulgarian here.