AI Security

Identifying and Exposing Failure Use Cases in AI Systems

Investigate how AI systems, including large language models (LLMs) and advanced multi-agent frameworks, fail under various conditions. Examine their susceptibility to input variations, adversarial attacks, or unexpected user manipulations. Explore whether input screening mechanisms in AI agent software effectively mitigate these vulnerabilities. Design experiments that stress-test these systems in diverse scenarios, revealing weak points that may compromise reliability or safety. Students will analyze patterns in failures and propose robust solutions to enhance system resilience.

Preventing Malicious Behavior in Future AI Systems

Envision scenarios where AI systems are connected to computer software, experimental devices, or autonomous processes with significant real-world impact. Evaluate how these systems might misuse their capabilities, such as creating harmful substances or bypassing safety constraints. Develop and test protocols that detect and prevent malicious actions, including frameworks for ethical safeguards, robust access control, and anomaly detection in AI behavior. Students will assess the balance between enabling AI autonomy and maintaining strict safety boundaries.

Designing Future-Proof Testing Protocols

Create comprehensive testing protocols that anticipate the misuse of AI systems in increasingly powerful and interconnected environments. Focus on ensuring that protocols address extreme yet plausible risks, including those involving experimental applications like drug development or autonomous decision-making. The protocols should incorporate ethical oversight, predictive failure analysis, and system-level audits to identify and mitigate potential dangers. Students will contribute innovative methodologies for securing the integrity of future AI systems.

Requirements

For students eager to shape the next wave of AI innovations, this project requires a deep foundation in at least one of the following areas: i) machine learning and AI, ii) programming, or iii) statistics. Participants will combine technical expertise with creativity to develop novel AI methods, positioning themselves as contributors to the future of transformative AI technologies.