About the AI Safety Institute 

The AI Safety Institute is the first state-backed organisation focused on advancing AI safety for the public interest. We launched at the Bletchley Park AI Safety Summit in 2023 because we believe taking responsible action on this extraordinary technology requires a capable and empowered group of technical experts within government.  

We have ambitious goals and need to move fast.  

1. Develop and conduct evaluations on advanced AI systems. We will characterise safety-relevant capabilities, understand the safety and security of systems, and assess their societal impacts. 
2. Develop novel tools for AI governance. We will create practical frameworks and novel methods to evaluate the safety and societal impacts of advanced AI systems, and anticipate how future technical safety research will feed into AI governance. 
3. Facilitate information exchange. We will establish clear information-sharing channels between the Institute and other national and international actors. These include stakeholders such as policymakers and international partners. 

Our staff includes senior alumni from OpenAI, Google DeepMind, start-ups and the UK government, and ML professors from leading universities. We are now calling on the worlds top technical talent to join us. This is a truly unique opportunity to help shape AI safety at an international level. 

As more powerful models are expected to hit the market over the course of 2024, AISIs mission to push for safe and responsible development and deployment of AI is more important than ever. 

 What we value: 

• Diverse Perspectives: We believe that a range of experiences and backgrounds is essential to our success. We welcome individuals from underrepresented groups to join us in this crucial mission. 
• Collaborative Spirit: We thrive on teamwork and open collaboration, valuing every contribution, big or small. 
• Innovation and Impact: We are dedicated to making a real-world difference in the field of frontier AI safety and capability, and we encourage innovative thinking and bold ideas. 
• Our Inclusive Environment: We are building an inclusive culture to make the Department a brilliant place to work where our people feel valued, have a voice and can be their authentic selves. We value difference and diversity, not only because we believe it is the right thing to do, but because it will help us be more innovative and make better decisions. 

The mission of the Autonomous Systems team is to prevent catastrophic risks from autonomous AI. 

The way we do this is by studying the space of potential risks from autonomous systems. We then build tools that measure and forecast this risk by interacting with frontier models. For example, we could be investigating the various ways an autonomous AI could prevent shutdown by exfiltrating its own weights and replicating itself on other hardware. We can then build tools to measure this risk as frontier models keep improving and conduct research into when exactly we believe the risk will present a material danger. Finally, we interact with other teams within the institute to make sure our research has real-world impacts on AI safety through interaction with the key labs and policy recommendations. 

The Autonomous Systems Team is looking for a senior prompt engineer, to help all research teams within AISI elicit the most dangerous capabilities from frontier models. This involves increasing the extent to which models directly reveal dangerous information, as well as improving the performance of our agent scaffolds.  

The role will require cross-functional collaboration, as youll be working closely with researchers and domain experts in all our workstreams (Chem/Bio, Cyber Security, Societal Impacts, Safeguards and Autonomous Systems). Youll receive mentorship and coaching from your manager and will regularly interact with world-famous researchers and other incredible staff. 

We are looking for some of the following skills, experience and attitudes. 

• Ability to modify behaviour of LLMs or induce target behaviour from a LLM via careful prompt design. 

• Good understanding of large language models and their architectures, including a comprehensive grasp of how transformer models like GPT work under the hood. 

• Mastery of prompting techniques like few-shot learning, prompt tuning, chain-of-thought prompting, automatic prompt generation, ReAct loop etc. 

• Expertise in evaluating and probing LLMs via metrics like BLEU, ROUGE, and/or evaluation frameworks for qualities like reasoning or coding ability. 

• Experience building practical applications that leverage LLMs for tasks like question-answering, text generation, coding assistance, etc. 

• Ability to conduct analysis of model performance via statistics and data visualization. 

• Good coding skills and familiarity with Python. 

• Strong written and verbal communication skills. 

Nice-to-have 

• Experience working with a world-class research team comprised of both scientists and engineers (e.g. in a top-tier frontier AI lab).
• Extensive Python experience, including understanding the intricacies of the language, the good vs. bad Pythonic ways of doing things and much of the wider ecosystem/tooling.
• Direct research experience (e.g. PhD in a technical field and/or spotlight papers at NeurIPS/ICML/ICLR).
• Deep theoretical knowledge of linguistics. 

The Department for Science, Innovation and Technology offers a competitive mix of benefits including:

• A culture of flexible working, such as job sharing, homeworking and compressed hours.
• Automatic enrolment into the Civil Service Pension Scheme, with an average employer contribution of 27%.
• A minimum of 25 days of paid annual leave, increasing by 1 day per year up to a maximum of 30.
• An extensive range of learning & professional development opportunities, which all staff are actively encouraged to pursue.
• Access to a range of retail, travel and lifestyle employee discounts.
• The Department operates a discretionary hybrid working policy, which provides for a combination of working hours from your place of work and from your home in the UK. The current expectation for staff is to attend the office or non-home based location for 40-60% of the time over the accounting period.

Job contact :

• Name : [email protected]
• Email : [email protected]

Recruitment team

• Email : [email protected]