About the AI Safety Institute 

The AI Safety Institute is the first state-backed organisation focused on advancing AI safety for the public interest. We launched at the Bletchley Park AI Safety Summit in 2023 because we believe taking responsible action on this extraordinary technology requires a capable and empowered group of technical experts within government.  

We have ambitious goals and need to move fast.  

1. Develop and conduct evaluations on advanced AI systems. We will characterise safety-relevant capabilities, understand the safety and security of systems, and assess their societal impacts. 
2. Develop novel tools for AI governance. We will create practical frameworks and novel methods to evaluate the safety and societal impacts of advanced AI systems, and anticipate how future technical safety research will feed into AI governance. 
3. Facilitate information exchange. We will establish clear information-sharing channels between the Institute and other national and international actors. These include stakeholders such as policymakers and international partners. 

Our staff includes senior alumni from OpenAI, Google DeepMind, start-ups and the UK government, and ML professors from leading universities. We are now calling on the worlds top technical talent to join us. This is a truly unique opportunity to help shape AI safety at an international level. 

As more powerful models are expected to hit the market over the course of 2024, AISIs mission to push for safe and responsible development and deployment of AI is more important than ever. 

 What we value: 

• Diverse Perspectives: We believe that a range of experiences and backgrounds is essential to our success. We welcome individuals from underrepresented groups to join us in this crucial mission. 
• Collaborative Spirit: We thrive on teamwork and open collaboration, valuing every contribution, big or small. 
• Innovation and Impact: We are dedicated to making a real-world difference in the field of frontier AI safety and capability, and we encourage innovative thinking and bold ideas. 
• Our Inclusive Environment: We are building an inclusive culture to make the Department a brilliant place to work where our people feel valued, have a voice and can be their authentic selves. We value difference and diversity, not only because we believe it is the right thing to do, but because it will help us be more innovative and make better decisions. 

As a Machine Learning Research Scientist working on safety cases, you will conduct foundational research to help take this ambitious new pillar of AISIs work forward. By building our understanding of how AI safety cases could be developed, you will help to expand AISIs programme of technical work beyond the existing workstreams focussed on evaluating model capabilities and safeguards. 

Safety cases are already used as standard in other industries and are structured arguments that a system is unlikely to cause significant harm if deployed in a particular setting. As the AI frontier develops, we expect safety cases could become an important tool for mitigating AI safety risks, whereby AI companies set out detailed arguments for how they have ensured their models are safe. We believe it is possible to significantly develop our understanding of what a good safety case would look like now, even though the field is far from knowing how to write a detailed safety case.

In this role, you would help push this understanding forward, both by direct technical research and via technical collaborations with external researchers and organizations. Key areas to cover include open problems that must be overcome to increase our confidence within specific safety agendas, and agenda-specific evaluations such as control and alignment evaluations. 

The role offers a unique opportunity to work closely alongside the worlds best technical talent, including Research Director Geoffrey Irving who leads the safety case workstream, as well as talented Policy / Strategy leads and other Research Engineers and Research Scientists.  You will also collaborate with external topic-level experts, partner organisations and policy makers to coordinate and build on external research. There will be significant scope to contribute to the overall vision and strategy of the safety case team as an early hire. 

We view work on safety cases at AISI as a critical component of the overall safety story, alongside our existing workstreams focused on evaluations of dangerous capabilities and safeguard effectiveness. 

RESPONSIBILITIES 

This role offers the opportunity to progress deep technical work at the frontier of AI safety and governance. Your work would likely include: 

• Direct research and writing of high-level safety case material (e.g. what methods might be used, and what properties one would need for these arguments to be correct).
• Strategy development and writing about how to get more safety case work to occur, what form they should take, and how to improve the chance that the results improve safety.
• Collaboration with external partners (e.g. labs, academics) on joint safety cases.
• Research organisational work (assembling teams, organising workshops) to create an environment where safety case work occurs.

We look for some of the following skills, experience and attitudes in a ML Research Scientist:  

• Relevant machine learning research experience in industry, relevant open-source collectives, or academia in a field related to machine learning, AI, AI security, or computer security.
• Broad knowledge of technical safety methods (T-shaped: some deep knowledge, lots of shallow knowledge).
• Strong writing ability.
• Motivated to conduct technical research with an emphasis on direct policy impact rather than exploring novel ideas.
• Ability to work autonomously and in a self-directed way with high agency, thriving in a constantly changing environment and a steadily growing team, while figuring out the best and most efficient ways to solve a particular problem.
• Bring your own voice and experience but also an eagerness to support your colleagues together with a willingness to do whatever is necessary for the teams success and find new ways of getting things done within government.
• Have a sense of mission, urgency, and responsibility for success, demonstrating problem-solving abilities and preparedness to acquire any missing knowledge necessary to get the job done.
• Comprehensive understanding of large language models (e.g. GPT-4). This includes both a broad understanding of the literature, as well as hands-on experience with things like pre-training or fine tuning LLMs. 

The following are also nice-to-have 

• Direct research experience (e.g. PhD in a technical field and/or spotlight papers at NeurIPS/ICML/ICLR).  
• Experience working with world-class multi-disciplinary teams, including both scientists and engineers (e.g. in a top-3 lab).  

We are interested in hiring individuals at a range of seniority and experience within this team, including in Senior ML Research Scientist positions. Calibration on final title, seniority and pay will take place as part of the recruitment process. We encourage all candidates who would be interested in joining to apply.  

CORE REQUIREMENTS  

• You should be able to spend at least 4 days per week working with us.
• You should be able to join us for at least 12 months.
• You should be able work from our office in London (Whitehall) for parts of the week, but we provide flexibility for remote work.

The Department for Science, Innovation and Technology offers a competitive mix of benefits including:

• A culture of flexible working, such as job sharing, homeworking and compressed hours.
• Automatic enrolment into the Civil Service Pension Scheme, with an average employer contribution of 27%.
• A minimum of 25 days of paid annual leave, increasing by 1 day per year up to a maximum of 30.
• An extensive range of learning & professional development opportunities, which all staff are actively encouraged to pursue.
• Access to a range of retail, travel and lifestyle employee discounts.
• The Department operates a discretionary hybrid working policy, which provides for a combination of working hours from your place of work and from your home in the UK. The current expectation for staff is to attend the office or non-home based location for 40-60% of the time over the accounting period.

Job contact :

• Name : [email protected]
• Email : [email protected]

Recruitment team

• Email : [email protected]