Data Engineer
- Public sector
- 15/07/2024
- £32452.00 /yearly
- Full Time & Permanent
Job expired!
Job Overview
Additional Detail
Job Description
ONS operates a flexible hybrid working model across the UK, with colleagues linked to one of our contractual locations in Newport, Titchfield (Fareham), London, Manchester and Edinburgh working between office and remote throughout the week.
As part of the hybrid working arrangement there is 40% a minimum office attendance requirement. Attendance is typically at your contractual office, with occasional travel to alternative locations. Due to estates constraints, there are currently temporary exceptions to this for colleagues based at Manchester who are required to attend the office for a minimum of 20% of their work time.
Job summary
The Office for National Statistics (ONS) is the UKs largest producer of official statistics, covering a range of key economic, social and demographic topics. These include measuring changes in the value of the UK economy, estimating the size, geographic distribution, and characteristics of the population, and providing indicators of price inflation, employment, earnings, crime, and migration.
Social Survey Transformation (SST) division is responsible for delivering the transformation of our social surveys as part of the wider transformation of the statistical system. Administrative data will be a key source of data for social statistics in the future; however, there will be a requirement for collection of supporting information and attitudinal data through surveys and SST is leading the transformation and delivery of these surveys. We are a team of over 80 people working across four Grade 6 areas. The division, working with our project management colleagues and teams across many directorates, are; developing a transformed version of the Labour Force Survey (LFS) using an online-first multi-mode collection approach; identifying and leading on all activities to help successfully decommission the LFS; providing survey expertise into the Future Population and Migration Statistics work.
Within SST the processing branch focusses on design, construction and use of processing systems to enable the creation of Reproducible Analytical Pipelines (RAPs) that enable the fast and effective provision of processed survey data to end users. This is all the way from raw survey returns to quality assured micro datasets for analytical use.
Within the branch the Technical team, which this role sits within, is responsible for the continued development of the Configurable Integrated Processing & Assurance System (CIPAS) for transformed social surveys. This will involve the use of modern open-source tools, such as Python, Pyspark, and potentially R using ONS strategic systems and potentially R, Flask, CSS, Javascript using ONS strategic systems.
Working collaboratively with other business areas both within the Surveys Directorate and elsewhere in ONS, all roles are aligned to the objective of producing high quality, social survey outputs that meet user needs, whilst remaining aligned with corporate priorities that include the recently established Surveys Strategy.
Job description
The Role:
- You will be involved in the continued development of the CIPAS for transformed social surveys. This will involve the use of modern open-source tools, such as Python, Pyspark, and R using ONS strategic systems and potentially Flask, CSS.
- The strategic systems currently include the Data Access Platform (DAP) but will over time expand to other developing strategic systems within cloud based environments such as GCP and AWS.
- The system is being developed to transform raw social survey data into clean, standardised, accurate datasets ready for analytical use. This is to be achieved via the creation of one modularised system, containing standardised methods, that apply rule based and statistical processes to the data.
- The system will be developed to be customisable by end users who have little to no coding knowledge and to suit each individual pipeline and input dataset. The result will be one system with the ability to create multiple statistical pipelines thereby reducing risk, improving auditability and making processing reproducible.
- Your primary focus will be to support the Data Processing Integration Lead with the ongoing technical development of CIPAS. With an initial focus on processing TLFS data but expanding to meet requirements for other social surveys in due course.
- You will work within a small coding team along with collaborators from other teams, including teams working on shorter term tactical solutions, within the division and across ONS to meet development and operational needs.
- You will be required to work in an Agile manner aided via the use of JIRA and Confluence.
Responsibilities
- Implement data flows to connect operational systems, data for analytics and business intelligence (BI) systems.
- Re-engineer manual data flows to enable scaling and repeatable use.
- Write ETL scripts and code to make sure the ETL process performs optimally.
- Write PySpark, Python or R code that will systemize the processing of data, based on initial design documents. This includes writing unit / integration tests and managing versions of code with Git.
- Support the build of pipelines by the operational team that can be easily maintained, adapted and re-run.
- Maintain and review documentation for statistical and data processing functions, as well as data processing pipelines.
- Report on the progress of your work to your LM and peers.
Person specification
Essential Skills Criteria:
- You can design, build and test data products based on feeds from multiple systems using a range of different storage technologies and/or access methods. You know how to create repeatable and reusable products.
- You can deliver data solutions in accordance with agreed organisational standards that ensure services are resilient, scalable and future-proof.
- You can design, code, test, correct and document simple programs or scripts under the direction of others.
- You understand core technical concepts related to the role and can apply them with guidance.
- You can execute test scripts under supervision. You understand the role of testing and how it works.
- You know how to log, analyse, manage problems and debug code in order to identify and implement the appropriate solution, ensuring that the problem is fixed.
Technical Skills:
- Proficient at working with data in PySpark, Python and SQL.
- Beneficial if you have experience with R, Flask, CSS or Javascript.
- Experience working to build data pipelines (analytical or otherwise).
Please refer to the Government Digital and Data capability framework for more details on each criteria.
Behaviours
We'll assess you against these behaviours during the selection process:
- Making Effective Decisions
- Working Together
- Communicating and Influencing
- Delivering at Pace
Technical skills
We'll assess you against these technical skills during the selection process:
- Technical skills assessed at interview, aligned to the person specification essential technical skills criteria of the role - Programming and Build.
Alongside your salary of £32,452, Office for National Statistics contributes £8,762 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.
The Office for National Statistics is part of the Civil Service, and as such we share a number of key benefits with other departments, whilst also having our own unique offerings to support our 5400 valued colleagues across the business.
Whether you are hearing about us for the first time or already know a bit about our organisation, we hope that the benefits pack attached (bottom of page) will give you a great insight into the benefits and facilities available to our colleagues, and our fantastic working culture.
This role is part of the cross-government Government Digital and Data (formerly DDaT) profession framework. As a role within Government Digital and Data (formerly DDaT) at the ONS, we also offer benefits such as:
Protected Learning Time to spend on your personal development and side-projects.
A supportive and active Community of Practice which you will be expected to contribute to, helping ensure you and your colleagues get the training, development and opportunities you need to progress your careers.
We are committed to supporting our peoples wellbeing by offering flexible ways of working that support a healthy work life balance. We are happy to explore opportunities with you about working flexibly in line with our hybrid working policies.
Inclusion & Accessibility
At ONS we are always looking to attract the very best people from the widest possible talent pool, and we are proud to be an inclusive, equal opportunities employer. As a member of the Business Disability Forum and a Disability Confident Leader were committed to ensuring that all candidates are treated fairly throughout the recruitment process.
As part of our application process, you will be prompted to provide details of any reasonable adjustments to our recruitment process that you need. If you would like to discuss any reasonable adjustments before applying, please contact the recruitment team in the first instance.
If you would like an accessible version of any of the attachments or recruitment documents below or linked to in this advert, please contact the recruitment team who will be happy to assist.
Selection process details
This vacancy is using Success Profiles (opens in a new window), and will assess your Behaviours, Experience and Technical skills.
Security Clearance
The requirement for SC clearance is to have been present in the UK for 3 consecutive years immediately prior to applying.
A lack of UK residency in itself is not always a bar to security clearance but the Department will need to consider eligibility by exception on a case-by-case basis. You will be asked to provide information regarding your UK residency during your application, and failure to provide this will result in your application being rejected.
If you are unsure that you meet the eligibility above, please read the information available on Gov.uk on this link, or contact the recruitment email on the advert before applying to discuss, as failure to meet the residency requirements will result in your security clearance application being rejected and any offer of employment being withdrawn.
At the point of SC application you will need to provide or give access to the following evidence:
- Departmental or company records (personnel files, staff reports, sick leave reports and security records)
- UK criminal records covering both spent and unspent criminal records
- Your credit and financial history with a credit reference agency
- Security Services records.
Applicant support
Applicants may utilise AI resources in their submissions; ensuring all information provided is factually accurate, truthful, and original, avoiding any form of plagiarism to maintain the authenticity and credibility of the application process.
Stage 1 Application & Sift
Assessment at application stage will be based on your CV, work history, skills, experience and personal statement. Your application should be tailored to demonstrate any skills, knowledge and experience that are relevant to the content of the role.
Your personal statement should be no longer than 1250 words. You should use this space to provide evidence for each essential and technical criteria within the person specification. As the criteria are scored, we would recommend that you give clear examples for each including the impact of your actions and set them out using the STAR acronym (Situation, Task, Action, Result).
Success Profiles Behaviour examples are not required at this stage.
In cases where there is a high number of applications the sift pass mark may be adjusted and candidates will be invited to interview in merit order, i.e. those scoring the highest.
Stage 2 Interview
Should you be invited to interview, you will be assessed using various assessment techniques aligned to the Civil Service Success Profiles, where you'll be assessed against all the behaviours and technical skills outlined in the advert. Further details will be shared with you should you be invited to interview.
Interviews will be conducted by Video.
A reserve list may be held for a period up to 12 months from which further appointments may be made.
Important Dates:
Sift date: from 31/7/2024
Interview dates from: w/c 12/8/2024
Dates are subject to change
For the full terms and conditions of the post, please see attachment below.
Feedback will only be provided if you attend an interview or assessment.
Security
Successful candidates must undergo a criminal record check.
Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window).
See our vetting charter (opens in a new window).
See our vetting charter (opens in a new window).
People working with government assets must complete baseline personnel security standard (opens in new window) checks.
Nationality requirements
Working for the Civil Service
The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.
We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).
We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window).
The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.
The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.
Diversity and Inclusion
The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window).
This vacancy is part of the Great Place to Work for Veterans (opens in a new window) initiative.
Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.
Contact point for applicants
Job contact :
- Name : Government Digital and Data Recruitment Team
- Email : [email protected]
Recruitment team
- Email : [email protected]
Further information
If you feel your application has not been treated in accordance with the Recruitment Principles and you wish to make a complaint, in the first instance, you should contact [email protected]. If you are not satisfied with the response you receive from the Department, you can contact the Civil Service Commission
