This repo is now archived.
Please view our handbook for news on our latest recruitment drive.
Information for potential applicants for Data Engineering and Analytics Engineers roles within the Data and Analysis division of the Ministry of Justice (MoJ). Here you will find additional information about what work we do.
Last updated: 20th February 2024 by oliver-critchfield
We have the current roles open at the moment:
- Analytics Engineer Apprenticeships
- Lead Data Engineer (Grade 7) on EOI
- Senior Data Engineer (SEO) on EOI
- Senior Analytics Engineer (SEO) on EOI
We are running a webinar event for the apprenticeship programme on Monday 4th March 2024 at 18:00 at which we'll share information on
- The context of Data and Analytics Engineering within the MoJ, our philosopy of data provision and the analytical teams we support
- A deep dive into Analytics Engineering in our team
- The curriculum and delivery and learning processes for the apprenticeship, delivered by our apprenticeship partner, QA
- Applying for positions in the Civil Service, with hints and tips for a successful application This will also be an opportunity to ask questions about the role. You can sign up for this here
We also ran a webinar event on Tuesday 25th April 2023 at 18:00 where we covered lots of information and taking questions 👉 you can watch the recording here
See below 👇 for more information on these roles and a few generic questions we have answered.
Welcome 👋
Data engineers and analytics engineers within the Ministry of Justice’s Data and Analysis directorate work across two teams, one which delivers solutions to internal stakeholder needs across the organisation, the Data Modelling and Engineering Team (DMET), and another that works on a cutting-edge data platform, the Data Platform Team. We have a section below on what the difference is between the two teams.
As a data engineer or analytics engineer in our teams you will get the opportunity to continue to build your coding skills, as well as develop new and exciting cloud computing capabilities using Amazon Web Service (AWS). We don’t just focus on developing technical skills though. You will also gain experience working directly with stakeholders to deliver important solutions to key Ministry of Justice needs. You won’t be alone either. As well as being part of a growing team within DMET or Data Platform, you will also be part of a large data and analytical community at the Ministry of Justice and use your expertise to shape the future of how data is used in decision making.
At all levels, our data engineers design and develop robust data pipelines so that data can be transformed into a form that data scientists and analysts can use. We are also responsible for managing access to data on our Analytical Platform. Data engineers will also be expected to directly work alongside key departments within the Ministry of Justice, and be able to explain technical solutions in non-technical ways.
At all levels, our analytics engineers work with end users of data to understand their requirements, as well as with data suppliers to understand the data sources. They capture business processes, and design conceptual models to provide comprehensive, intuitive, and resilient content to underpin all relevant analytical use cases. They bring an enterprise-wide perspective by working with colleagues to cross-reference requirements across domains, before translating conceptual designs into computer code to deliver facts, dimensions, derived tables and business metadata to end users. They work to ensure data quality is measured and presented to users, and to inform improvement by communicating findings to data owners. They promote content to analytical user communities and gather feedback to shape further development and integration.
The Data Modelling and Engineering team works across several areas of the Ministry of Justice to define, design, and deliver Data Engineering and Data Modelling as a service. These areas include Prison and Probation services, Courts and Tribunal services, the Office for Public Guardian, and MoJ Corporate services. We also have cross-cutting areas such as Data First and BOLD which are Government data-linking programmes we are involved with. Our Data Engineers and Analytics Engineers are supported by a Service Management approach which helps technical leads engage and work with stakeholders.
The Data Platform team is responsible for running our existing Analytical Platform, which is used by over 500 analysts and data scientists. We are also at the beginning of our journey to develop a new Data Platform for all of MoJ, built on the principles of Data Mesh, to ensure that everyone, at all levels of the department, has the right data at the right time to make effective decisions that improve justice outcomes.
Data engineers work within multidisciplinary teams alongside technical, product, delivery and user-centred design colleauges to build and maintain platform components that are used by data producers, data users and data consumers to share, find, access, process and manage data.
For example, we are responsible for:
- Airflow, which enables people to automate complex data and analytical workflows
- dbt, which enables people to quickly and easily transform and model raw data to make it more useful and useable
- various tools that enable data producers to send us their data in a self-service way, and build standardised pipeline infrastructure
Our work is incredibly varied but has a strong infrastructure focus and involves elements of software development as well as data engineering.
The Analytical Platform (AP) provides access to recent versions of open–source analytical software, such as RStudio and JupyterLab, allowing analysts to work in the way that suits them best. Out Data Engineers may be responsible for developing methods of transferring data onto, within and off the Analytical Platform as well as transforming data into a usuable product. Our Data Modellers will understand the data needs of the customer and develop methods to deliver data tables. Find out more information about the Analytical Platform here
The team has developed a number of Python and R tools, and Web Apps, for users on the Analytical Platform. This includes pydbtools, which is a package for querying MoJAP athena databases with useful features including temp table creation, and Create a Derived Table, a tool for creating persistent derived tables in Athena.
We continue to develop more tools to make using the Analytical Platform easier for our users, and information on some of these is available in the Analytical Platform User Guidance.
splink is a Python package for probabilistic record linkage (entity resolution). Under its hood it is a PySpark package that implements the Fellegi-Sunter model of record linking, and enables parameters to be estimated using the Expectation Maximisation algorithm. It is used across government for data linking and won several awards over the years. Splink was developed and is maintained by a Data Linking team which sits within the same directorate as and works closely with our Data Engineering teams.
The roles are national. We have a number of offices around the England and Wales. See this link to find your closest one 👉 here
We're open to flexible and hybrid working arrangements that include regular home working, but you'll need to go to one of these offices from time to time.
We are really keen to break away from the archetypal views of a Data Engineer and Analytics Engineer. We really encourage anyone with the relevant skills and willingness to learn to apply. We will support you to learn skills you may not yet had the opportunity to learn, but show an eagerness to.
We’ve setup a mailbox at 👉 [email protected] for any questions.
And of course, there’s always the webinar too 👉 sign up here