Mid/Senior Data Engineer
About this role
You will support complex client engagements where data engineering is used to stabilise business-critical processes, improve reporting confidence and establish repeatable data foundations across enterprise systems. Bringing strong hands-on experience in data profiling, cleansing, mapping, reconciliation and integration across complex business systems, ideally with exposure to procurement, workforce, finance, ERP or source-to-pay data. You should be comfortable working iteratively with architects, process owners and business stakeholders to identify root causes, support tactical fixes, improve reporting confidence, and help establish repeatable data foundations for future transformation. Working on client data foundation engagements that combine discovery, stabilisation and remediation. Typical work will include understanding process and system landscapes, identifying data and reconciliation issues, supporting tactical fixes, and helping clients define the data architecture, reporting and governance foundations needed for longer-term transformation.What You'll Be Doing as a Data Engineer:Design, build and improve ETL and ELT pipelines that support data ingestion, profiling, reconciliation, cleansing and reporting across enterprise source systems.Building data catalogues, data flows, interface views and trusted source views Design and architect modern data solutions that align with business objectives and technical requirements, supporting current-state and target-state data architectureHelp clients improve confidence in operational, workforce, procurement and financial reporting through timely, accurate and reconcilable data.Build highly scalable and performant data solutions leveraging cloud platforms and open-source softwareDevelop data models to handle enterprise-level analytical needsOptimise large-scale data processing systems for performance and cost-efficiencyImplement robust data quality frameworks and monitoring solutionsEvaluate new technologies to enhance our data engineering capabilitiesCollaborate with stakeholders to translate business requirements into technical specificationsPresent technical solutions to leadership and non-technical stakeholdersContribute to the development of the Methods Analytics Engineering Practice by participating in our internal community of practiceYour Impact:Enable business leaders to make informed decisions with confidence through timely, accurate data insightsEstablish reusable engineering standards, patterns and documentation that support quality, maintainability and repeatable Data Foundations delivery across future engagements.Drive adoption of modern data architectures and platformsDeliver seamless data solutions that enhance user experienceElevate the technical capabilities of the entire data engineering teamHelp cultivate a data-driven culture within the organisationEstablish technical standards and patterns that ensure quality and maintainability You Will Demonstrate:Experience working with data from Ariba, Workday, SAP S/4HANA or comparable procurement, workforce, timesheet, finance, supplier invoice or locally maintained spreadsheet sources.Hands-on experience profiling data quality issues, defining cleansing rules, mapping data between systems, validating reconciliation outputs and documenting exceptions for business review.Ability to work iteratively with architects, process owners, finance, procurement, workforce and operational stakeholders to turn ambiguous business issues into clear data analysis, engineering actions and controlled tactical fixes.Understanding of data ownership, stewardship, lineage, metadata, controls and data quality monitoring, with the ability to produce documentation that can be reused as part of an enduring data governance model.Experience implementing and advocating for test-driven development methodologies in data pipeline workflows, including unit testing, integration testing, and data quality validation frameworksProven experience leading technical aspects of data projectsStrong data architecture and modelling skills with the ability to design scalable data solutionsDeep understanding of data warehouse design principles and methodologiesAdvanced knowledge of optimisation techniques for large-scale data processingStrong proficiency in SQL and Python for handling complex data problemsHands-on experience with Apache Spark (PySpark or Spark SQL)Experience with the Azure data stackKnowledge of workflow orchestration tools like Azure Data Factory or Apache AirflowExperience with containerisation technologies like DockerProficiency in dimensional modelling techniquesExperience with CI/CD pipelines for data solutionsStrong communication skills for translating complex technical concepts You may also have some of the desirable skills and experience:Experience designing and implementing data mesh or data fabric architecturesKnowledge of cost optimisation strategies for cloud data platformsExperience with data quality frameworks and implementationExperience with data visualisation tools like Power BI or Apache SupersetExperience with other cloud data platforms like AWS, GCP or OracleExperience with modern unified data platforms like Databricks or Microsoft FabricExperience with Kubernetes for container orchestrationUnderstanding of streaming technologies (Apache Kafka, event-based architectures)Experience with high-performance, large-scale data systems Security Clearance:UKSV (United Kingdom Security Vetting) clearance is required for this role, with Security Check (SC) as the minimum standard, either already held or with a willingness to undergo the process. Some roles/projects may require Developed Vetting (DV) clearance; while not mandatory, a willingness to obtain DV clearance would be beneficial. As part of the onboarding process candidates will be asked to complete a Baseline Personnel Security Standard (BPSS); details of the evidence required to apply may be found on the government website GOV.UK – Government baseline personnel security standard. If you are unable to meet this and any associated criteria, then your employment may be delayed, or rejected. Details of this will be discussed with you at interview.
What this role is, and what else it is called
Employers in London advertise this kind of work as Data Analyst, Senior Data Engineer, Data Engineer and Data Scientist too, so it is worth searching more than one wording. It is a senior Data, AI & Machine Learning role in London, advertised as fully remote, so it is open to candidates working from home.
About hiring at Methodsdigital
Methodsdigital has 4 other live technology roles on its careers page, across Data, AI & Machine Learning, IT Support & Service Desk and Networks & Telecommunications. Of those, 40% are advertised as fully remote and 40% as hybrid. Elsewhere in its adverts Methodsdigital asks for Microsoft 365, Scrum, Linux, PostgreSQL and Microsoft SQL Server. See all Methodsdigital roles, salaries and stack.
How this role compares to the market
1,965 live UK roles list Python right now, from 581 employers. The median advertised salary is £80,000; 30% are advertised as remote. Browse Python roles.
1,453 live UK roles list AWS right now, from 501 employers. The median advertised salary is £80,000; 28% are advertised as remote. Browse AWS roles.
London has 4,313 live IT vacancies across 1,364 employers, with a median advertised salary of £70,000. Browse London roles.
Senior roles make up 1,627 of the live UK IT market and advertise a median of £80,000.
Market figures are today's snapshot of live UK roles on employer careers pages; see hiring trends.
Technologies mentioned
Detected on Methodsdigital's careers page. ProdReady Recruitment lists this vacancy as an aggregator and is not the employer; applications go to the employer's own site. More IT jobs in London.