Data Management and Compliance Reporting

Descriptions

The Department for Environment, Food and Rural Affairs (Defra) will be inviting proposals to provide the Management of ‘Air Quality’ Data and the Formal Reporting of Air Quality for the ‘Data Management and Compliance Reporting’ contract. This project will be split into two lots. Lot 1: Delivers the collection, aggregation, storage, management, and dissemination of air pollutant monitoring data. Monitored air quality data is collected through two routes a) via a self-service portal (provided by the Contractor) for non-automated data and b) automatic receipt and upload as well as being harvested from known URI’s. For the automatic data it is initially termed “provisional” until data quality checks confirm it as ratified. Modelled meteorological data as well as UV and stratospheric ozone is also acquired and stored. Aggregations and statistics are calculated, including exceedance statistics against UK statutory limits. All changes to the data are logged, and all site metadata and measurement configurations are stored. The data is then made available to various authorities, scientists and academics, and third-party data modelling suppliers, for various purposes in various forms. Pollutant data is to be made available in specialist formats for formal reporting, daily “air quality” forecast validation as well as other uses. Data is made available as R objects, XML files, RSS feeds, APIs, and to support gov.uk services being developed and the legacy UK-AIR web service. Lot 2: Delivers the formal reporting of air quality across the UK as required under the Air Quality Standards Regulations (AQSR) (2010) and the Air Quality (Miscellaneous Amendment and Revocation of Retained Direct EU Legislation) (EU Exit) Regulations (2018). E-reporting is required annually by 30th September each year in a pre-defined format that meets the requirements of the INfrastructure for SPatial InfoRmation (INSPIRE) Regulations. Rules around reporting are defined within Implementing Provision Decisions . The results will be assessed, and accessible summaries and technical reports of compliance will need to be produced. Data is made available as XML files, shapefiles and csv’s. Data is also reported to the European Monitoring and Evaluation Programme (EMEP) . Defra requires a supplier or suppliers to carry out the services detailed in both Lots for the Data Dissemination Unit for 3 years between April 2026 to March 2029 with a potential for 2 x 1 year extensions. Further information on Lots 1 and 2: Lot 1: The work to be delivered by the successful Contractor is as follows: • Host a data providers portal and create templates for data submission (for non-automatic data). • Host an annual data providers workshop and send reminders to ensure data is provided to defined timescales. • Manage automatic and non-automatic data acquisition, acquiring data either through the Data Providers Portal or harvesting from known URI’s. • Collect all site metadata and measurement configurations. • Calculate statistics and aggregations on an hourly basis, or overnight as appropriate. • When data is ratified (by QA/QC Contractors), collect this ratified data and update the data in the database accordingly. Log all changes to the data. • Undertake consistency, integrity and reality checks on data flows. • Store all data and demonstrate how this is done securely and sustainably. • Make this data available to various authorities, scientists and academics, and third-party data modelling suppliers, for various purposes in various forms. • Data is made available as R objects, XML files, RSS feeds and API’s. Data is also to be made available to support the legacy UK-AIR web service. The skills required for Lot 1 include: • Proven ability to manage large and complex datasets, including routine and ad hoc calculation, aggregation, and storage of statistics in accordance with defined standards. • Demonstrated experience working with air quality data, including its collection, processing, and interpretation, or a proven ability to quickly gain domain expertise in air quality datasets. • Strong data engineering and management skills, including the ability to design, implement, and support modern data delivery mechanisms (e.g. APIs, XML feeds). • Ability to respond to technical and non-technical data queries quickly and accurately, demonstrating strong problem-solving and communication skills. • Expertise in producing accurate, timely data outputs fully compliant with legislative and regulatory requirements. • Experience in managing data efficiently and sustainably, ensuring data systems are robust, scalable, and maintainable over time. • Track record of managing external data providers, ensuring data completeness, accuracy, and delivery in line with defined timescales and contractual obligations. Lot 2: The work to be delivered by the successful Contractor is as follows: • Gather supporting information, data and metadata relating to e-Reporting. • Produce and quality assure all data flows required for e-reporting against the AQSR (2010) and EU Exit Regulations. • Deliver on this reporting by 30th September each year in a pre-defined format that meets INSIPRE requirements. • This requires reporting against air quality zones, assessment regimes, assessment methods, validated and provisional measurement data, validated modelled data, and attainment against objectives. • Each dataflow is required as an XML file or shapefile. • The measurement data and aggregations are to be made available as csv files. • Analyse the results and produce an annual compliance assessment report, with visualisations, trends and an accessible summary document. • Report data from ‘supersites’ and five rainwater sites to the EMEP on an annual basis. The skills required for Lot 2 include: • Extensive knowledge of air quality data, including structure, sources, and standards, as well as the reporting requirements under UK and/or EU legislation • Strong data management and processing capabilities, including demonstrated ability to generate and validate XML files, shapefiles, and other structured data formats required for statutory reporting. • Proven experience collaborating with technical specialists, such as air quality modellers, emissions inventory teams, and data providers, to support accurate, timely and coherent data submissions.To include email address for responses to be sent to dgc-laww-buyer@defra.gov.ukEngagement process descriptionThe Department for Environment, Food and Rural Affairs (Defra) will be inviting proposals to provide the Management of 'Air Quality' Data and the Formal Reporting of Air Quality for the 'Data Management and Compliance Reporting' contract. This project will be split into two lots, the primary purpose of Lot 1 is to collect, process and store air quality monitoring data from the national monitoring networks, to aggregate this data and calculate statistics, and to make this data available to various parties. The purpose of Lot 2 is to deliver formal reporting against the Air Quality Standards Regulations (2010) and to produce accessible summaries and technical reports of compliance against UK limit values, target values and long-term objectives. This requires producing XML dataflows that adhere to a strict reporting schema. Data is also reported to the European Monitoring and Evaluation Programme. A draft specification for this opportunity can be requested via email to dgc-laww-buyer@defra.gov.uk inbox by referencing this opportunity. Depending on interest received, Defra will hold an online Market Engagement Online Event on the week commencing 30th June 2025 to discuss any technical elements of this project. Please register your interest for this event no later than 20th June 2025. The Authority may contact you to confirm your interest and to discuss your capability to fulfil this requirement. In order to understand the markets appetite and ability to resource and bid for this work, we are requesting you respond to this notice to confirm if you are interested in bidding for this work and that you can meet the requirements outlined above. Please confirm your interest in bidding for this opportunity and/or attending the Market Engagement Online Event by emailing dgc-laww-buyer@defra.gov.uk no later than 3pm on 20th June 2025. It should also be noted that this notice invites individuals and organisations to express an interest in this exercise only. It is NOT a pre-qualification questionnaire, is NOT part of any pre-qualification or selection process, and is NOT a call for competition.