(a) the procedural steps for ensuring that the auditing organisation to be selected fulfils the conditions laid down in Article 37(3) of Regulation (EU) 2022/2065; (b) the procedural steps for cooperation and assistance by the audited provider in the performance of audits, including accessing relevant information with a view to obtaining audit evidence; (c) the definition and selection of auditing methodologies; (d) the templates for the audit report and the audit implementation report.
Commission Delegated Regulation (EU) 2024/436 of 20 October 2023 supplementing Regulation (EU) 2022/2065 of the European Parliament and of the Council, by laying down rules on the performance of audits for very large online platforms and very large online search engines
Corrected by
- Corrigendum to Commission Delegated Regulation (EU) 2024/436 of 20 October 2023 supplementing Regulation (EU) 2022/2065 of the European Parliament and of the Council, by laying down rules on the performance of audits for very large online platforms and very large online search engines, 32024R0436R(01), March 8, 2024
(1) "auditing organisation" means an individual organisation, a consortium or other combination of organisations, including any sub-contractors, that the audited provider has contracted to perform an independent audit in accordance with Article 37 of Regulation (EU) 2022/2065; (2) "audited service" means a very large online platform or a very large online search engine designated in accordance with Article 33 of Regulation (EU) 2022/2065; (3) "audited provider" means the provider of an audited service which is subject to independent audits pursuant to Article 37(1) of that Regulation; (4) "audited obligation or commitment" means an obligation or commitment referred to in Article 37(1) of Regulation (EU) 2022/2065 which forms the subject matter of the audit; (5) "audit criteria" means the criteria against which the auditing organisation assesses compliance with each audited obligation or commitment; (6) "audit evidence" means any information used by an auditing organisation to support the audit findings and conclusions and to issue an audit opinion, including data collected from documents, databases or IT systems, interviews or testing performed; (7) "misstatement" means an intentional or unintentional omission, misrepresentation or error in the declarations or data reported or provided by the audited provider to the auditing organisation, or in the testing environment made available by the audited provider to the auditing organisation; (8) "audit risk" means the risk that the auditing organisation issues an incorrect audit opinion or reaches an incorrect conclusion concerning the audited provider’s compliance with an audited obligation or commitment, considering detection risks, inherent risks and control risks with respect to that audited obligation or commitment; (9) "detection risk" means the risk that the auditing organisation does not detect a misstatement that is relevant for the assessment of the audited provider’s compliance with an audited obligation or commitment; (10) "inherent risk" means the risk of non-compliance intrinsically related to the nature, the design, the activity and the use of the audited service, as well as the context in which it is operated, and the risk of non-compliance related to the nature of the audited obligation or commitment; (11) "control risk" means the risk that a misstatement is not prevented, detected and corrected in a timely manner by means of the audited provider’s internal controls; (12) "materiality threshold" means the threshold beyond which deviations or misstatements by the audited provider, individually or aggregated, would reasonably affect the audit findings, conclusions and opinions; (13) "reasonable level of assurance" means a high but not absolute level of assurance, which allows the auditing organisation to assert in its audit opinion and audit conclusions whether the audited provider complies with the audited obligations or commitments, based on sufficient and appropriate evidence; (14) "internal control" means any measures, including processes and tests, that are designed, implemented and maintained by the audited provider, including its compliance officers and management body, to monitor and ensure the audited provider’s compliance with the audited obligation or commitment; (15) "vetted researcher" means a researcher vetted in accordance with Article 40(8) of Regulation (EU) 2022/2065; (16) "audit procedure" means any technique applied by the auditing organisation in the performance of the audit, including data collection, the choice and application of methodologies, such as tests and substantive analytical procedures, and any other action taken to collect and analyse information to collect audit evidence and formulate audit conclusions, not including the issuing of an audit opinion or of the audit report; (17) "test" means an audit methodology consisting in measurements, experiments or other checks, including checks of algorithmic systems, through which the auditing organisation assesses the audited provider’s compliance with the audited obligation or commitment; (18) "substantive analytical procedure" means an audit methodology used by the auditing organisation to assess information to infer audit risks or compliance with the audited obligation or commitment.
(a) individually fulfil the requirements laid down in Article 37(3), points (a) and (c), of Regulation (EU) 2022/2065; (b) jointly fulfil the requirement laid down in Article 37(3), point (b), of Regulation (EU) 2022/2065.
(a) a description of the internal controls put in place with respect to each audited obligation and commitment, including related indicators and all present and historical measurements, and benchmarks used by the audited provider to assert or monitor compliance with the audited obligations and commitments, as well as any supporting documentation; (b) its preliminary analysis of inherent and control risks, where the audited provider has performed such an analysis, and any supporting documentation; (c) information about any relevant decision-making structures, competences of departments of the provider, including the compliance function pursuant to Article 41 of Regulation (EU) 2022/2065, relevant IT systems, data sources, processing and storage, as well as explanations of relevant algorithmic systems and their interactions.
(a) the exhaustive list of audited obligations and commitments; (b) the responsibilities of the audit organisation, including, where applicable, detailed for each legal person constituting the auditing organisation, and the parties empowered to sign the audit report; (c) the procedures and contact points made available by the audited provider for the auditing organisation to request access to data referred to in Article 5(2); (d) the timeframe for the audit, including the start and end date of the audit procedures and the completion of the audit report; (e) a procedure on how disputes between the audited provider and the auditing organisation arising from the performance of the audit shall be resolved.
(a) "positive", where the auditing organisation concludes with a reasonable level of assurance that the audited provider has complied with an audited obligation or commitment; (b) "positive with comments", where the auditing organisation concludes with a reasonable level of assurance that the audited provider has complied with an audited obligation or commitment, but: (i) the auditing organisation includes remarks on the benchmarks provided by the audited provider pursuant to Article 5(1), point (a); or (ii) the auditing organisation recommends improvements that do not have a substantive effect on its conclusion;
(c) "negative", where the auditing organisation concludes with a reasonable level of assurance that the audited provider has not complied with an audited obligation or commitment.
(a) "positive" if the auditing organisation has reached a "positive" audit conclusion for all of the audited obligations or commitments; (b) "positive with comments" if the auditing organisation has reached at least one audit conclusion that is "positive with comments" for an audited obligation or commitment and has not reached a "negative" audit conclusion for any of the audited obligations or commitments; (c) "negative" if the auditing organisation reached a "negative" audit conclusion for at least one audited obligation or commitment.
(a) inherent risks; (b) control risks; (c) detection risks.
(a) the nature of the audited service and the societal and economic context in which the audited service is operated, including probability and severity of exposure to crisis situations and unexpected events; (b) the nature of the obligations and commitments; (c) other appropriate information, including: (i) where applicable, information from previous audits to which the audited service was subjected; (ii) where applicable, information from reports issued by the European Board for Digital Services or guidance from the Commission, including guidelines issued pursuant to Article 35(2) and (3) of Regulation (EU) 2022/2065, and any other relevant guidance issued by the Commission with respect to the application of Regulation (EU) 2022/2065; (iii) where applicable, information from audit reports published pursuant to Article 42(4) of Regulation (EU) 2022/2065 by other providers of very large online platforms or of very large online search engines operating in similar conditions or providing similar services to the audited service.
(a) the audit criteria, for assessing compliance with each audited obligation or commitment, defined on the basis of information pursuant to Article 5(1), point (a), and the materiality threshold tolerated and expressed in qualitative or quantitative terms, as appropriate; (b) all tests and substantive analytical procedures and audit evidence that the auditing organisation intends to use to assess compliance for each audited obligation or commitment.
(a) professional judgment and scepticism in assessing information, including concerning internal controls of the audited provider, that leads the auditing organisation to formulate reasonable doubts; (b) external indications pointing to audit risks, in particular reports from the European Board for Digital Services referred to in Article 35(2) of Regulation (EU) 2022/2065, guidance from the Commission including through guidelines referred to in Article 35(3) of that Regulation, and any other relevant guidance issued by the Commission with respect to the application of Regulation (EU) 2022/2065, and audit reports issued pursuant to codes of conduct or crisis protocols referred to in Articles 45, 46 and 48 of that Regulation; (c) information related to events occurring during the performance of the audit, including crisis situations, that require additional actions from the audited provider to ensure compliance with certain audited obligations or commitments.
(a) the performance of tests and substantive analytical procedures for the internal controls the audited provider has put in place for each of the audited obligations or commitments; (b) the performance of substantive analytical procedures to assess compliance with each audited obligation and commitment, including as regards algorithmic systems; (c) the performance of tests, including with respect to algorithmic systems, concerning the audited obligations and commitments in relation to which the auditing organisation has reasonable doubts, as referred to in paragraph 4, and concerning audited obligations and commitments where the auditing organisation deems necessary to perform tests in its choice of methodology pursuant to paragraph 1.
(a) it is relevant and sufficient to reduce audit risks identified in accordance with Article 9, and to enable the auditing organisation to provide audit conclusions and opinions in accordance with Article 8; (b) it is reliable, according to the auditing organisation’s professional judgment and scepticism.
(a) the representativeness of the sample for the period referred to in Article 3(2) and (3); (b) relevant changes to the audited service during that period; (c) relevant changes to the context in which the audited service is provided during that period; (d) relevant features of algorithmic systems, where applicable, including personalisation based on profiling or other criteria; (e) other relevant characteristics or partitions of the data, information and evidence under consideration; (f) the representation and appropriate analysis of concerns related to particular groups as appropriate, such as minors or vulnerable groups and minorities, in relation to the audited obligation or commitment.
(a) whether the audited provider has diligently identified, analysed, and assessed the systemic risks in the Union referred to in Article 34(1), first subparagraph, of Regulation (EU) 2022/2065, including by assessing: (i) how the audited provider identified the risks that are linked to its service, taking into account regional and linguistic aspects of the use made of its service, including when specific to a Member State, and whether the risks are appropriately identified; (ii) how the audited provider analysed and assessed each risk, including how it considered the probability and severity of the risks, and whether the assessment was appropriate; (iii) how the audited provider identified, analysed and assessed the factors referred to in Article 34(2), first subparagraph, of Regulation (EU) 2022/2065, whether they were appropriately identified, and to what extent such factors influence the risks identified in paragraph 1 of that Article; (iv) what sources of information the audited provider used, how it collected the information, including whether and how it relied on scientific and technical insights; (v) whether and how the audited provider tested assumptions on risks with groups most impacted by the specific risks;
(b) whether the risk assessment was performed within the timeframes set out in Article 34(1), second subparagraph, of Regulation (EU) 2022/2065 and, where applicable, within the timeframes set for activities established as risk mitigation measures for the detection of systemic risks pursuant to Article 35(1), point (f) of that Regulation; (c) how the audited provider identified functionalities that are likely to have a critical impact on the risks for which risk assessments shall be conducted prior to their deployment, pursuant to Article 34(1), second subparagraph, of Regulation (EU) 2022/2065, whether those functionalities were correctly identified, and whether the risk assessment was appropriately conducted; (d) whether the audited provider correctly identified the supporting documentation that should be preserved with respect to the risk assessment and whether it has put in place the necessary means to ensure the preservation of that documentation for at least three years, pursuant to Article 34(3) of Regulation (EU) 2022/2065, and whether the documentation was preserved accordingly.
(a) the internal controls that the audited provider has put in place to monitor the performance of risk assessments regarding each factor referred to in Article 34(2), first subparagraph, of Regulation (EU) 2022/2065; such assessment shall: (i) be based on substantive analytical procedures, for those internal controls; (ii) be based on tests of whether those internal controls are reliable and diligently conceived, executed and monitored; (iii) evaluate how the compliance officer or officers performed their tasks with respect to Article 41(3), points (b), (d), (e) and, where applicable, (f), of Regulation (EU) 2022/2065 and how the management body of the audited provider was involved in the decisions related to risk management pursuant to Article 41(6) and (7) of that Regulation;
(b) the actions, means and processes put in place by the audited provider to ensure compliance with Article 34 of Regulation (EU) 2022/2065 and the results thereof; such assessment shall be based on: (i) substantive analytical procedures; (ii) tests, including of algorithmic systems, where the auditing organisation has reasonable doubts, following the results of the substantive analytical procedures and the assessment of internal controls, or where the auditing organisation deems necessary to perform tests in its choice of methodology pursuant to article 10(1).
(a) the risk assessment report for the relevant audited period, which has been drawn up by the audited including, where necessary, confidential information that is not part of the information published pursuant to Article 42(2) of that Regulation, and all supporting documents; (b) where relevant, other risk assessments reports of the audited provider and their supporting documents; (c) information submitted by the audited provider pursuant to Article 5; (d) all relevant transparency reports of the audited provider referred to in Article 15(1) of Regulation (EU) 2022/2065; (e) any other test results, documentation, evidence, statements made in response to written or oral questions addressed by the auditing organisation to the personnel of the audited provider, and observations made on premises, where applicable; (f) other relevant evidence, including based on information made available by the audited provider; (g) where available, reports referred to in Article 35(2) of Regulation (EU) 2022/2065 and guidance from the Commission, including guidelines issued pursuant to Article 35(3) of that Regulation and any other relevant guidance issued by the Commission with respect to the application of Regulation (EU) 2022/2065.
(a) how the audited provider identified risk mitigation measures for each of the systemic risks referred to in Article 34(1) of Regulation (EU) 2022/2065, and whether the identification of such risk mitigation measures was carried out in a diligent manner; (b) how the audited provider assessed whether the risk mitigation measures in Article 35(1), points (a) to (k), of Regulation (EU) 2022/2065 were applicable to the audited service and whether the conclusion of that assessment was appropriate, including as regards those measures which were not applied by the audited provider; (c) whether the mitigation measures put in place by the audited provider are reasonable, proportionate and effective for mitigating the respective risks, including by: (i) assessing whether they respond collectively to all the risks, with particular consideration of the risks concerning the exercise of fundamental rights; (ii) assessing comparatively how the risks were addressed before and after the specific risk mitigation measures were put in place; (iii) assessing whether the risk mitigation measures were appropriately designed and executed.
(a) the internal controls the audited provider has put in place to monitor the application of risk mitigation measures referred to in Article 35(1) of Regulation (EU) 2022/2065 and whether they are reasonable, proportionate and effective; such assessment shall: (i) be based on substantive analytical procedures for those internal controls; (ii) be based on tests, of whether those internal controls are reliable and diligently conceived, executed and monitored; (iii) evaluate how the compliance officer or officers performed their tasks with respect to Article 41(3), points (b), (d), (e) and, where applicable, (f), of Regulation (EU) 2022/2065, and how the management body of the provider was involved pursuant to Article 41(6) and (7) of that Regulation;
(b) mitigation measures put in place by audited providers; such assessment shall be based on: (i) substantive analytical procedures; (ii) tests, including of algorithmic systems, where the auditing organisation has reasonable doubts, following the results of the substantive analytical procedures and the assessment of internal controls, or where the auditing organisation deems necessary to perform tests in its choice of methodology pursuant to Article 10(1).
(a) the reports on risk assessment and risk mitigation for the relevant audited period, which have been drawn up by the audited provider including, where necessary, confidential information that is not part of the information published pursuant to Article 42(2) of Regulation (EU) 2022/2065, and all supporting documents; (b) where relevant, other reports on risk assessment and risk mitigation of the audited provider and their supporting documents; (c) information submitted by the audited provider pursuant to Article 5; (d) all relevant transparency reports of the audited provider referred to in Article 15(1) of Regulation (EU) 2022/2065; (e) where relevant, past reports on risk mitigation and their supporting documents, which concern periods not covered by the audited period, including, where necessary, confidential information that is not part of the information published pursuant to Article 42(2) of Regulation (EU) 2022/2065; (f) any other test results, documentation, evidence, statements made in response to written and or oral questions addressed by the auditing organisation to the personnel of the audited provider, and observations made on premises, where applicable; (g) other relevant evidence, including based on information made available by the audited provider; (h) where available, reports referred to in Article 35(2) of Regulation (EU) 2022/2065 and guidance from the Commission, including guidelines issued pursuant to Article 35(3) of that Regulation and any other relevant guidance issued by the Commission with respect to the application of Regulation (EU) 2022/2065.
(a) whether and how the audited provider identified the relevant systems involved in the functioning and use of their service that significantly contribute to the serious threat and whether those systems were appropriately identified; (b) whether and how the audited provider defined and monitored the significant contribution to the serious threat and whether its assessment was appropriate; (c) any other requirement specified in the Commission’s decision referred to in Article 36(1) or (7), second subparagraph, of Regulation (EU) 2022/2065, as appropriate.
(a) whether and how the audited provider identified measures to prevent, eliminate or limit any contribution to the serious threat; (b) whether and how the measures taken by the audited provider addressed the gravity of the serious threat, the urgency, and whether the measures were appropriate in this respect; (c) whether and how the audited provider identified the parties concerned by the measures and their legitimate interests, and how the audited provider assessed the actual or potential impact of the measures on those parties’ rights, including fundamental rights, and legitimate interests; (d) whether the measures taken by the audited provided were effective and proportionate; (e) any other requirement specified in the Commission’s decision referred to in Article 36(1) or (7), second subparagraph, of Regulation (EU) 2022/2065, as appropriate.
(a) a list and the text of all codes of conduct referred to in Articles 45 and 46 of Regulation (EU) 2022/2065 and crisis protocols referred to in Article 48 of that Regulation, to which the audited provider is a signatory; (b) a detailed list of commitments within those codes of conduct and crisis protocols that the audited provider has taken; (c) where applicable, the key performance indicators agreed under each code of conduct and crisis protocol; (d) where applicable, any available measurements, data and documentation, and any reports prepared by the audited provider with respect to the compliance of the audited provider with the commitments taken, including access to all relevant information and data related to the functioning of the services offered by the audited provider relevant to the implementation of the code of conduct or the crisis protocol; (e) where applicable, other measurements, data and documentation prepared by signatories of the code of conduct or the crisis protocol, and the assessments by the Commission or the Board referred to in Article 45(4) of Regulation (EU) 2022/2065.
|
|
| SECTION G: Any other information the auditing body wishes to include in the audit report (such as a description of possible inherent limitations). |
| SECTION A: General Information |
|
