Security Tips for Procuring AI Solution

When selecting AI technologies, tools, services, etc., departments or users should prioritize security, including risk assessment and human oversight, throughout the acquisition process so as to ensure lawful, secure, privacy-protective, reliable and suitable for intended purpose acquisition. The following outlines some points recommended by PCPD when evaluating the AI solutions.

• Define requirements:
  • Define the intended purpose and requirements of using AI solution.
  • Convey the key privacy and security obligations and ethical requirements to potential AI suppliers.
  • Specify the technical and governance standards that potential AI suppliers should follow.
• Sourcing appropriate AI solutions:
  • Review of the expertise and reputation of AI suppliers.
  • Evaluate performance, accuracy, scalability, and reliability of the AI solution, as well as the level of human oversight required.
  • Examine the interface, ease of use, and accessibility to determine whether the tool can be easily adopted by users.
  • Understand what kind of data need to be feed to the AI solution.
  • Require vendors to provide sufficient information of:
    • The core functionality and limitations of the AI solution.
    • Level of human oversight required: whether and how human reviewers have been involved in the training and development of the AI models to reduce the risk of significant impacts on individuals during deployment.
    • Any known issues relating to accuracy, bias, privacy or security of the AI solution.
• Conduct Risk assessments:
  • Review how the AI solution collect, use, share, store, retain, protect and delete data, with attention to privacy and confidentiality.
  • Understand whether data will be used for model training, service improvement or profiling.
  • Review vendor’s history regarding data breaches or vulnerabilities, check customer reviews, and support offerings, etc.
  • Identify the risk including privacy, security and ethical risk arising from the AI solution.
    • Understand the volume, sensitivity and quality of the data involve.
    • If there is any personal data need to be used for customizing the AI solution, fed into the AI systems, or to be collected by the AI system, anonymisation techniques should be applied, as far as possible, to adhere to the data minimization principle, and having regard to the Data Protection Principles (“DPPs”) of PCPD’s Personal Data (Privacy) Ordinance (“PDPO”).
    • Understand how personal data may be transferred in and out of the AI systems and evaluate any security measures on AI-generated output would be in place to mitigate the risk of personal data leakage.
    • Identify the potential impacts (including benefits and harms) of the AI system on the affected individuals, the organisation and the wider community.
  • Analyze and evaluate whether the risks can be mitigated, accepted and managed.
    • Evaluate the probability, severity and duration of the impact, and identify the mitigation measures to minimize the risk of harm.
    • Adopt risk-based approach to define appropriate level of human oversight required in the use of the AI system.
      • High-risk AI system should take “human-in -the-loop” approach to retain control of the decision making.
      • Low-risk AI system may take “human-out-of-the-loop” approach which allow the AI system to adopt output or decision making.
      • Other than above risk levels, a “human-in-command” approach may consider, where human actors utilize the output of the AI system, oversee the operation of the AI system and intervene whenever necessary.
    • Any AI use cases with risks that are unacceptable, they should be disallowed.
• Evaluate Security practices:
  • Understand the data protection mechanisms, e.g. encryption standards, access controls, audit capabilities, etc. of the AI solution, and incident response protocols of vendor.
  • Review the ability to delete inappropriate or inaccurate data within the AI solution.
  • Require vendors to disclose any third-party dependencies and provide documentation regarding their security practices.
  • Establish clear security requirements and agreements with vendors. Adopt contractual or other means to prevent unauthorized or accidental access, processing, erasure, loss or use of the personal data.
• Compliance with Data Privacy regulations:
  • Ensure compliance with the requirements under PCPD’s Personal Data (Privacy) Ordinance (“PDPO”), including the six Data Protection Principles (“DPPs”) when departments handling personal data in the process of procuring, implementing and using AI solutions.
  • If the use of AI solution involves processing personal data which will be transferred to places outside Hong Kong, ensure compliance with the requirement of related jurisdiction (refer to the Security Measure Checklist for Cross-border Data Transfer).
  • Ensure align with the University’s policy in Protection of Personal Data.
• Contractual Controls:
  • Any data processor agreements need to be signed, if the procured AI solution involves the engagement of data processors.
  • Engagement with vendor may raise data (including personal data) protection compliance issues, which should be clearly addressed in the service agreements signed between the parties.

Reference:

• PCPD: Artificial Intelligence Model Personal Data Protection Framework
  (https://www.pcpd.org.hk/english/resources_centre/publications/files/ai_protection_framework.pdf)

Published on:  Apr 2026