Face Morphing Attacks on Secure Documents and an AI-Based Detection Architecture

A face-morphing attack involves combining the facial features of two or more people into a single application photo. The goal is for the resulting photo to exhibit sufficient similarity in a facial recognition system to both the actual applicant and another person with whom they are collaborating. A genuine passport created with such a photo can pass both physical and cryptographic checks; the problem is not that the document is fake, but that the biometric reference recorded on the document is flawed from the start.

NIST emphasizes that the risk increases particularly in processes where the applicant can upload their own photo and the photo’s digital history is unknown. The most effective measure is to take the photo in person at a trusted station under the control of the issuing authority. If this is not possible, morphing attack detection should be integrated into the application and border control processes as separate tasks.

Two Different Detection Scenarios

Single-image MAD (S-MAD)

Only the suspicious portrait is examined at the application office. The model searches for blending traces around the face, double edges, inconsistent texture, local frequency differences, and artifacts remaining from the production chain. S-MAD may be sensitive to the morph production method used by the attacker and to subsequent recompression of the image.

Differential MAD (D-MAD)

At the border checkpoint, the reference portrait stored on the chip is evaluated in conjunction with a reliable live capture. In addition to the face matching score, identity consistency and morph indicators between the two images are analyzed. Presentation attack detection and controlled capture are also required to ensure the reliability of the live image.

Recommended Operational Architecture

1. Apply ICAO portrait quality controls to the application photo.
2. Record the source of the photo and the capture method.
3. Generate the S-MAD score as an independent risk indicator.
4. During new registration and renewal, investigate whether the same portrait is associated with multiple identities using a 1:N search.
5. At border control, run D-MAD against the chip portrait using a live image.
6. Refer incidents exceeding the threshold to a trained expert for review.

A single model score should not automatically serve as grounds for rejection. The model version, capture device, quality metrics, and review result must be stored together.

Example decision orchestration using Python

from dataclasses import dataclass
from enum import Enum

class Decision(str, Enum):
    PASS = "PASS"
    REVIEW = "REVIEW"
    REJECT = "REJECT"

@dataclass
class MorphSignals:
    single_image_mad: float
    differential_mad: float | None
    face_match: float | None
    capture_trusted: bool
    quality_ok: bool

def decide(s: MorphSignals) -> Decision:
    if not s.quality_ok:
        return Decision.REVIEW

    # Güvenilmeyen başvuru fotoğrafında daha sıkı inceleme.
    if not s.capture_trusted and s.single_image_mad >= 0.55:
        return Decision.REVIEW

    # D-MAD ve face match farklı görevlerdir.
    if s.differential_mad is not None:
        if s.differential_mad >= 0.60:
            return Decision.REVIEW
        if s.face_match is not None and s.face_match < 0.45:
            return Decision.REVIEW

    return Decision.PASS

The thresholds here are examples; in a real system, they must be calibrated using independent test data, target false acceptance rate, demographic performance, and operational costs.

How should we capture the result?

{
  "documentId": "opaque-reference",
  "captureTrusted": false,
  "quality": "PASS",
  "singleImageMad": {"score": 0.71, "model": "smad-2.3"},
  "differentialMad": null,
  "decision": "REVIEW",
  "reasonCodes": ["UNTRUSTED_CAPTURE", "SMAD_THRESHOLD"],
  "humanReview": "PENDING"
}

Performance and security metrics

The morph detector should not be evaluated solely based on overall accuracy. The false alarm rate for bona fide photos, the detection rate for different morph generation methods, performance after printing/scanning and JPEG compression, and performance across different devices and demographic groups should be reported separately. When a new attack technique emerges, the model’s generalization ability should be reevaluated.

Facial images are biometric data. Access to raw photos must be restricted, retention periods must be defined, and production data must not be transferred to the model development environment without proper controls.

Conclusion

The most effective defense against morphing attacks is a reliable photo capture process. AI-based S-MAD and D-MAD are risk sensors that complement this process. A secure implementation combines photo quality, 1:N search, liveness detection, face matching, morph detection, and expert review into an explainable decision flow.

References

• NISTIR 8584 — Considerations for Implementing Morph Detection in Operations
• NISTIR 8430 — Utility of 1:N Face Recognition for Morph Detection