Video Kyc Hackathon

An AWS-native Video KYC (VCIP) reference implementation for Indian fintechs, banks, and regulated businesses. It combines real-time video verification with AI services for liveness detection, ID OCR, face matching, and NLP risk insights, and persists a complete, RBI-aligned audit trail.

Stack: Node.js (ESM) · Express · Socket.IO (WebRTC signaling) · AWS Rekognition / Textract / Comprehend · Amazon S3 · DynamoDB · vanilla HTML/CSS/JS frontends.

---

Table of Contents

• Problem Statement
• Product Objective
• Key Features
• AI Use Cases
• How VCIP / OCR / Document Extraction Works
• Architecture Overview
• Frontend Overview
• Backend Overview
• Database / Schema Overview
• API Overview
• AWS / Cloud Usage
• Folder Structure
• Setup on a New Machine
• Environment Variables
• Running the App
• Testing the Project
• Demo Data
• Known Limitations
• Future Improvements
• AI Product Manager Skills Demonstrated
• Claude Code / AI-Assisted Development Workflow

---

Problem Statement

Customer onboarding in Indian fintech/banking is gated by KYC. In-person KYC is slow and costly; unsupervised digital KYC is fraud-prone (deepfakes, document tampering, identity mismatch). The RBI's Video-based Customer Identification Process (VCIP) allows remote onboarding but requires liveness, genuine document capture, face match to the ID, geo-tagging, and a tamper-evident audit trail.

Building this from scratch means stitching together video infrastructure, multiple AI models, secure storage, and compliance-grade logging — a large surface area most teams underestimate.

See it running: docs/walkthrough/WALKTHROUGH_CAPTURED.md — the UI captured live without AWS credentials (home, verifier dashboard, KYC session, Smart Compliance Assistant console).

Product Objective

Provide a working reference implementation of an RBI-aligned VCIP flow that:

• runs a real-time assisted video session between a customer (merchant) and a verifier,
• automates the identity checks (liveness → ID OCR → face match) using managed AWS AI services,
• captures every step as an auditable artifact in a single-table DynamoDB design,
• keeps all data in the ap-south-1 (Mumbai) region for data-residency.

---

Key Features

Every feature below maps to code in server.js / web/.

• Real-time assisted video KYC — merchant↔verifier video via WebRTC, signaled over Socket.IO; live chat and step-by-step verification updates.
• AI liveness detection — AWS Rekognition Face Liveness session creation + result scoring.
• ID OCR / document extraction — AWS Textract AnalyzeID (PAN/Aadhaar/Passport fields) and DetectDocumentText (free-text OCR).
• Face match — detect the largest face on the ID, crop it with sharp, and CompareFaces against the liveness/selfie reference (similarity threshold 85%).
• Face registration & verification — index faces into a Rekognition Collection and search by image.
• NLP risk insights — AWS Comprehend sentiment, entities, key phrases, dominant language, and PII detection + masking, plus a combined OCR→NLP document pipeline and batch analysis.
• Audit trail — every artifact (liveness, OCR, match, NLP, summary) stored in DynamoDB under a composable sort-key pattern; evidence images stored in S3.
• Verifier dashboard & console — UI to monitor active sessions and review analysis.

---

AI Use Cases

Capability	AWS Service	Where in code
Face liveness (anti-spoofing)	Rekognition CreateFaceLivenessSession, GetFaceLivenessSessionResults	/api/liveness/*
Face match (selfie ↔ ID portrait)	Rekognition DetectFaces, CompareFaces	/api/face-match
Face registry (1:N)	Rekognition IndexFaces, SearchFacesByImage, Collections	/api/face/register, /api/face/verify
ID field extraction	Textract AnalyzeID	/api/id/analyze
Document OCR	Textract DetectDocumentText	/api/docs/comprehend
Sentiment / entities / key phrases	Comprehend	/api/comprehend/{sentiment,entities,keyphrases}
PII detection + masking	Comprehend DetectPiiEntities	/api/comprehend/pii
Language detection	Comprehend DetectDominantLanguage	used in /api/docs/comprehend, batch

How VCIP / OCR / Document Extraction Works

End-to-end flow (assisted session):

1. Create session — POST /api/kyc/create-session mints a sessionId, writes a SUMMARY artifact to DynamoDB, and returns a join URL.
2. Join video — merchant and verifier open the session; Socket.IO relays WebRTC offer/answer/ICE so the two browsers establish a peer video connection. Verification steps and chat sync in real time.
3. Liveness — POST /api/liveness/session creates a Rekognition Face Liveness session (audit images land in S3). GET /api/liveness/result fetches the confidence score (isLive when ≥ 80) and stores the reference image + result on the SUMMARY artifact. (A mock-upload-selfie path exists for demos without the Cognito-backed client SDK.)
4. ID OCR — POST /api/id/analyze runs Textract AnalyzeID on the uploaded ID image(s), returns the extracted IdentityDocumentFields, and stores the raw image in the docs S3 bucket.
5. Face match — POST /api/face-match detects the largest face on the ID, crops it (sharp), pulls the liveness/selfie reference from S3, and CompareFaces returns a similarity score (matched when ≥ 85).
6. NLP (optional) — /api/docs/comprehend chains Textract OCR → Comprehend (language, sentiment, entities, key phrases); /api/comprehend/pii masks PII for safe display.
7. Complete — the verifier marks the session complete; the final decision/notes are written to the SUMMARY artifact and the in-memory session is cleaned up.

---

Architecture Overview

flowchart LR
  subgraph Browser[Browser UIs - web/]
    IDX[index.html]
    KYC[kyc-session.html merchant]
    LIV[liveness.html demo]
    COMP[comprehend.html]
    VDASH[verifier-dashboard.html]
    VCON[verifier-console.html]
  end

  subgraph Server[Express + Socket.IO - server.js]
    REST[REST API]
    WS[WebRTC signaling]
  end

  Browser <-->|HTTP /api/*| REST
  Browser <-->|WebSocket| WS

  REST --> REK[(AWS Rekognition)]
  REST --> TEX[(AWS Textract)]
  REST --> NLP[(AWS Comprehend)]
  REST --> S3[(Amazon S3)]
  REST --> DDB[(DynamoDB)]

• Single Node process serves both the static frontends (/web) and the REST API, and hosts the Socket.IO server used purely for WebRTC signaling (the media stream itself is peer-to-peer).
• AWS SDK v3 clients are created once per service and reused.

Frontend Overview

Plain, framework-free HTML/CSS/JS served statically from /web (no build step):

Page	Purpose
index.html	Landing page; create a KYC session, links to dashboards
kyc-session.html	Merchant video room (WebRTC + chat + step status)
liveness.html	4-step API demo: liveness → selfie → ID analyze → face match
comprehend.html	NLP demo (sentiment/entities/PII over text or documents)
verifier-dashboard.html	Verifier view of active sessions
verifier-console.html	Verifier review/analysis console

Note: the app root / is not a page — the UI lives under /web/index.html.

Backend Overview

server.js (Express, ESM) provides:

• REST endpoints for liveness, ID OCR, face match, face registry, NLP, session management, and timelines.
• A Socket.IO connection handler implementing WebRTC signaling and live verification/chat events.
• AWS SDK v3 integrations (Rekognition, Textract, Comprehend, S3, DynamoDB) and helper utilities for NLP normalization, language auto-detect, PII masking, and DynamoDB artifact writes.

setup-aws.js is a one-time idempotent bootstrapper that creates the S3 buckets, the DynamoDB table, and the Rekognition collection if they don't already exist.

Database / Schema Overview

DynamoDB single-table design — table vcip_sessions_v2, PAY_PER_REQUEST:

Attribute	Type	Role
sessionId	S	Partition key
artifactType	S	Sort key

One session = one partition; each step is a separate artifact row under it:

artifactType	Meaning
SUMMARY	Session rollup + latest state/decision
LIVENESS#<ts>	Liveness result (inferred from artifact convention)
OCR#<ts>	ID/document OCR result (inferred)
MATCH#<ts>	Face-match outcome (inferred)
FACE#<ts>	Face registration metadata
VERIFY#<ts>	Face verification attempt
NLP#<ts>	Comprehend analysis
MASTER#<date>	Master KYC record (seed/demo)

This lets the timeline endpoint return an ordered history of everything that happened in a session with a single query.

API Overview

All routes are defined in server.js. (A live, self-describing list is available at GET /_routes.)

Method	Path	Purpose
GET	/health	Health check
GET	/_routes	List mounted routes
POST	/api/liveness/session	Create Rekognition liveness session
GET	/api/liveness/result	Fetch liveness score/result
POST	/api/liveness/mock-upload-selfie	Upload a mock reference selfie (demo)
POST	/api/id/analyze	Textract AnalyzeID on ID image(s)
POST	/api/face-match	Compare ID portrait vs reference selfie
POST	/api/kyc/create-session	Create a real-time KYC session
GET	/api/kyc/active-sessions	List active sessions (verifier dashboard)
POST	/api/comprehend/sentiment	Sentiment analysis
POST	/api/comprehend/entities	Entity extraction
POST	/api/comprehend/keyphrases	Key-phrase extraction
POST	/api/comprehend/pii	PII detection + masking
POST	/api/comprehend/save	Persist an NLP artifact to a session
POST	/api/comprehend/batch	Batch NLP over multiple texts
POST	/api/docs/comprehend	Document OCR → NLP pipeline
GET	/api/session/:sessionId/timeline	Ordered artifact timeline for a session
POST	/api/face/register	Index a face into the collection
POST	/api/face/verify	Search the collection by face image

Socket.IO events: join-kyc-session, webrtc-offer, webrtc-answer, webrtc-ice-candidate, verification-step-complete, chat-message, complete-kyc-session, disconnect.

AWS / Cloud Usage

• Region: ap-south-1 (Mumbai) for RBI data-residency.
• Rekognition: liveness, face detection/compare, face collections.
• Textract: AnalyzeID, DetectDocumentText.
• Comprehend: sentiment, entities, key phrases, language, PII.
• S3: AUDIT_BUCKET (liveness/session evidence), DOCS_BUCKET (ID document images).
• DynamoDB: vcip_sessions_v2 audit store.
• KMS (optional): set KMS_KEY_ID to encrypt liveness output.
• IAM: the credentials used need access to the five services above. (Least-privilege policy is a recommended TODO — see Known Limitations.)

---

Folder Structure

.
├── server.js                 # Express + Socket.IO API + WebRTC signaling
├── setup-aws.js              # One-time AWS resource bootstrapper
├── seed.sh                   # Seed demo rows into DynamoDB (vcip_sessions)
├── seed-items.json           # Demo data (v1 table)
├── seed-items-v2.json        # Demo data (v2 layout)
├── package.json
├── .env.example              # Environment variable template (copy to .env)
├── README.md
├── API.md                    # Auto-generated endpoint reference (npm run api-spec)
├── COMPLIANCE_MATRIX.md      # RBI-VCIP requirement→code traceability (npm run compliance)
├── REVIEW.md                 # Multi-agent code-review findings
├── DEMO_SCRIPT.md / AI_WORKFLOWS.md / PROMPTS_LIBRARY.md / INTERVIEW_NOTES.md
├── tools/                    # Developer & PM tooling (see below)
│   ├── doc-drift.mjs         #   detect docs/code route drift
│   ├── api-spec.mjs          #   generate API.md from server.js
│   ├── compliance-matrix.mjs #   generate COMPLIANCE_MATRIX.md
│   ├── gen-synthetic-data.mjs#   synthetic, non-PII test data
│   └── smoke.mjs             #   smoke-test a running server
└── web/                      # Static frontends (no build step)
    ├── index.html
    ├── kyc-session.html
    ├── liveness.html
    ├── comprehend.html
    ├── verifier-dashboard.html
    └── verifier-console.html

Developer & PM tooling (tools/)

Command	Does	Output
npm run doc-drift	Diffs server.js routes against the docs	drift report (exit 1 on drift)
npm run api-spec	Generates the endpoint reference from the code	API.md
npm run compliance	RBI-VCIP requirement → feature → code traceability	COMPLIANCE_MATRIX.md
npm run gen-data	Synthetic, non-PII ID/selfie images + session JSON (--load to push)	synthetic-data/ (gitignored)
npm run smoke	Smoke-tests a running server (health, create-session, timeline)	pass/fail

These tools (and the multi-agent review in REVIEW.md) are the Implemented Claude Code workflows documented in AI_WORKFLOWS.md.

Claude Code config (committed under .claude/ + root CLAUDE.md, .mcp.json): project memory, slash commands (/secret-scan, /repo-audit, /new-endpoint, /review, /compliance-check, /env-doctor), subagents (security-auditor, code-reviewer, aws-cost-auditor, test-writer), a vcip-kyc skill, hooks (pre-push secret-scan that blocks on an AKIA… hit, post-edit doc-drift, a Stop status sign-off), and MCP servers (Playwright + filesystem; GitHub/AWS/Context7 in .claude/MCP_SETUP.md). See AI_WORKFLOWS.md §4.

Not committed (gitignored): .env, .claude/, large AWS CLI installers, screenshots, and the migration helper scripts created during the account move.

Setup on a New Machine

Prerequisites

• Node.js 18+ (recommended; an engines pin is a TODO) and npm
• An AWS account + IAM credentials with Rekognition, Textract, Comprehend, S3, and DynamoDB access
• (Optional) AWS CLI v2 for running seed.sh
• sharp builds a native binary on install — on Apple Silicon/Linux this works out of the box with a modern Node; if install fails, ensure build tools are present.

# 1. Clone
git clone https://github.com/<your-account>/video-kyc-hackathon.git
cd video-kyc-hackathon

# 2. Install dependencies
npm install

# 3. Configure environment
cp .env.example .env
#   then edit .env with your own AWS credentials and resource names

# 4. Create the AWS resources (S3 buckets, DynamoDB table, Rekognition collection)
node setup-aws.js

# 5. Start the server
npm start            # or: npm run dev   (nodemon)

New AWS accounts: Textract and Comprehend may return SubscriptionRequiredException until the account finishes activation (payment/identity verification). This is an AWS-side activation state, not a code issue.

Environment Variables

Copy .env.example → .env. Never commit .env.

Variable	Example	Description
AWS_ACCESS_KEY_ID	your_aws_access_key_here	IAM access key
AWS_SECRET_ACCESS_KEY	your_aws_secret_key_here	IAM secret key
AWS_REGION	ap-south-1	AWS region (keep ap-south-1 for residency)
AUDIT_BUCKET	vcip-audit-demo	S3 bucket for liveness/session evidence
DOCS_BUCKET	vcip-docs-demo	S3 bucket for ID document images
DDB_TABLE	vcip_sessions_v2	DynamoDB audit table
REKOGNITION_COLLECTION	vcip-faces	Rekognition face collection id
PORT	9000	HTTP/WebSocket port
KMS_KEY_ID	(optional)	KMS key ARN to encrypt liveness output

S3 bucket names are globally unique — pick names not already taken if you create your own.

Running the App

npm start
# Server: http://localhost:9000
# UI:     http://localhost:9000/web/index.html
# Health: http://localhost:9000/health  ->  {"ok":true,"port":"9000"}

• Merchant flow: open the UI → Start KYC Session → use the join URL.
• Verifier flow: open /web/verifier-dashboard.html.
• API demo: open /web/liveness.html to exercise liveness → ID OCR → face match individually.

Testing the Project

There is no automated test suite yet (TODO: add tests). Verify manually:

• curl localhost:9000/health → {"ok":true,...}
• curl localhost:9000/api/kyc/active-sessions → {"sessions":[]}
• Create a session in the UI, then GET /api/session/<id>/timeline to confirm the SUMMARY artifact.
• Use /web/liveness.html to confirm Rekognition (liveness session) and, once the AWS account is fully activated, Textract (ID analyze).

Demo Data

seed.sh / seed-items.json write synthetic, non-PII demo rows (session SP-DEMO-001, merchant "Acme Pvt Ltd") into the vcip_sessions table:

./seed.sh          # requires AWS CLI v2 configured for ap-south-1

The seed files target the older vcip_sessions table name; the live app uses vcip_sessions_v2. Adjust the table name if you want seed data visible in the running app.

---

Known Limitations

• No automated tests — verification is manual today. (TODO)
• In-memory session state — activeSessions lives in process memory; restarting the server drops live sessions (the DynamoDB audit record persists). Not horizontally scalable as-is.
• Security hardening is opt-in (demo-safe by default). Across four review-driven passes (21 fixed + 5 partial of 32 REVIEW.md findings; 6 open — see its status index) the app now supports: signed role tokens via POST /api/auth/login (AUTH_USERS) with requireAuth/requireRole on /api + Socket.IO (roles enforced server-side); allowlisted CORS; per-IP rate limiting; bounded/validated uploads; safe-ID validation (no S3/DDB key traversal); write-once (WORM) DynamoDB audit artifacts; OCR/face-match audit records; consent capture; geo-validation (India bbox); data-residency enforcement (refuses non-India regions); image downscaling + reused AWS clients; AWS-error→4xx mapping; paginated timelines; and a gated /_routes — all off by default so the demo runs unchanged (see REVIEW.md → Remediation log + status index, and COMPLIANCE_MATRIX.md). Remaining TODO: a DB-backed per-user identity store, S3 Object Lock for full WORM at rest, recorded-video/verifier-identity linkage, Comprehend batching, and API success/error-envelope normalization (deferred — would change response shapes the current UI depends on). Security env vars are documented in .env.example.
• Rekognition Face Liveness client — the production liveness widget requires Amazon Cognito; a mock-upload-selfie path is provided for demos.
• IAM breadth — the app assumes broad service permissions; a least-privilege policy is a TODO.
• [email protected] is end-of-life and has known advisories — plan to upgrade to 2.x.
• No engines pin / .nvmrc — Node version isn't enforced. (TODO)
• New-account AWS activation can gate Textract/Comprehend (SubscriptionRequiredException).

Future Improvements

• Add unit/integration tests and a CI workflow.
• Externalize session state (Redis/DynamoDB) for multi-instance scale.
• Real authentication, RBAC, and verifier audit logging.
• Least-privilege IAM policy + KMS-by-default encryption.
• Production Rekognition Face Liveness (Cognito) integration in the merchant UI.
• Upgrade multer, pin Node, add Docker/IaC for reproducible deploys.

---

AI Product Manager Skills Demonstrated

• Regulatory product translation — turned RBI VCIP requirements into concrete features (liveness, OCR, face match, geo, audit) and an artifact-based audit data model.
• Build-vs-buy judgment — composed managed AWS AI services rather than training models, optimizing for time-to-value and compliance.
• End-to-end thinking — connected frontend, real-time video, backend APIs, AI services, storage, and audit into one coherent flow.
• Data-residency & privacy awareness — region pinning, PII masking, and an audit-first design.
• Pragmatic scoping — clear separation of demo paths vs production gaps, documented as explicit TODOs.

Claude Code / AI-Assisted Development Workflow

This repository's documentation suite and parts of its operational tooling were produced with Claude Code (Anthropic's agentic CLI). See:

• AI_WORKFLOWS.md — AI-assisted product & dev workflows and prompt frameworks.
• PROMPTS_LIBRARY.md — reusable prompts by category.
• DEMO_SCRIPT.md — interview walkthrough.
• INTERVIEW_NOTES.md — AI-PM narrative + interview Q&A.

Honesty note: Sections marked inferred are derived from code conventions, and TODO marks genuine gaps. No feature is claimed that isn't present in server.js / web/.

---

📖 How this was built, in order — both lives of this repo: BUILD-STORY.md

🤖 AI capabilities in this project

This repo carries the densest agentic toolkit in the portfolio (most of it on the vcip-mcp branch):

• Custom MCP server — mcp-server/: the 6-tool "KYC Orchestrator" with client-side PII masking and a three-layer human-in-the-loop gate on the only irreversible write.
• Subagents (code-reviewer, test-writer, security-auditor, aws-cost-auditor), slash commands (/review, /compliance-check, /repo-audit, …), hooks (pre-push secret scan, doc-drift, stop-status) and the vcip-kyc Agent Skill — all under .claude/.
• Cross-vendor harness — the same agents/hooks mirrored for OpenAI Codex (.codex/, AGENTS.md): the method is vendor-portable.
• Prompt library & workflows — PROMPTS_LIBRARY.md, AI_WORKFLOWS_* docs; hardened CLAUDE.md project memory with a session handoff protocol.