AI Training Bot (AITB) — System Card

Owner: Tiny Codes Software Pte Ltd (UEN 202405170H) · For: Ministry of Social and Family Development (MSF)
Version: 0.3 (living document) · Status: an AI Verify (Generative AI) self-assessment, Project Moonshot technical benchmarks, and adversarial red-teaming were completed on 01 July 2026 (see para 7). Sections marked Pending (Phase 4) are validated against the built system during Testing & AI Assurance.

This system card communicates, in one place, the key facts about the AITB AI system for stakeholders (MSF as deployer, end-user officers, and relevant authorities). It complements the AI Use Disclosure and the Privacy Policy, and supports the AITB AI Verify self-assessment.

1. Intended use

Purpose: an internal training tool for MSF frontline officers to practise difficult client conversations with an AI avatar, and to receive transcript-anchored feedback.
Users: MSF frontline officers (trainees) and administrators (who configure scenarios, personas and assessment frameworks).
Out of scope: AITB makes no real decisions about real members of the public; it is not used for eligibility, casework, employment or any operational decision. It operates on Official Open / synthetic data only — no real client personal data.

2. How it works (components & models)

AITB orchestrates third-party foundation models accessed via API; Tiny Codes does not train its own models.

Function	Model / service
Speech understanding, turn-taking, in-character persona reasoning	OpenAI Realtime (`gpt-realtime-2`), text-output mode
Avatar voice (per-persona Singaporean-accented)	ElevenLabs (`eleven_v3`)
Post-session assessment	A reasoning-capable LLM (GPT-5-class)

GenAI inference runs on the OpenAI API with Southeast Asia data residency; persistent data is stored in Singapore.

Architecture & mechanics. The realtime speech model runs in text-output mode — it understands the officer and reasons in character but does not speak; its reply text is voiced by the text-to-speech service (a hybrid "brain/ears + mouth" split). The avatar's emotional state is carried via a structured side-channel parsed by the application. Deterministic rule-based application logic enforces guardrails, scenario boundaries, the 10-minute session limit, and binds the assessment to the configured rubric. AITB therefore combines foundation models with rule-based application logic rather than relying on a single model. No model is trained or fine-tuned by Tiny Codes.

3. Data & provenance

Training content (scenarios, personas) is synthetic / Official Open — fictional, no real client PII.
AITB does not use session data to train or fine-tune any model, and does not reuse it for external analytics without explicit MSF approval.
Foundation-model training data is the respective providers' (OpenAI, ElevenLabs); see their published model documentation. Tiny Codes does not control or have visibility into that corpus.
Generated data: session voice recordings, transcripts, emotion signals and assessment outputs. Voice recordings are retained for two purposes — (i) human review of training interactions and (ii) speech-to-text (STT) accuracy verification (checking transcript accuracy against the source audio). They are treated as sensitive operational data: encrypted in transit and at rest, access-controlled (RBAC, least privilege), stored in Singapore (AWS ap-southeast-1), PDPA-handled, and subject to a configurable retention/deletion policy (default 90 days, MSF-configurable per data class via the RetentionPolicy; secure deletion / crypto-erasure). Retained audio is not used to train or fine-tune any model, or for external analytics, without explicit MSF approval.

4. Performance & accuracy

Targets: ≥90% transcript/language accuracy per official language (English, Mandarin, Malay, Tamil); end-to-end speech-to-speech response under 3s for 95% of interactions.
Confidence handling: confidence-aware speech recognition with clarification/recovery when interpretation confidence is low.
STT accuracy verification: retained voice recordings are used to verify transcript/speech-to-text accuracy against the ≥90% per-language target and to support human review of assessments; recordings are the evidentiary source for checking transcription accuracy.
Initial model-level benchmarks (Project Moonshot, 01 Jul 2026): general capability mmlu 91.3; Singapore-context factual accuracy singapore-facts-mcq 100, singapore-facts-tf 66.67 (moderate — see note). Privacy-safety benchmark mlc-prv-en 100.
Note on singapore-facts-tf: the base model's Singapore-specific true/false factual accuracy is only moderate. This has low impact here — AITB role-plays fictional personas and produces transcript-anchored communication assessment; it is not a factual Q&A system and does not rely on the model's world-knowledge for scoring. It is disclosed for awareness and informs scenario grounding (the avatar should not be relied on for factual claims).
End-to-end measured results: Pending (Phase 4) — per-language transcript/language accuracy (≥90% target), latency and concurrency are measured against the built system during Testing & AI Assurance and UAT; the benchmarks above are model-level and do not substitute for this.

5. Limitations

Speech-recognition and per-language quality can vary; lower-resource languages (Malay, Tamil) are typically weaker than English/Mandarin.
The avatar is a simulation, not a real person, and can occasionally be inconsistent or imperfect.
Assessment outputs are advisory indicative signals, not determinations.

6. Risk assessment & mitigations

Security: a Cloud Security Risk Assessment (CSRA) and required security clearances are a pre-condition to deployment; encryption in transit/at rest, RBAC, audit logging, PDPA compliance. See the Security & Data Protection approach.
AI guardrails: content filtering, scenario-boundary enforcement, prompt-injection defence, and bias monitoring across the four languages (see the bias/fairness section of the AI Use Disclosure).
Human oversight: AI output is advisory; a human remains responsible. No consequential automated decisions are made about people.

7. Assessment, red-teaming & safety testing

AI Verify (Generative AI) self-assessment — completed 01 Jul 2026. Using the AI Verify Foundation's Process Checks Tool, AITB was assessed against 104 process checks: 86 Yes, 6 No, 12 Not Applicable (most N/A because Tiny Codes does not train/fine-tune models). The 6 "No" items are honest pre-deployment gaps — staff training records, external-actor feedback exercises, and deployment-phase hardening (blue-green/canary) — none material to the advisory use case. The Summary Report is shared with MSF.
Technical benchmarks (Project Moonshot) — 01 Jul 2026. Privacy-safety mlc-prv-en 100; capability mmlu 91.3; singapore-facts-mcq 100; singapore-facts-tf 66.67 (see para 4).
Adversarial red-teaming (Project Moonshot) — 01 Jul 2026. Seven attack modules (obfuscation/evasion, group-directed and Singapore-context toxicity, illicit-instruction and multi-turn jailbreak personas) were run against the model endpoint; every adversarial variant was refused or safely deflected — no jailbreak or harmful-content bypass. AITB's application guardrails add further protection on top of the model.
Still scheduled (Phase 4): end-to-end per-language accuracy/latency validation, security/penetration testing (a pre-condition to deployment), and extended/periodic red-teaming. Results are added here as they complete.

8. Governance & documentation

AI Use Disclosure · Privacy Policy · internal Transparency & User Communication Policy and Data Protection Policy (available to MSF)
AI Verify (Generative AI) self-assessment — completed 01 Jul 2026 using the AI Verify Foundation's Process Checks Tool (104 checks: 86 Yes / 6 No / 12 N/A); the Summary Report (with the Project Moonshot technical benchmarks and red-teaming) is shared with MSF.
Technical documentation, user/administrator guides and admin training are delivered as part of the engagement.

9. Version history

Version	Date	Notes
0.1	Bid stage	Initial system card; quantitative and red-teaming sections pending Phase-4 validation.
0.2	01 Jul 2026	Added AI Verify self-assessment outcome (84/8/12) and initial Project Moonshot technical-test results; end-to-end accuracy/latency and broader red-teaming remain Phase-4.
0.3	01 Jul 2026	Adversarial red-teaming completed (7 modules, no bypass); checks 4.1.1 & 7.2.1 upgraded to Yes → revised tally 86/6/12; end-to-end accuracy/latency and pen-testing remain Phase-4.