Description
Data Infrastructure and Technology is the capability to make sound decisions about how data is collected, stored, processed, governed and fed into AI systems — and to see why those choices set the ceiling on what any model can do. In practice it shows up as matching workloads to the right storage, judging data quality and bias before a model is trained, separating training from inference needs, designing batch or streaming pipelines, running disciplined experiments, writing data contracts, monitoring drift, and leading incident response and compliance when things go wrong. For executives shaping AI strategy, it grounds investment, risk and expectation-setting in data reality. The capability deepens through repeated exposure to real pipelines, failures, reviews and iteration rather than through study alone.
Stacks
AWSAzureGoogleSMACKELK
Expected Behaviors
✎
LEVEL 1
Fundamental Awareness
In executive briefings and AI strategy discussions, explains why data quality and scale set the ceiling on what AI systems can deliver, names common data defects such as bias, noise, staleness and privacy exposure, and traces real AI failures back to their data causes. Contrasts data-centric and model-centric approaches, outlines Responsible AI principles and accountability mechanisms, distinguishes CAIO, CDO, CAO and CTO mandates and organizational models, and frames realistic stakeholder expectations and cost implications.
🌱
LEVEL 2
Novice
Working on defined AI use cases with guidance, matches workloads to appropriate storage and acquisition methods, plans training and inference data collection separately, and checks sources against privacy, licensing and fairness requirements. Profiles datasets, measures quality dimensions against thresholds, cleanses data and handles missing values reproducibly, engineers and scales features, splits and samples data correctly, addresses class imbalance, organizes labeling with quality controls, and aligns feature availability between training and serving.
🌍
LEVEL 3
Intermediate
Owns delivery of data and experimentation platforms for AI products. Weighs batch against streaming on latency, throughput and cost, builds orchestrated ELT pipelines and real-time streaming systems with state management and schema evolution, and designs inference architectures across batch, edge and hybrid patterns. Runs controlled experiments end to end with sound metrics, infrastructure, statistical analysis and governance trails, and applies data mesh, data product and data contract practices across domains.
⭐
LEVEL 4
Advanced
Directs enterprise AI and generative AI data practice. Chooses between pre-training, fine-tuning and retrieval approaches, sets curation standards for large-scale corpora, and judges legal, ethical, quality and cost risk in training data. Governs fine-tuning and RAG dataset creation, approves synthetic data and mixing ratios, and establishes continuous validation, drift detection, experiment tracking and monitoring with automated retraining. Reviews pipelines, runs shadow deployments, and leads bias mitigation and data-as-a-product adoption.
🏆
LEVEL 5
Expert
Sets organization-wide direction for data observability, incidents and governance in AI. Defines observability and incident taxonomies, owns response frameworks, root cause standards, prevention architecture and model rollback strategy. Establishes ownership and stewardship models, enterprise data quality frameworks, catalog, metadata and lineage standards, and carries accountability for GDPR, CCPA, HIPAA and PCI DSS compliance, privacy-preserving and access control policy, retention and consent, and the overall AI data strategy.
Micro Skills
✎
LEVEL 1
Fundamental Awareness
Explain how data quality and scale determine an AI system's capabilities and limitations ("garbage in, garbage out")
Describe the categories of data problems (incompleteness, bias, noise, staleness, contamination, privacy) and their impact on AI
Identify the data problem behind real-world AI failures
Explain data-centric AI and how it differs from model-centric AI (systematic quality improvement, curation, targeted collection, iterative refinement)
Describe strategic data practices (documentation, bias detection, representative sampling, synthetic data for rare scenarios, continuous monitoring)
Describe the core responsibilities of a Chief AI Officer and the organizational, technical and business challenges of the role
Compare the mandates of the CAIO, CDO, CAO and CTO and how their core responsibilities differ
Describe how data governance responsibilities divide between the CAIO and CDO (quality standards, access policies, training data, model lineage, compliance, ethics)
Compare centralized, federated, decentralized and hub-and-spoke AI organizational structures and their advantages and disadvantages
Identify the success factors for the CAIO role (strategic alignment, executive sponsorship, value orientation, ethical framework, change management)
Define Responsible AI, its core principles (ethics, fairness, reliability, privacy, security) and their business impact
Identify sample, measurement, algorithm and deployment bias and a mitigation for each
Describe transparency and accountability mechanisms (explainability, model cards, human oversight, auditability, redress)
List the Responsible AI implementation steps (assessment, design, testing, monitoring, governance) and explain how they mitigate bias and increase transparency
Describe the components of a data ecosystem and the layers of the modern data stack (ingestion, storage, processing, analytics, governance, orchestration)
Compare legacy and modern data stacks on scalability, security, observability, manageability, integration and cost model
Describe the costs, risks and rewards of data economies and how data costs affect the ROI of AI initiatives
Explain how data limitations should shape stakeholder expectations of AI capabilities
Identify the data causes and crisis-management lessons in an AI ethics incident case study
🌱
LEVEL 2
Novice
Match a workload to the right storage system (database, data warehouse, data mart or data lake), including OLTP versus OLAP needs
Choose object, block, file, columnar or key-value storage for a use case by access method, scalability, performance and cost
Classify candidate data sources as internal, external or third-party and note their trade-offs in control, cost and scope
Apply the data collection framework (objectives, source selection, collection methods, ethics and compliance, monitoring) to plan collection for an AI use case
Plan training and inference data collection separately (volume, diversity, labeling, freshness, collection frequency)
Use data acquisition methods (APIs, web scraping, forms, sensors and IoT, data partnerships, batch transfers) suited to budget and data velocity
Check data collection against privacy, data ownership and licensing, security, fairness and transparency requirements
Measure data quality dimensions (accuracy, completeness, consistency, timeliness, validity, uniqueness, integrity, reasonableness, relevance, representativeness, interpretability) with appropriate methods
Apply training-data quality checks (representativeness, class balance, label accuracy, temporal consistency) and inference-data checks (low-latency validation, schema enforcement, drift detection, confidence scoring)
Apply data cleansing techniques (deduplication, outlier handling, error correction, standardization, structural fixes) in a reproducible, documented way that preserves lineage
Handle missing values by choosing deletion, imputation, prediction models or flagging based on why, how much and in what pattern data is missing
Apply feature engineering techniques (aggregation, decomposition, interaction features, binning, encoding, text features, derived metrics)
Apply scaling and normalization (min-max, z-score, robust scaling, log transform) suited to the data distribution
Profile a dataset using column, cross-column, structure, content and time-series analysis and visual techniques (histograms, correlation matrices, missing-value maps)
Use anomaly detection methods (statistical, clustering, classification, rule-based, time-series) to flag outliers
Record storage requirements for training (capacity, throughput, cold storage, columnar formats, dataset versioning) and inference (latency, availability, caching, consistency)
Track data quality metrics against agreed thresholds and data quality SLAs
Score an organization's data maturity against a framework's dimensions using multi-source evidence
Split a dataset into training, validation and test sets with ratios suited to its size, keeping the test set untouched until final evaluation
Use the validation set for hyperparameter tuning, early stopping and model selection
Apply random, stratified or time-based sampling appropriate to the data
Use k-fold cross-validation to evaluate models on small datasets
Detect class imbalance and evaluate models with precision, recall, F1 and AUC-ROC instead of accuracy
Apply class-imbalance remedies (stratified sampling, cost-sensitive learning, oversampling, SMOTE, ADASYN, undersampling)
Prepare labeling semantics, instructions, tools and quality thresholds before labeling begins
Choose manual, semi-automated or automated labeling for a task based on cost, quality and scalability
Use human-in-the-loop labeling methods (active learning, weak supervision, interactive labeling)
Apply labeling quality controls (multiple annotators, gold-standard testing, inter-annotator agreement, statistical checks, audit trails)
Generate synthetic data with rule-based, statistical or deep-learning methods to supplement real data for rare scenarios
Check that training and inference data requirements align (distribution, labels, latency, compute resources, data quality, feature availability)
Fix feature availability problems at inference time (future leakage, external dependencies, expensive features, privacy restrictions)
Apply inference latency techniques (model optimization, caching, batching, edge deployment, asynchronous processing)
Collect production feedback (explicit, implicit, delayed ground truth, A/B tests) for model retraining
Apply feature engineering and feature selection, checking the effect on performance, efficiency and interpretability
Use a feature store to keep feature definitions consistent between training and inference
🌍
LEVEL 3
Intermediate
Evaluate batch versus stream processing for a use case on latency, throughput and cost
Design training and inference processing architectures using batch, real-time, edge or hybrid inference patterns
Build scheduled batch pipelines with processing and orchestration tools (Apache Spark, Apache Airflow, dbt) using ELT patterns
Build real-time pipelines with stream processing frameworks (Apache Kafka, Apache Flink)
Design a real-time data processing system (ingestion, stream processing, state management, serving) using event sourcing, CQRS, windowing, exactly-once processing and schema evolution
Design an A/B test with randomization, control groups, sample size and duration, guarding against selection bias and confounders
Define experiment success metrics and the data collection needed to measure them accurately
Implement experimentation infrastructure (feature flags, consistent traffic assignment, real-time event tracking)
Analyze experiment results with significance tests, confidence intervals and effect sizes, correcting for multiple comparisons and sequential testing
Evaluate experiment tracking and management platforms for integration, scalability, statistical rigor, governance and privacy compliance
Implement experiment governance (approval workflows, review processes, decision logging, audit trails)
Analyze how data mesh principles (domain ownership, data as a product, self-serve infrastructure, federated governance) would change a centralized data architecture
Design a data product that is discoverable, addressable, self-describing, secure and trustworthy, with metadata, APIs, documentation and SLAs
Evaluate data fabric components and how a data fabric complements a data mesh
Write a data contract specifying schema, business semantics, quality guarantees, freshness SLAs, ownership, security classes and versioning policy
Implement data contract enforcement through version-controlled definitions, CI validation, runtime monitoring and change governance
Analyze proposed schema changes for breaking impact under a data contract's versioning policy
⭐
LEVEL 4
Advanced
Select the data approach for a generative AI use case: pre-training, task-specific or instruction fine-tuning, or retrieval-augmented generation
Establish processing standards for foundation-model training data (deduplication, quality filtering, privacy scrubbing, tokenization, validation)
Review large-scale training data for legal and ethical risk (copyright, consent, fair use), quality (misinformation, bias, toxicity), infrastructure cost and freshness
Lead creation of fine-tuning datasets (human annotation, automated generation, augmentation, active learning) to diversity, consistency, balance, validation and iteration standards
Establish the data preparation process for RAG knowledge bases (document processing, chunking, metadata enrichment, quality control, index optimization)
Approve synthetic data with a quality assurance framework (fidelity, diversity, safety checks, utility validation) while keeping validation sets real
Manage synthetic-to-real mixing ratios and guard against model collapse from low-quality synthetic data
Establish continuous data validation (schema, statistical, completeness, consistency checks) across the ML pipeline
Establish drift detection for deployed models (covariate, prior probability, concept and prediction drift) using statistical tests, distance metrics and model-based methods
Lead the response to data and concept drift, from alert thresholds and escalation to retraining
Establish experiment tracking standards (code and data versions, hyperparameters, metrics, artifacts, environment) for reproducibility and model selection
Direct model monitoring and feedback loops across performance, data quality, system health and business impact, with automated retraining, traffic routing, data collection and feature updates
Review training-phase data pipelines for idempotency, scalability, monitoring and reproducibility
Review inference-phase data pipelines for training-serving skew, feature freshness against cost, and fallback strategies
Run shadow deployments to validate new data pipelines and models against production before rollout
Lead the identification and mitigation of bias across the AI data pipeline
Optimize generative model inference for performance and cost-efficiency
Guide prompt engineering practice and evaluate its effect on model behavior
Lead teams in treating data as a product and enforcing data contracts across teams
🏆
LEVEL 5
Expert
Define the organization's data observability strategy across freshness, quality, volume, schema and lineage, beyond monitoring known metrics
Architect observable data pipelines with comprehensive logging, stage health checks, circuit breakers and data contracts
Set the observability implementation approach for AI systems (automated profiling, anomaly detection, smart alerting, dynamic thresholds)
Define the organization's taxonomy of data incidents and AI-related failures (confidentiality, integrity, availability; data quality, pipeline, schema, drift, bias and fairness, integration)
Own the data incident response framework (detection, assessment, escalation, containment, resolution, documentation) and its communication protocols (severity levels, escalation matrix, status updates, blameless reviews)
Set the standard for root cause analysis (evidence gathering, timeline analysis, 5 Whys, impact assessment, action items) and cross-functional problem solving
Architect incident prevention through proactive monitoring, data contracts, automated testing, quality gates and resilient design (redundancy, graceful degradation, regular audits)
Define the model rollback strategy (blue/green, A/B testing, canary releases), with immediate containment for production incidents and learning for training incidents
Define data ownership and stewardship with a RACI model, escalation paths, KPIs and cross-functional committee structures
Set the enterprise data quality management framework (accuracy, completeness, consistency, timeliness, validity, uniqueness; profiling, scorecards, cleansing, preventive controls)
Own the data catalog and metadata strategy (asset discovery, business glossary, rich metadata, classification) and its success metrics
Set the data lineage standard (origin, transformations, movements, destinations) for impact analysis and trust
Own regulatory compliance for AI data across GDPR, CCPA, HIPAA and PCI DSS
Define privacy-preserving and access control policy (anonymization, differential privacy, masking and tokenization, RBAC, ABAC, zero trust)
Set data classification, retention and consent management policies
Advise on continuous experimentation practices that optimize model performance safely
Define a data strategy framework for an AI application (executive summary, problem definition, framework, implementation advice) covering data requirements, quality and bias, privacy and security, compliance and acquisition