← Retour à l’examen

Guide d’étude gratuit

Big Data Analyst — Free Curriculum

This free curriculum covers every domain assessed in the Oxford Certified Big Data Analyst examination: big data platforms, processing, SQL, NoSQL, pipelines, data quality, statistics, machine learning, performance, privacy and storytelling. Read each chapter, note the key points, and then sit the 85-question examination.

Prêt ? Passez l’examen gratuit 105 min · 85 questions
Chapitre 1

What makes data "big"

Big data is commonly described by the "Vs": volume (amount), velocity (speed of generation and processing), variety (structured tables, logs, text, images, sensor data), veracity (trustworthiness) and value (usefulness for decisions). Traditional single-server tools struggle when one or more of these dimensions grows beyond their limits.

The analyst's job is not to handle data for its own sake but to produce reliable insight. Data democratisation gives more people governed, self-service access to trusted data and tools — with security and definitions in place.

  • Volume, velocity, variety, veracity, value
  • Big data needs distributed storage and processing
  • Insight and decisions are the goal
  • Democratisation requires governance, not open access
À retenir: Big data matters only when it leads to better, trusted decisions.
Chapitre 2

Storage architectures: lakes, warehouses and lakehouses

A data lake stores raw data of any format at low cost and applies structure when data is read (schema-on-read). A data warehouse stores cleaned, integrated, structured data optimised for analytical queries (schema-on-write). The lakehouse combines low-cost lake storage with warehouse capabilities such as ACID transactions, schemas and fast SQL.

Dimensional modelling organises warehouse data into star schemas: fact tables hold measurable events (sales amounts, quantities) with keys to dimension tables that describe context (date, customer, product). Slowly changing dimensions of Type 2 keep history by adding a new row with validity dates when an attribute changes. Master data management maintains one consistent version of key entities, and a data catalogue documents datasets, definitions, owners and lineage.

  • Lake = raw, any format, schema-on-read
  • Warehouse = clean, modelled, schema-on-write
  • Lakehouse = both, with transactions and SQL
  • Star schema: facts + dimensions; SCD Type 2 keeps history
  • Catalogue and MDM make data discoverable and consistent
À retenir: Choose the architecture that balances flexibility, performance, cost and governance.
Chapitre 3

Batch and stream processing

Batch processing handles accumulated data at scheduled intervals — nightly reports, monthly billing. Stream processing handles events continuously, enabling reactions within seconds, such as fraud detection or live monitoring. Platforms such as Apache Kafka store events in topics: ordered logs that producers write to and consumers read from.

Streams are unbounded, so aggregations use windows — time-based or count-based segments. Events can arrive late or out of order; event-time processing uses the time the event occurred rather than the time it was processed, so results stay correct. Real-time dashboards are valuable only when someone can act in real time.

  • Batch for scheduled, large-volume work; streams for immediate reactions
  • Kafka topics are ordered event logs
  • Windows make unbounded streams measurable
  • Plan for late and out-of-order events with event time
  • Real time only where action is real time
À retenir: Match processing latency to the speed of the decision it supports.
Chapitre 4

Distributed computing with Hadoop and Spark

HDFS stores large files as blocks replicated across nodes for fault tolerance. Apache Spark processes data in parallel across partitions and keeps intermediate results in memory, making iterative analytics much faster than disk-based MapReduce. Spark uses lazy evaluation: transformations build a plan that runs only when an action (write, count, collect) is called.

Shuffles redistribute data across the network for joins and group-bys and are expensive. Broadcasting a small table to every worker avoids shuffling a large one. Data skew — a few partitions much larger than others — makes some tasks run far longer. Adding nodes does not always help: skew, shuffles and poor file layouts can still dominate run time.

  • Partitions enable parallelism; HDFS replication provides fault tolerance
  • Spark: in-memory and lazy
  • Minimise shuffles; broadcast small lookup tables
  • Detect and fix skew
  • More nodes are not a universal fix
À retenir: Distributed performance depends on data layout and movement, not just cluster size.
Chapitre 5

File formats, partitioning and performance

Columnar formats such as Parquet and ORC read only the required columns and compress similar values efficiently; uncompressed CSV is the least efficient for analytics. Partitioning tables by a commonly filtered column such as date enables partition pruning. Millions of tiny files slow queries; compact them into fewer, appropriately sized files.

In cloud warehouses, cost and speed improve by selecting only needed columns, filtering on partitioned or clustered columns, pre-aggregating data for dashboards (summary tables or materialised views) and caching repeated query results. Approximate algorithms such as HyperLogLog estimate distinct counts on huge datasets quickly with a small, known error.

  • Prefer columnar, compressed formats
  • Partition on common filters; avoid tiny files
  • Avoid SELECT *; filter early
  • Pre-aggregate for dashboards and cache repeated queries
  • Use approximate counts when exactness is not required
À retenir: Efficient layout and focused queries cut both run time and cost.
Chapitre 6

Data pipelines and orchestration

ETL transforms data before loading it into the target; ELT loads raw data first and transforms it inside a powerful platform such as a cloud warehouse. Change data capture (CDC) streams inserts, updates and deletes from source databases. Orchestrators such as Apache Airflow schedule, sequence and monitor pipeline tasks and dependencies.

Robust pipelines are idempotent — re-running them produces the same result — so restarts do not duplicate data; techniques include overwriting partitions and merging on unique keys. Late-arriving data is handled by reprocessing affected partitions. Good pipelines have monitoring, alerting, data-quality checks, automated tests, documentation, clear ownership and lineage.

  • ETL vs ELT
  • CDC keeps downstream data in sync
  • Orchestrate dependencies and monitor runs
  • Design idempotent loads
  • Test, monitor and document every pipeline
À retenir: Reliable pipelines are what make trusted analytics possible.
Chapitre 7

Advanced SQL for analysts

WHERE filters rows before aggregation; HAVING filters groups after aggregation. Window functions — SUM() OVER (PARTITION BY … ORDER BY …), RANK(), ROW_NUMBER(), LAG() — compute running totals, rankings and comparisons without collapsing rows; RANK() leaves gaps after ties. Common table expressions (WITH) break complex queries into readable steps.

Joins need care: a LEFT JOIN keeps all left rows with NULLs where no match exists; joining to a table whose key is not unique causes fan-out and inflated totals. NULL means unknown: use IS NULL, remember that aggregates such as AVG ignore NULLs and that COUNT(column) excludes NULLs while COUNT(*) counts all rows.

  • WHERE before grouping; HAVING after
  • Window functions for running totals and ranks
  • CTEs for readability
  • Check join-key uniqueness to avoid fan-out
  • Handle NULLs deliberately
À retenir: Most wrong numbers come from joins and NULLs — verify them first.
Chapitre 8

NoSQL data stores

Key-value stores provide very fast lookups by key. Document databases store flexible JSON-like documents with nested fields. Wide-column stores handle very high write volumes such as time-series and event data. Graph databases model relationships and are ideal for networks, recommendations and fraud rings.

The CAP theorem states that during a network partition a distributed system must choose between consistency and availability. Different databases make different trade-offs, so choose the store that fits the access pattern.

  • Key-value: fast lookups
  • Document: flexible nested records
  • Wide-column: massive writes and time series
  • Graph: relationships
  • CAP: consistency vs availability during partitions
À retenir: Choose the data store by access pattern, not by fashion.
Chapitre 9

Data quality and governance

Data quality dimensions include completeness, accuracy, consistency, timeliness, validity and uniqueness. Reconciling row counts and totals against source systems detects missing or duplicated loads. Data owners are accountable for definitions, quality and access decisions; stewards maintain them day to day.

Conflicting definitions — for example different "active customer" counts in marketing and finance — are resolved by agreeing a governed definition, documenting it in the catalogue and certifying one dataset for reporting. Lineage shows where data came from and how it was transformed.

  • Measure quality dimensions explicitly
  • Reconcile loads with sources
  • Assign owners and stewards
  • One governed definition per key metric
  • Track lineage end to end
À retenir: Trust in data is earned through visible ownership, definitions and checks.
Chapitre 10

Statistics and experimentation

The median resists outliers better than the mean; standard deviation measures spread. A p-value below the chosen significance level (for example 0.05) indicates that the observed data would be unlikely if the null hypothesis were true — it does not measure effect size or importance. A 95% confidence interval means that, using the same method repeatedly, about 95% of such intervals would contain the true value.

Correlation does not prove causation. Simpson's paradox can reverse trends when groups are combined; survivorship bias ignores cases that dropped out. Larger samples reduce random error and increase power. In A/B tests, fix sample size and duration in advance or use a proper sequential method — repeatedly "peeking" and stopping at the first significant result inflates false positives. Sampling is legitimate when it is representative and the error is acceptable.

  • Median for skewed data; standard deviation for spread
  • Understand what p-values and confidence intervals do and do not say
  • Correlation ≠ causation; watch for Simpson's paradox and survivorship bias
  • Plan experiments; do not peek and stop early
  • Representative samples can answer questions efficiently
À retenir: Good statistics protect decisions from convincing-looking noise.
Chapitre 11

Machine learning for analysts

Features are input variables; feature engineering (for example "days since last purchase") often improves models significantly. Data leakage occurs when training uses information unavailable at prediction time and produces unrealistically good results. Evaluate models on hold-out data or with cross-validation, choose metrics that reflect business costs, and check performance over time and across segments.

Regression problems use metrics such as MAE or RMSE; classification uses precision, recall and related measures. Clustering such as k-means groups similar records without labels, for example for customer segmentation.

  • Engineer meaningful features
  • Prevent leakage
  • Evaluate on unseen data and over time
  • Pick metrics that match the problem
  • Use clustering for segmentation
À retenir: A model is only as good as its evaluation is honest.
Chapitre 12

Privacy and responsible analytics

Personal data must be protected throughout analytics. GDPR principles include purpose limitation, data minimisation and storage limitation. Pseudonymisation replaces direct identifiers with codes, but combining datasets can still re-identify individuals, so anonymisation must be assessed carefully.

Protect data with role-based access, column-level masking, minimisation, retention limits and secure environments — never copy production customer data to personal laptops.

  • Purpose limitation and minimisation
  • Pseudonymised ≠ anonymous
  • Beware re-identification when joining datasets
  • Mask, restrict and delete what is not needed
À retenir: Privacy protection is part of analytical quality.
Chapitre 13

Visualisation and data storytelling

Choose visuals by purpose: histograms for distributions, line charts for trends, scatter plots for relationships, waterfall charts for how a total is built from contributions, heat maps for intensity across two dimensions. Trustworthy dashboards show data freshness, definitions and sources and use consistent metrics.

Effective storytelling starts with the key message and recommendation, supported by a few focused visuals. When an executive asks why sales fell, explain the main drivers clearly rather than sending dozens of charts.

  • Match the chart to the question
  • Show freshness, definitions and sources
  • Lead with the message and recommendation
  • Fewer, focused visuals
À retenir: Insight that is not understood does not change decisions.

Prêt ? Passez l’examen gratuit

Passez l’examen et obtenez gratuitement votre badge numérique. Après la réussite, vous pourrez commander le certificat officiel.

Commencer l’examen gratuit