Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 61 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,76 @@
# MPATE-UE-1113 — Music Technology Coursework
# MPATE-UE-1113 Final Project — Does AI-Generated Cantopop Respect Cantonese Tone?
by Candy Xie

My personal work for NYU's *Music Technology* course (Spring 2026). Python-based audio analysis, music information retrieval, and clustering — built around `librosa`, `music21`, `numpy`, and `scikit-learn`.
A tone-tune mapping audit comparing **human-sung Cantopop** against **Suno-generated Cantopop** to test whether AI music generation respects Cantonese's six-tone system.

## Question

Cantonese is a tone language: pitch direction between adjacent syllables has to roughly match the lexical tonal direction, or the lyric becomes unintelligible. Wong & Diehl (2002) showed human Cantopop respects this rule **75–92%** of the time. **Does Suno?**

## Method

1. **Citation tones** — pick one 7–14 syllable snippet per track (no 變調 sandhi, no English loanwords, no proper nouns). Hand-annotate each character's Cantonese tone (1–6) and reduce to a 3-level target (High / Mid / Low) following Wong & Diehl's tone-ending mapping.
2. **F0 extraction** — auto-segment the snippet into syllables (voiced-region detection on Parselmouth's pitch contour), then use [Parselmouth](https://parselmouth.readthedocs.io/) (Python wrapper for Praat) to compute the median F0 per syllable.
3. **Direction comparison** — for every adjacent syllable pair, compare *expected* tonal direction (up/flat/down) against *actual* F0 direction. Count violations.
4. **Validate** — confirm the human songs land in Wong & Diehl's 75–92% band; read off Suno's match rate against the same baseline.

## Dataset

Three human/Suno pairs matched by tempo. Audio in `assignments/06-final-project/audio/`.

| Tempo | Human (track, artist) | Suno (generated, V5.5) |
| -------- | ---------------------------------- | ----------------------------------- |
| Mid | 隔離 — Jace Chan (2023) | 玻璃 |
| Ballad | 高山低谷 — Phil Lam 林奕匡 (2014) | 雨窗一封 |
| Uptempo | 紅日 — Hacken Lee 李克勤 (1992) | 旺角快車 *(mis-rendered, see note)* |

> **Note on the uptempo Suno track.** The track was prompted as 旺角快車 but Suno hallucinated lyrics from a different prompt (雨窗一封). The audit scores what Suno actually sang — itself an instructive failure mode.

## Files

- `assignments/06-final-project/cantonese_tone_audit.py` — core framework: tone mapping, F0 extraction, violation counting
- `assignments/06-final-project/audit_driver.py` — runs the audit across all 6 tracks, emits the comparison figure
- `assignments/06-final-project/audio/` — 3 human + 3 Suno mp3s
- `assignments/06-final-project/figures/human_vs_suno.png` — bar chart of match rate vs Wong & Diehl baseline

## Run

```bash
pip install praat-parselmouth librosa numpy pandas matplotlib
cd assignments/06-final-project
python audit_driver.py
```

Outputs:
- `figures/human_vs_suno.png` — bar chart of match rate per tempo bucket vs the Wong & Diehl baseline band
- per-snippet expected-direction tables and pair-by-pair violation reports to stdout
- top Suno violations grouped by track (b-roll material for the project video)

The CJK glyphs in matplotlib labels rely on a system CJK font (PingFang / Heiti / Hiragino — all preinstalled on macOS). On Linux, install `fonts-noto-cjk` and add `'Noto Sans CJK TC'` to `matplotlib.rcParams['font.family']` near the top of `audit_driver.py`.

## Findings

See `figures/human_vs_suno.png` and the violation tables in the audit's stdout. Discussion and concrete failure pairs are walked through in the accompanying video.

---

# MPATE-UE-1113 Coursework

My personal work for NYU's *Music, Mind and Artificial Intelligence* course (Spring 2026). Python-based audio analysis, music information retrieval, and clustering — built around `librosa`, `music21`, `numpy`, and `scikit-learn`.

## Contents

- **`labs/`** — Lab notebooks
- `01-python-basics` · `02-hello` · `03-leap-year` — warm-ups
- `04-sine-wave` · `05-sine-square-saw` · `06-play-audio` — synthesis & playback
- `07-librosa-tempo` — tempo / beat tracking with librosa
- **`assignments/`** — Five graded assignments
- **`assignments/`** — Graded assignments
- `01-written-1+2` — written responses
- `02-librosa` — audio feature extraction
- `03-clustering` — clustering songs by audio features
- `04-playlisting` — playlist generation
- `05-project-2` — final project
- `05-project-2` — project 2
- `06-final-project` — final project (Cantonese tone audit, see top of README)
- **`demos/`** — Side explorations (drum clustering, music21 analysis)
- **`notes/`** — Personal study notes (sine-wave fundamentals)
- **`misc/`** — Tempo-extraction tutorial I wrote up while reviewing for an assignment
Expand Down
Binary file not shown.
Binary file added assignments/06-final-project/audio/gou-san.mp3
Binary file not shown.
Binary file not shown.
Binary file added assignments/06-final-project/audio/hong-jat.mp3
Binary file not shown.
Binary file not shown.
Binary file added assignments/06-final-project/audio/jace_gelei.mp3
Binary file not shown.
Loading