# JavaScript interfaces

Two classes do the work: `KokoroInnoTuner` (reference clip → voice pack) and
`KokoroTTS` (text + pack → audio). Everything else in `js/` is a supporting
module you can use on its own.

```js
import { KokoroInnoTuner, KokoroTTS } from './js/index.js';
```

All modules are plain ES modules with no bundler. The only third-party code
comes from CDNs (`@huggingface/transformers`, `phonemizer`,
`onnxruntime-web`); `KokoroInnoTuner` accepts an injected `ort` so a page that
already loaded transformers.js can share one WASM runtime.

---

## Voice pack format

A Kokoro voice is a flat `float32` tensor of shape `[510, 1, 256]`
(522,240 bytes):

| axis | meaning |
|---|---|
| 510 | row index = number of phonemes in the utterance (clamped to 0…509) |
| 256 | style vector: `[0:128]` timbre, `[128:256]` prosody |

Kokoro selects `pack[phoneme_count]`; the tuner repeats the timbre half on all
rows and varies the prosody half by row. This is exactly the layout of
`hexgrad/Kokoro-82M/voices/*.pt` and `onnx-community/Kokoro-82M-v1.0-ONNX`'s
`voices/*.bin`, so exported packs are drop-in for either.

Helpers: `packToVoiceBin(pack)` (identity, documented intent) and
`voiceBinToPack(arrayBuffer)` (validates the length, returns `Float32Array`).

---

## `KokoroInnoTuner`

### `KokoroInnoTuner.load(options?) → Promise<KokoroInnoTuner>`

| option | default | meaning |
|---|---|---|
| `weightsUrl` | Hub `model.safetensors` | tuner weights (24 MB) |
| `weightsBuffer` | — | use an already-loaded `ArrayBuffer`/`Uint8Array` |
| `encoderUrl` | `./models/speaker_encoder.onnx` | speaker encoder graph |
| `encoderBuffer` | — | same, preloaded |
| `ort` | CDN `ort.wasm.min.mjs` | onnxruntime module namespace to reuse |
| `onProgress` | — | `({phase, loaded, total, ratio})` where phase is `weights` \| `encoder` \| `ready` |

Only the tensors the tuner needs are decoded; the 7.4 M encoder parameters are
skipped because they live in the ONNX file.

### `tuner.enroll(input) → Promise<{pack, weights, report}>`

| input | meaning |
|---|---|
| `pcm` | mono `Float32Array` in −1…1 |
| `sampleRate` | rate of `pcm` (any rate; resampled internally) |
| `prosodyHead` | `true` to apply the learned prosody delta (default `true`) |
| `f0Ceiling` | override the automatic pitch-tracking ceiling in Hz |
| `onProgress` | `(phase) => void`, phases: `features`, `embedding`, `prosody`, `blend`, `head`, `pack`, `done` |

Throws if the clip is shorter than 3 s. Only the first 30 s are used.

`report`:

```js
{
  duration, analysed,            // seconds; analysed = min(duration, 30)
  sampleRate,
  f0MeanSt, f0SdSt, voicedFraction,   // pitch statistics
  syllablesPerSecond, tiltDbPerOct, tiltZ, tiltWarning,
  f0Ceiling,                     // Hz used by the pitch tracker
  prosodyHead, headStats,        // [mean, sd] fed to the head, or null
  weights,                       // { "am_michael": 0.61, ... } stock packs blended
  embeddingNorm,
}
```

`weights` is the NNLS mixture over the 28 stock packs; it is useful for
explaining why a voice sounds the way it does.

### Standalone use

```js
const tuner = await KokoroInnoTuner.load({ onProgress: console.log });
const { pack, report } = await tuner.enroll({ pcm, sampleRate });
const bin = packToVoiceBin(pack);            // Float32Array(130560)
```

---

## `KokoroTTS`

### `KokoroTTS.load(options?) → Promise<KokoroTTS>`

| option | default | meaning |
|---|---|---|
| `transformers` | CDN `transformers.min.js` | module namespace (pass the npm build in Node) |
| `modelId` | `onnx-community/Kokoro-82M-v1.0-ONNX` | ONNX repo |
| `dtype` | `"q8"` | `"q8"` is the CPU sweet spot; `"fp32"`/`"q4"` also work |
| `device` | `"wasm"` in browsers, `"cpu"` in Node | transformers.js device |
| `onProgress` | — | transformers.js progress events |

### `tts.generate(text, {pack, speed, language, onChunk}) → Promise<{audio, sampleRate, phonemes}>`

* `pack` — `Float32Array(510 * 256)`, validated.
* `speed` — `1` default (0.6–1.5 is the useful range).
* `language` — `"a"` American English (default), `"b"` British.
* `onChunk({index, total, phonemes})` fires per sentence.

Text is sentence-split, phonemized with eSpeak, and each chunk longer than 480
phonemes is split on word boundaries. Chunks are concatenated with 120 ms of
silence. Returns mono `Float32Array` audio at 24,000 Hz.

### `tts.speakPhonemes(phonemes, {pack, speed}) → Promise<Float32Array>`

Lower-level: one phoneme string, one ONNX run. Useful for streaming or for
feeding your own phonemes.

### `tts.encode(phonemes, pack) → {input_ids, style, row}`

Exposes the style-row selection (`row = input_ids.length - 2`, clamped) for
debugging or for driving the ONNX session yourself.

### Utilities

* `audioToWav(audio, sampleRate)` → `Blob` (`audio/wav`), handy for `<audio>`
  or a download.
* `splitSentences(text)` / `splitPhonemes(phonemes, limit)`.

---

## `phonemize(text, language, {norm})`

Text → eSpeak phone string, including the normalisation rules Kokoro was
trained with (numbers, money, abbreviations, punctuation, `r → ɹ`, `x → k`, …).

```js
import { phonemize } from './js/index.js';
await phonemize("It's 5:30 and $12.50, Dr. Smith.", 'a');
```

In Node (or to reuse an existing eSpeak build) inject the implementation:

```js
import { phonemize as espeak } from 'phonemizer';
import { setPhonemizer } from './js/index.js';
setPhonemizer(espeak);
```

---

## Supporting modules

| module | exports | notes |
|---|---|---|
| `dsp.js` | `fft`, `rfftPower`, `irfft`, `resamplePoly`, `quantile`, `hammingWindow` | no browser APIs |
| `fbank.js` | `kaldiFbank(pcm16k)` | `{data, frames}` mel-major |
| `prosody.js` | `stats`, `rate`, `tilt`, `ceiling`, `headStats`, `trackF0` | pitch in semitones re 100 Hz |
| `nnls.js` | `nnls(A, m, n, b)` | Lawson-Hanson |
| `safetensors.js` | `parseSafetensors(buffer, include?)` | F32/F16/BF16/F64 → Float32Array |
| `speaker-encoder.js` | `SpeakerEncoder.load({modelUrl\|modelBuffer, ort})`, `.embed(features, frames)` | ONNX I/O `features → embedding` |
| `audio.js` | `decodeAudioBlob`, `startRecording`, `downloadBlob`, `formatDuration` | browser only |

---

## Error handling

All async methods reject with `Error` and human-readable messages: short clip,
wrong pack size, missing tensor, HTTP failure, unavailable microphone. The page
shows `error.message` in the status bar.

## Notes for integrators

* Run the heavy calls from a user gesture (button click): browsers require it
  for `AudioContext`, and it lets you show progress immediately.
* Models are cached by the browser (HTTP cache / Cache Storage), so repeated
  loads are cheap.
* The tuner and Kokoro both need WASM; there is no WebGPU path.
* If you use transformers.js elsewhere on the page, pass
  `transformers.env.backends.onnx` as `ort` to the tuner and skip the second
  runtime download. `js/app.js` does this with a fallback.