1. Introduction
People working on model training often open-source models, share training code, or introduce new architectures. There's much less public discussion of how the data is actually acquired and processed. To me, data is more than preparation for training. For TTS in particular, its scale, quality, and diversity have a major influence on how the model ultimately performs.
Acquiring speech data involves more steps than acquiring text. For LLMs, we can collect large amounts of text from websites, documents, and existing corpora, then clean and filter it. With TTS, finding a large collection of audio links doesn't mean we have usable data. We still need to check whether the resources are accessible, whether they can be downloaded reliably, and whether the files are complete and suitable for downstream processing.
Among the tools and open-source projects I've explored, I haven't found a solution that covers the full workflow of discovery, verification, and acquisition. Lately, I've been building my own speech data engine to bring these steps together.
I can't open-source the code yet, but I'd like to share the problems I've run into, the approaches I've tried, and the tradeoffs behind some of the design decisions. There's still plenty to improve. This post is a record of where the work stands, and I hope it's useful to others facing similar problems.
2. Data Discovery
When building a large speech dataset, the obvious first step is to look for more sources. But I've gradually realized that the number of sources doesn't tell us how much content we're actually covering. Before looking for another website, I wanted to understand how much content the sources we've already found can actually provide.
Speech data is scattered across the internet, but much of it is already organized. Podcast directories, video channels, film and TV databases, and audiobook catalogs give us ways to find content and see how it is connected. I prefer to start with those existing indexes instead of searching for individual audio files.
Different types of data have different starting points.
| Data Type | Common Sources | Discovery Method |
|---|---|---|
| Podcasts | Apple Podcasts, gPodder | Find RSS feeds through show directories, then extract the audio URLs for individual episodes. |
| YouTube | Channels, playlists | Retrieve video and channel metadata through the YouTube Data API. |
| Film and TV | TVmaze, media catalogs | Use title, season, and episode information to find matching media resources. |
| Audiobooks | LibriVox, Project Gutenberg | Discover content through APIs, RSS, or catalog indexes. Locate text and audio separately. |
A similar approach appears in the construction of speech datasets such as GigaSpeech and YODAS. GigaSpeech brings together audiobooks, podcasts, and YouTube content, while YODAS also uses channel relationships to expand discovery. What I take from these projects is the value of working with the way content is already organized, rather than relying entirely on keyword searches.
In my own system, I built a Source Library to manage podcast sources, RSS URLs, and the relationships between them. Film and TV resources use a separate catalog, organized by title, season, episode, and media version. This makes it easier to expand discovery while keeping track of where each candidate resource came from.
From RSS to Audio URLs
For podcasts, RSS is one of my main entry points.
An RSS feed typically stores show and episode information in XML. Each entry can include a title, publication date, GUID, and an enclosure URL pointing to the audio file. Once I have a show's RSS URL, I can enumerate the audio links it provides without visiting every episode page.
I organize these records into a Source Inventory, which stores each candidate audio URL and its source before passing it on for verification.
But finding an RSS feed doesn't mean you've found all of its audio.
A Coverage Gap in the RSS Pipeline
While checking 200 RSS feeds, I found a problem that would have been easy to overlook.
The returned RSS documents contained about 71,000 episode entries, but the old pipeline imposed a limit of 99 items per feed. As a result, it could read only about 20,000 entries. Many of those episodes weren't hard to find. They were already in the RSS documents we'd received, but never made it into our candidate inventory.
That made me realize that before looking for more sources, I needed to check whether I was fully reading the ones I already had. Data missed during discovery won't be recovered automatically at the download stage.
I later changed the RSS parser to read the complete returned document, within the available resource budget. I also kept raw snapshots, processing progress, and failure records so I could inspect and resume the work later.
This still doesn't solve the problem of missing historical content. Some shows have older episodes that the publisher no longer includes in the current feed. Parsing the full document fixes omissions in my own pipeline; it can't recover content the source hasn't provided.
Provenance Should Survive Deduplication
As discovery expanded, I found that different sources can point to the same resource.
For example, the same podcast might be listed in both Apple Podcasts and gPodder. Reading both directories can lead to discovering the same RSS feed, and the same batch of audio URLs, more than once.
Simply deleting duplicate records makes the inventory smaller, but it can also remove useful information about where those records came from.
In the Source Library, I merge duplicate feeds while retaining the directories, shows, and episodes associated with them. These records preserve the data's provenance. They let me trace resources back to their sources and provide useful leads for later reviews of data-use permissions.
There are also different levels of deduplication. Discovery can identify duplicate feeds and audio URLs. Verification can then merge records that resolve to the same URL after redirects. Whether two different URLs contain the same audio still needs to be checked after the files have been acquired.
For me, deduplication is about reducing redundant work while preserving the relationship between the data and its original sources. By the end of this stage, I have an inventory of candidate audio URLs with traceable origins.
3. Data Verification
After finding a large number of audio URLs through RSS and other content directories, I needed to check how many of these links still worked.
The reality was more complicated than I'd expected. Some links were dead, some went through several redirects, and others returned a successful HTTP status code but served an HTML page instead of audio.
The most direct check would be to download each file and try to decode it. With millions of candidate URLs, though, that would be expensive in both bandwidth and storage. I separated pre-download verification into its own stage, which I call Locator. Its job is to identify resources worth acquiring at a relatively low cost. It doesn't determine whether the audio will ultimately be suitable for training.
3.1 Lightweight Verification
I started by asking whether I could check a link without downloading the entire file.
I start with an HTTP HEAD request and inspect the status code, Content-Type, Content-Length, and other response metadata. If that provides enough information to make a decision, there's no need to read the file itself.
In practice, not every server handles HEAD correctly. Some leave out file metadata; others reject the request altogether. A file extension or response type alone also doesn't always tell me whether the content is actually audio.
When HEAD doesn't provide enough information, I try a Range GET request with Range: bytes=0-4095. I retain up to 4 KiB from the start of the file to identify common audio formats. This can rule out some invalid resources without a full download. The 4 KiB limit applies only to the sample retained for format detection, not to the request's total network traffic.
Not every failed request is worth following up. If HEAD returns 404 or 410, or times out, the current pipeline records that outcome without trying GET. A timeout doesn't necessarily mean the resource is dead. It only means that this attempt couldn't complete verification.
My goal is to get reliable enough information within a limited request budget, rather than verify every link at any cost.
3.2 Scaling Verification
Once verification grows from a few thousand URLs to millions, the cost of an individual request is only part of the problem. How network resources are allocated matters just as much.
Increasing concurrency seems like an obvious optimization. But in my tests, higher throughput didn't necessarily mean more URLs passed verification.
In one comparison using the same set of 2,000 candidate URLs, I tested concurrency settings of 16 and 32. Raising concurrency to 32 increased throughput by 50.3%, but timeouts rose from 10 to 15, and four fewer URLs passed verification.
One experiment with such a small difference doesn't establish that 16 is better than 32. It did remind me that throughput shouldn't be the only metric. Since the purpose of verification is to give the downloader reliable candidates, I kept concurrency at 16 under the operating conditions at the time.
Alongside the global concurrency limit, I also limit requests to each host so that work doesn't pile up on a single server. Each request has an explicit timeout and retry budget. Verification results are saved periodically, so an interrupted run can resume without starting over.
None of these choices is particularly complicated. At this scale, though, I think they matter more than simply pushing request rates higher.
3.3 Verification Records and Deduplication
I also found that a simple "available" or "unavailable" flag wasn't enough.
A 404 and a network timeout can both prevent a URL from passing verification, but they describe different problems. I keep the request method, response details, and failure reason so I can distinguish clearly unavailable resources from those I couldn't confirm on that attempt.
Podcast audio URLs also frequently redirect to a CDN. I record both the original asset_url and the resolved final_url. This preserves the source trail while allowing me to merge duplicate records that point to the same final address. URL deduplication still doesn't establish that the audio content is unique; content deduplication needs to happen after downloading.
By the end of this stage, I have candidate audio URLs that have passed preliminary checks, with their source information and verification records intact. They're ready to be handed off for acquisition.
4. Data Acquisition and Delivery
Discovery and verification leave us with a set of candidate audio URLs. Acquiring the actual files still requires different strategies for different types of data.
4.1 Podcasts and Direct Audio Downloads
Podcast acquisition is relatively straightforward. Audio URLs verified by Locator can be passed to a downstream downloader to retrieve the complete files.
The current system focuses on discovering, verifying, and handing off audio URLs. Full-file downloads happen downstream, so I won't go into that part here.
4.2 Film and TV Acquisition
Film and TV resources are more complicated. The same title can have several media versions with different resolutions, languages, and audio tracks. For TTS, I'm more interested in getting a suitable original-language audio track than in video quality. I first select candidate files using title, season, episode, and version information, then inspect the actual tracks.
The bigger challenge, though, is acquiring large numbers of media files reliably and efficiently.
Acquisition Strategy
I initially tried downloading media through BitTorrent with qBittorrent. Download speed and completion time depended on resource availability, seeders, and network conditions. For older or less popular titles in particular, completion times were hard to predict.
I later changed the strategy to check first whether a remote server already has the target resource cached. If a file is cached and I have the appropriate usage rights, I can download it directly through a temporary HTTPS URL, reducing the time spent waiting for acquisition through BitTorrent.
This doesn't necessarily make an individual file download faster, but it reduces uncertainty for some downloads. What I can acquire still depends on the remote cache's coverage.
Audio Extraction
Once I have the media file, I use FFprobe to inspect its audio tracks, then extract the target track with FFmpeg's -c:a copy, preserving the original encoding where possible and avoiding unnecessary recompression.
The current workflow still requires downloading the entire video file before extracting audio locally. Even when I only need the audio, I have to pay the bandwidth cost of transferring the video. Getting separate audio tracks directly from authorized media sources could reduce both transfer and storage costs in the future.
4.3 Integrity and Provenance
A completed download doesn't guarantee a valid file. Downstream processing still needs to check file sizes, checksums, and decoding. For film and TV resources, it also needs to confirm that the extracted audio track is the right one and is complete. These checks belong mainly to the downstream pipeline, rather than Locator.
The acquired files also need to stay linked to their original sources, metadata, and verification records. That makes it possible to trace a file back to the original show or title, even after multiple redirects, and helps with troubleshooting and source reviews later on.
4.4 Compliance and System Boundaries
A publicly accessible resource isn't automatically authorized for model training. Access, permission to download, and the right to use data for training are separate matters, each subject to the relevant licensing terms and legal requirements.
The current system primarily records sources and performs technical checks. Those records can support later reviews, but they don't establish training rights. Data-use permissions still need to be confirmed separately.
5. Lessons and Outlook
Looking back, my biggest takeaway is that the larger the scale, the more carefully you have to balance efficiency, cost, and reliability. More sources don't necessarily mean better coverage, and higher concurrency doesn't necessarily mean more URLs pass verification. Making the most of existing sources and reducing uncertainty in acquisition matter just as much as adding more resources.
I've now run verification at a scale of tens of millions of audio URLs, but that doesn't translate into an equivalent amount of usable speech data. Acquisition is still limited by missing RSS history, dead links, and cache coverage for film and TV resources. Even successfully downloaded files need further processing and filtering before they meet training requirements. I'd like to measure the actual output of each stage more systematically, so I can identify the main bottlenecks in coverage and acquisition efficiency.
I'm also interested in whether this acquisition workflow could be useful more broadly. Much as the DeepSeek Search / Fetch API packages search and content retrieval as services, speech data discovery, verification, and acquisition could perhaps be offered in a similar way. Developers with the appropriate data-use rights wouldn't have to rebuild all of that infrastructure themselves. It's an idea I want to explore further.