Decision on data architecture/flow about common-voice

[mikehenrty]

The basic concept here is that metadata surrounding these voice clips will be kept separately from the voice clips themselves. You can find the voice clip using the metadata, but you should not be able to find the metadata given a voice clip. We will store the clips in the filesystem, and store metadata (including voice file path) in the db.

Later, when we publish the voice data, we can include non-PII metadata with the audio clips.

[kdavis-mozilla]

Just to solidify things a bit, there are two classes of metadata:

• PII - For example an email address
• Non-PII - For example gender

We do not want it to be the case that

You can find the voice clip using the metadata

when the metadata is PII metadata. This should be true internally and externally.

However, internally and externally we do want to be able to find voice clips that have a particular set of non-PII metadata, e.g. all voice clips that fit a particular demographic profile.

[mikehenrty]

We'll need to do this in M3, after we have designed overall architecture.

[geroter]

First draft of basic architecture is here:
https://docs.google.com/drawings/d/12FasUel5NH6poKLcJaxlYr64U6N6YEsyNs-wZJx_y04/edit

[mikehenrty]

The doc linked by George will be the living documentation for this. Closing.