Validating meaningless sentences in the Sentence Collector?

Common Voice
#1

Hi all,

in the french corpus of the Sentence Collector, I’m seeing recently quite a bunch of grammatically correct but completly meaningless sentences (like “I’m driving my pizza with an elephant on my cheese”). What are we supposed to do with those ?

Regards,

Cedric

Discussion of new guidelines for uploaded sentence validation
#2

It’s fine, we don’t expect anything that has any sense as long as it’s readable and french

#4

I came to the forum looking to see if anybody else had already reported this.

To give a concrete example: The architect declares to be acting on the psychoanalysts by touching sixty-six ratchets. (“L’architecte déclare agir sur les psychanalistes en touchant soixante-dix crécelles.”)

I strongly believe such sentences should be excluded for the following reasons:

  1. They confuse speakers, leading to lower quality recordings, as expressed above by @Michael_Maggs.
  2. They lower the quality of the corpus from a machine learning point of view, because:
    • The distribution of sentences contained in the corpus will no longer be representative of real spoken sentences, both in terms of grammatical structure (which is very repetitive for auto-generated sentences) and in terms of real-world word usage.
    • Many algorithms may be relying internally on a language model, which will get confused when it encounters nonsensical sentences.
2 Likes
Discussion of new guidelines for uploaded sentence validation
#5

Where do these come from? It sounds like someone wrote a script to insert random words into boilerplate sentences, just to increase the quantity.

#6

Well, if we are starting to talk about automatically generated sentences, that’s another question. From the ~5000 validations I did on sentence collector, I did not saw such kind of sentence. Maybe someone is messing with us ?

#7

I assume someone with DB access will be able to see who added all these French sentences. My guess is that they used either a set of templates or a PCFG-based generator, since most of the sentences are grammatically correct.

#8

As Gregor already said on Slack, anyone is allowed to read the data source:

https://kinto.mozvoice.org/v1/buckets/App/collections/Sentences_Meta_fr/records

Once you have identified the username, you can filter by appending ?username=foo to the URL.

#9

Thanks for that Michael. After a quick investigation, seems like it comes from user bf5man, and he added in the source field:“These are my own based on an open source tool.”

Regards,

Cedric

#10

it’d be great if we reach out to him so we can adjust contributions

#11

I’ve added this to the list at Discussion of new guidelines for uploaded sentence validation as an example of the type of sentence to be rejected.

#12

Hi all,

I’m now rejecting spoken versions of the meaningless sentences. It is a pity as we are wasting the time of these volunteers. How could we remove these sentences from bf5man ?

Cedric

#13

I can delete those, but I won’t get to before Wednesday evening earliest.

#14

That’s great thanks !

#15

This is done now. I’ve also added a script for me to run this, in case this should happen in the future again.