Chroma transcription metrics

[chf2117]

Implements and tests #197

[craffel]

Thanks Caleb. Apart from PEP8 stuff, you'll also need to add separate evaluation for chroma metrics to mir_eval.transcription.evaluate, e.g. forcing the value of chroma in the kwargs dict to be True, then False, so that both are evaluated (right @justinsalamon)?

Do we have any implementations to test this against? It's a small enough change that I am ok with not testing it against something existing, though we should be very certain in its correctness then :)

[justinsalamon]

Thanks Caleb. Apart from PEP8 stuff, you'll also need to add separate evaluation for chroma metrics to mir_eval.transcription.evaluate, e.g. forcing the value of chroma in the kwargs dict to be True, then False, so that both are evaluated (right @justinsalamon)?

yes

Do we have any implementations to test this against?

yes, see here.

[chf2117]

yes, see here.

The music-ir link is dead for me. Do you know of any others?

[justinsalamon]

The music-ir link is dead for me. Do you know of any others?

The MIREX website is temporarily down, my understanding is that they're working on it. I'll check if I have the result files stored locally somewhere, otherwise we'll have to wait till it's back up.

[craffel]

Now that the MIREX website is back up, can we move forward on this?

[chf2117]

Yes, I'll try to wrap it up in the next couple days

[chf2117]

I'm a bit confused about comparing to MIREX results. The existing output*.json files don't match the differences from this comment, so I'm not sure what to look for when comparing the results.

[craffel]

I'm a bit confused about comparing to MIREX results. The existing output*.json files don't match the differences from this comment, so I'm not sure what to look for when comparing the results.

@justinsalamon can you provide some guidance here?

[bmcfee]

Bumping this one, since it seems like everyone disappeared after summer?

[TGabor]

Is it still planned to integrate this functionality and does anybody know which points are still open?

[craffel]

This needs to be picked up again. I don't think anyone is planning on doing so.