Case weights with sequence to sequence models?


I've got a bunch of variable-length sequences. They're basically vectors of zeros with the occasional one. Event rate is maybe 1%.

I want to build a model that will take sequences that I get in the wild -- same form and event rate -- and classify their inputs into zeros or ones.

Since the label is so rare, these models will struggle to classify. But since the "batch" is the sequence, applying weights to the batch doesn't make any sense. Most of the batch's labels will be zeros, because most of the sequences are zeros.

I had been planning on using a combination of LSTMs and 1D convs. Is there some way to upweight certain parts of a vector-valued label?

And if it exists, is it implemented in tensorflow?

The simplest solution would be to cut out traning examples that highlight the label: spans of the sequence with a higher event rate. But this messes up the natural context that the label is found in, and I'll only do this if I don't think of a better option.


Posted 2020-10-07T19:51:54.363

Reputation: 479

No answers