Which neural network should I use to transform the pixels of a video overtime?


I want to train a network with video data and have it transform pixel values overtime on an input video. This is for an art project and does not need to be super elaborate, but the videos I want to render out of this might be big in resolution and frame count.

Which neural network would be appropriate for this task? I think I'm looking for a convolutional network (but I'm not too sure of that either). Which framework could easily allow me to do this?

Now, I'm no proper programmer, but self-learned on the go. I know some Javascript, but rather would like to learn more Python. Ideally, the easier and simpler the better though: I would be perfectly happy with something like "Uber Ludwig" (except maybe that it's from Uber).


Posted 2019-12-10T23:09:50.017

Reputation: 21

Use PyTorch if performance is not an issue and you are experiment with novel architectures. Use Tensorflow for super fast implementations and well known architectures. – DuttaA – 2019-12-11T05:42:07.323

Hi and welcome to this community. Could you please specify how you want to transform the pixels of the video? What's your ultimate goal? – nbro – 2019-12-11T13:58:34.683

regarding this project: i want to train a network with a certain type of footage so it develops some kind of abstract idea of what visually happens there over time. in a second step, i then want to apply this 'abstract idea' to other imagery (so: generate new images!), either stills or video. – user3647558 – 2019-12-11T22:30:14.663

that said, i spend quite some time today looking at ludwig - which seems super cool, only it does not seem to be able to output images: https://uber.github.io/ludwig/user_guide/#image-output-features-and-decoders

– user3647558 – 2019-12-11T23:58:18.653

No answers