What's the most suitable format to store huge number of neurons states?



I'd like to know which common file format is more efficient in terms of simplicity and storage space for storing the state of artificial neural network.

I'm not talking about memory storage, but file storage, so the data can be loaded later on.

My first guess would be XML, but having millions of connections and weights would generate huge amount of data. Another thing would be to dump object instances into binary file using some export/serialize functions, but the disadvantage is that the file isn't common and it's language specific.

Are there any common file format standards which can be used for exporting huge artificial neural network into the file to be loaded by another program? If so, which one.


Posted 2016-08-03T12:02:55.280

Reputation: 9 163

Question was closed 2016-08-13T00:12:44.823



One option is NeuroML, one of the goals of which is:

To facilitate the exchange of complex neuronal models between researchers, allowing for greater transparency and accessibility of models

In general, the matrices associated with large neural network models are likely to be sparse. Hence a 'homebrew' alternative to the above would be to use something like the Matrix Market format.


Posted 2016-08-03T12:02:55.280

Reputation: 6 685


In general purpose for this kind of applications, One can use databases such as sqlite, mysql, mssql etc. It simplifies read / write operations, allows for a common language to interact with different databases from different vendors and platforms.

Orhan G. Hafif

Posted 2016-08-03T12:02:55.280

Reputation: 109