## Is there a historical timeseries of market depth & transactions available?

18

9

Has anyone stored the historical market depth & tick data from MtGox? Is it available publically? Would be useful if you were making a bot.

Related, there is a BitcoinCharts API which provides some timeseries information http://bitcoincharts.com/about/markets-api/

– ACyclic – 2011-10-19T00:30:27.647

I think that's the main problem. It really is too much data. – nmat – 2011-10-26T14:22:34.897

1I grabbed a simple script I had and I got it fetching market depth every 5 seconds and saving the differences (I should use websockets, but I didn't have a script ready). After 2 days I have ~5MB of data so maybe this is not that hard to do. The data could be even saved in a more compressed format (I am just dumping what MtGox gives me). It's just a matter of setting up a server running 24/7 with a script like this. – nmat – 2011-10-28T14:49:20.293

12

Current market depth available at

All trades which happened in the last 24 hours available at

Retrieves 100 trades that happend after specified transaction

A script is required to retrive all historical data, like the one posted by mizerydearia. The idea is to cycle download request while incrementing "transaction tid" aka "lasttrade" to download all data in steps.

#!/bin/bash
outputfile=data
while test -n "$lasttrade";do wget --quiet https://mtgox.com/code/data/getTrades.php?since=$lasttrade
if test "cat getTrades.php\?since\=$lasttrade" != "[]";then cat getTrades.php\?since\=* >>$outputfile;
echo >> $outputfile; lasttrade=cat getTrades.php\?since\=$lasttrade|sed "s/,/\n/g"|grep tid|tail -n 1|cut -d ":" -f 2|cut -d "\"" -f 2;
lasttradets=cat getTrades.php\?since\=*|sed "s/,/\n/g"|grep date|tail -n 1|cut -d ":" -f 2;
echo "$(date -d @$lasttradets)  tid \$lasttrade"
else
echo Finished
fi
done


See MtGox - API for more http API calls.

Any idea how big this file is today? I'm at about 3.5 gb right now and still going. – jordan.baucke – 2013-02-22T16:01:50.240

@jordan.baucke what size did you get? – kirill_igum – 2013-12-08T03:33:59.733

I'm running a similar script -- it's currently up to 10/20/2011 @ ~ 155 MB uncompressed -- I'll let you know when it completes, and see if I can dump it up online somewhere. – BrainSlugs83 – 2014-01-13T04:32:40.500

Well, I got to 3/11/2013 -- and it's only about ~670 MB so far. So there must be a bug in the above script. Probably related to the fact that it's parsing the JSON with Regex. Maybe an error is being thrown every so often, and it's starting over at the error code (using it as a TID) or something? – BrainSlugs83 – 2014-01-13T09:46:53.900

I got up to date (1/14/2014) this morning -- completely uncompressed, the JSON is about 1.33 GB -- compressed in a regular .zip file, it's about 160 MB. I used the URL and the approach described above (slightly different implementation), and although it does not mention USD in the URL, the results appear to be in USD only (maybe that happened when they changed the API? This could also be the reason for the much smaller file size). At any rate, I've posted my dump here: http://bit.ly/1dMzqhJ

– BrainSlugs83 – 2014-01-14T12:15:58.873

3

To get ALL mtgoxUSD trades since the start you can use the following url:

WARNING. This generates a 49mb csv at the moment and can take some time, so please only follow the link if you actually need the data.

i think this is daily not tick data – kirill_igum – 2013-12-08T03:33:09.667

2

Just noting here that MtGox tid's are not included in the data you get from bitcoincharts in that csv call API, and they switched from numerical indexing to microsecond timestamping a while back. In trying to maintain a db with integrity, you need the unique tid data... The history file is now over 100MB also.

1

# Original MtGox Shared Dataset

Once upon a time there was a Google BigQuery dataset that was updated often with MtGox's official trade record. That's no longer being published because... ask someone who should explain it.

### The BitTorrent Version

If you'd rather not BigQuery, but you still want that data, there's a torrent, here's the full-metal-magnet.

magnet:?xt=urn:btih:c87cd62f9484993f0153a47fc51f06fb43ed71ee&dn=931531665892-mtgox-trades.csv&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.ccc.de%3A80&tr=udp%3A%2F%2Ftracker.istole.it%3A80

## The MtGox Leak Database

In 2014 the Magic The Gathering Online eXchange (MtGox) server was compromised. Source code, user account details and trade data were made public.

For academic research the trade data is useful, here's a magnet to the original "public service release" file, MtGox2014Leak.zip. BEWARE: The TibanneBackOffice.zip file contains a trojan

magnet:?xt=urn:btih:b6545ecc7db8d44c8cbc4e93989edf8221af75f5&dn=2014+Mt.+Gox+Leak&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80&tr=udp%3A%2F%2Ftracker.istole.it%3A6969&tr=udp%3A%2F%2Ftracker.ccc.de%3A80&tr=udp%3A%2F%2Fopen.demonii.com%3A1337

Alternatively, cryptoanarchic.me have kindly provided an HTTP mirrored version of MtGox2014Leak.zip.

Those are the two data sources I'm using in my research.