What is a midstate? How does using a midstate speed up hashing?

4

1

This defines "midstate" as the "precomputed hash state after hashing the first half of the data."

I've heard "midstate" used in the context of speeding up hashing. How does using a midstate speed up hashing?

Geremia

Posted 2015-03-23T07:14:53.333

Reputation: 3 965

Possible duplicate of http://bitcoin.stackexchange.com/q/5034/18196.

– morsecoder – 2015-03-23T18:44:25.023

Answers

5

People often talk about SHA256 like it's a single operation, but it isn't. Rather, the input is broken up into 64-byte chunks, and then each chunk is put into a compression function. The state of the hash partway through hashing something does not depend on future parts of the data. Since the nonce is found in the second chunk, changing the nonce doesn't change the the state of the hash function after hashing the first chunk.

This is useful in other applications. For example, if you're hashing a large file, it means that you can hash it without loading the entire file into memory at once or loading it from disk multiple times.

The practical consequence of this for Bitcoin miners is that 99.99999998% of the time, you only need 2 iterations of the SHA256 compression function to check a possible block solution, instead of 3. In other words, you can mine 50% faster. Of course, since everyone else does this, that just means that the difficulty is higher for everyone.

midstate is literally the state midway through hashing.

See also: Did Satoshi intend to allow midstate computation to speed up hashing?

Nick ODell

Posted 2015-03-23T07:14:53.333

Reputation: 27 521

I thought it was broken up into 32 bit chunks. At least that's how I interpret sha256_generic.c.

– Geremia – 2015-05-28T03:54:37.243

2@Geremia It's split into 64 byte chunks, then copied, 32 bits at a time, into the message schedule array. 16 times 32 divided by 8 equals 64. – Nick ODell – 2015-05-28T09:19:42.497

But we're talking about double SHA-256 (SHA-256d) here. How does that factor into things? – Geremia – 2015-06-18T04:40:15.780

SHA-256d (b) on input data is: b=SHA(a=SHA(data)). Thus is a the "midstate"? – Geremia – 2015-06-18T04:41:15.083

2@Geremia a isn't the midstate itself. You can use the midstate to more efficiently compute a because the first half of data doesn't change. b must be done from scratch (although because a is only 32 bytes long, computing b only involves a single SHA-256 block). – Christopher Gurnee – 2015-06-19T22:32:24.103

1related discussion – Nick ODell – 2015-06-19T23:19:37.473

0

Unless I'm corrected otherwise, I'll offer this analogy:

There's two closets filled with files. You've searched through Closet 1 already (personally, or work done by another team) and want to save yourself the time of searching the same closet again the next day. So you (cryptographically) seal the door so as to be certain the state remains (no one alters files). This allows you to concentrate your efforts on the second closet.

Again, I am on the fence between whether to comment or answer but I've chosen the latter

Wizard Of Ozzie

Posted 2015-03-23T07:14:53.333

Reputation: 4 920

This doesn't seem like a very good analogy. You're not trying to narrow the search space, you're trying to speed up individual attempts. – Nick ODell – 2015-03-27T04:57:15.210