Computing diffs in Unison Share contributions can be very slow

[gvolpe]

Hi all 👋🏽 ,

This may be a known internal problem, but I didn't see any open issue about it, so I thought I would share today's experience. I submitted a small contribution to base that changes the implementation of a function.

The message displayed says "Unison Share will never show a stale diff, and sometimes it takes a minute or two to compute...". In this case, it took about 36 minutes to compute the diff. I took a screenshot at the 30 min mark.

Not sure if this was an edge case, but may be worth updating the message to indicate it may take longer in large codebases?

[hojberg]

Hey @gvolpe

Thanks for reporting. We've seen these slow times as well and are working on improving them. Ideally this should take a couple of minutes at the most if not be instant.

cc @mitchellwrosen @aryairani @ChrisPenner

[ChrisPenner]

@hojberg I think it's probably going to take some significant work to get these times down unfortunately so I'd suggest changing the wording of the message to set expectations until we can improve it :)

[pchiusano]

I suspect if we avoid computing syntactic hashes unless the AST hash has changed, it will be quite fast (similar to the old diff which listed all transitive dependents whose hashes changed). Right now my understanding is we're deserializing all the terms in the project to compute their syntactic hash, then computing diffs based on the syntactic hashes, which is accurate, but for a big project like base it's slow.

Most PRs touch a small fraction of the names in a project, and it's only those names which have updated AST hashes need checking to see if their syntactic hash has changed or not. New names don't need to be syntactically hashed at all, just the updates.

[rlmark]

Hey @gvolpe! Hope all is well! Recently, the team made some big strides in the diff computing performance. I believe they landed at the end of July / start of August. I've personally observed that changes that would take many minutes now load in a few seconds, or less. Give it a try and let us know what you think!

[gvolpe]

That's great to hear! As soon as I get any contribution idea, I'll be happy to test it myself 😁

[ChrisPenner]

Some diffs are still much slower than we'd like, but will have yet another patch to further improve this coming soon, so while we've gone from hours to minutes, hopefully we can cut it down even more in the coming weeks :)