Add a benchmark for dask.array.block comparing it to numpy and straight copy#18
Open
Add a benchmark for dask.array.block comparing it to numpy and straight copy#18
Conversation
Member
|
Is this ready to merge? |
hmaarrfk
commented
Oct 4, 2018
| self.da_block = da.block(self.block) | ||
| self.da_concatenate = da.concatenate(self.arr_list) | ||
| if mode.startswith('block optimized'): | ||
| self.da_block.dask, _ = fuse_linear(self.da_block.dask) |
Contributor
Author
There was a problem hiding this comment.
@jakirkham is there a better way to optimize the graph here?
Member
There was a problem hiding this comment.
This is a pretty typical strategy for optimizing. Does a few different things for Dask Arrays in particular.
Contributor
Author
There was a problem hiding this comment.
Thanks. I would like to get a
- Typical dask
user(i.e. no optimization called) - Call to block with recommended optimizations to the graph.
- Calling persist with the above combos.
Member
There was a problem hiding this comment.
Calling persist will trigger the standard Dask Array graph optimizations in either case.
Contributor
Author
There was a problem hiding this comment.
Got it, I'll remove that "mode".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@mrocklin I guess it wasn't as slow as I remember :S.
Dask is doing better than numpy, at least for now 👯♀️